2024-03-14T08:48:11.732 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-14T08:48:11.733 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661 --verbose --owner scheduled_yuriw@teuthology --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} --name yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/orig.config.yaml 2024-03-14T08:48:11.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T08:48:11.895 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661 branch: wip-yuri-testing-2024-03-12-1240-reef description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} email: yweinste@redhat.com first_in_suite: false job_id: '7598661' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2024-03-12-1240-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2024-03-12-1240-reef sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 8503 sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri-testing-2024-03-12-1240-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAh63oA83n0cc4tnDfIENESbeLZfvffjLhjv3u9kdLTpFVdZjgr4jWfTSVyFijT5tRxuy1fSLG+09VqENeQ8GGI= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAatpitHNt+TFVbP3UicmJSYlTf30fDW9oPsOZKeenJamK3ncp0pbi6YADcnolpGBU9msxgnPutue05jBsV9C0s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_multifs_auth teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-13_19:31:43 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-14T08:48:11.895 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa; will attempt to use it 2024-03-14T08:48:11.896 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks 2024-03-14T08:48:11.896 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-14T08:48:11.898 INFO:teuthology.task.internal:Checking packages... 2024-03-14T08:48:11.926 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f465319e3c83d48f999325459bcdfd0346449c0' 2024-03-14T08:48:11.927 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-14T08:48:11.927 INFO:teuthology.packaging:ref: None 2024-03-14T08:48:11.927 INFO:teuthology.packaging:tag: None 2024-03-14T08:48:11.927 INFO:teuthology.packaging:branch: wip-yuri-testing-2024-03-12-1240-reef 2024-03-14T08:48:11.927 INFO:teuthology.packaging:sha1: 2f465319e3c83d48f999325459bcdfd0346449c0 2024-03-14T08:48:11.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2024-03-12-1240-reef 2024-03-14T08:48:12.069 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-825.g2f465319 2024-03-14T08:48:12.070 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-14T08:48:12.076 INFO:teuthology.task.internal:no buildpackages task found 2024-03-14T08:48:12.076 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-14T08:48:12.082 INFO:teuthology.task.internal:Saving configuration 2024-03-14T08:48:12.098 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-14T08:48:12.102 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T08:48:12.127 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 08:42:57.830150', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAh63oA83n0cc4tnDfIENESbeLZfvffjLhjv3u9kdLTpFVdZjgr4jWfTSVyFijT5tRxuy1fSLG+09VqENeQ8GGI='} 2024-03-14T08:48:12.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 08:42:57.828363', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAatpitHNt+TFVbP3UicmJSYlTf30fDW9oPsOZKeenJamK3ncp0pbi6YADcnolpGBU9msxgnPutue05jBsV9C0s='} 2024-03-14T08:48:12.146 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-14T08:48:12.151 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-14T08:48:12.151 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-03-14T08:48:12.151 INFO:teuthology.run_tasks:Running task console_log... 2024-03-14T08:48:12.215 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f18ae6ec160>, signals=[15]) 2024-03-14T08:48:12.216 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-14T08:48:12.221 INFO:teuthology.task.internal:Opening connections... 2024-03-14T08:48:12.221 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T08:48:12.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T08:48:12.300 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T08:48:12.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T08:48:12.373 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-14T08:48:12.379 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-03-14T08:48:12.400 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-03-14T08:48:12.400 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:ID="rhel" 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="fedora" 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8.6" 2024-03-14T08:48:12.456 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T08:48:12.457 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T08:48:12.458 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-03-14T08:48:12.483 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-03-14T08:48:12.504 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-03-14T08:48:12.504 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.6" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T08:48:12.564 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T08:48:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T08:48:12.565 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-03-14T08:48:12.583 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-14T08:48:12.592 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-14T08:48:12.597 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-14T08:48:12.597 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-03-14T08:48:12.599 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-03-14T08:48:12.623 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-14T08:48:12.629 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-14T08:48:12.629 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-03-14T08:48:12.656 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-03-14T08:48:12.689 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-14T08:48:12.742 INFO:teuthology.run_tasks:Running task kernel... 2024-03-14T08:48:12.756 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-14T08:48:12.756 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-03-14T08:48:12.756 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-14T08:48:12.757 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T08:48:12.757 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-14T08:48:12.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T08:48:12.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T08:48:12.777 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-03-14T08:48:12.777 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-03-14T08:48:12.834 INFO:teuthology.orchestra.run.smithi154.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:12.834 INFO:teuthology.task.kernel:Running kernel on smithi154: 4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:12.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2024-03-14T08:48:12.836 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:12.836 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:12.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-03-14T08:48:14.327 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:14.327 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2024-03-14T08:48:14.471 INFO:teuthology.orchestra.run.smithi154.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:14.471 INFO:teuthology.orchestra.run.smithi154.stdout:Unable to read consumer identity 2024-03-14T08:48:14.627 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:14.627 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:14.627 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:14.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:14.714 INFO:teuthology.orchestra.run.smithi154.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:14.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:16.419 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-14T08:48:16.549 INFO:teuthology.orchestra.run.smithi154.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-03-14T08:48:17.178 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-03-14T08:48:18.427 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2024-03-14T08:48:20.787 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 274 kB/s | 24 kB 00:00 2024-03-14T08:48:21.531 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-03-14T08:48:21.531 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-03-14T08:48:21.607 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-03-14T08:48:21.608 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-03-14T08:48:21.608 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-03-14T08:48:21.762 DEBUG:teuthology.orchestra.run.smithi191:> echo no | sudo yum reinstall kernel || true 2024-03-14T08:48:22.041 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 434 kB/s | 24 kB 00:00 2024-03-14T08:48:22.096 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:22.097 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2024-03-14T08:48:22.331 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:22.331 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:22.331 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:22.642 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:02 ago on Thu 14 Mar 2024 08:48:20 AM UTC. 2024-03-14T08:48:22.716 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-14T08:48:22.723 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-14T08:48:22.736 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-03-14T08:48:22.736 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-03-14T08:48:22.770 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2024-03-14T08:48:22.820 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-03-14T08:48:22.820 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-03-14T08:48:22.820 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-03-14T08:48:22.821 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum reinstall -y kernel || true 2024-03-14T08:48:22.997 DEBUG:teuthology.orchestra.run.smithi154:> echo no | sudo yum reinstall kernel || true 2024-03-14T08:48:23.156 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:23.157 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2024-03-14T08:48:23.334 INFO:teuthology.orchestra.run.smithi154.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:23.334 INFO:teuthology.orchestra.run.smithi154.stdout:Unable to read consumer identity 2024-03-14T08:48:23.374 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:23.375 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:23.375 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:48:23.573 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:23.573 INFO:teuthology.orchestra.run.smithi154.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:23.573 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:23.677 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:03 ago on Thu 14 Mar 2024 08:48:20 AM UTC. 2024-03-14T08:48:23.740 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-14T08:48:23.740 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-14T08:48:23.784 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2024-03-14T08:48:23.831 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T08:48:23.880 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:02 ago on Thu 14 Mar 2024 08:48:21 AM UTC. 2024-03-14T08:48:23.968 INFO:teuthology.orchestra.run.smithi154.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-14T08:48:23.975 INFO:teuthology.orchestra.run.smithi154.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-14T08:48:23.998 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:23.999 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:23.999 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-03-14T08:48:23.999 DEBUG:teuthology.parallel:result is None 2024-03-14T08:48:24.048 INFO:teuthology.orchestra.run.smithi154.stderr:Error: No packages marked for reinstall. 2024-03-14T08:48:24.095 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum reinstall -y kernel || true 2024-03-14T08:48:24.430 INFO:teuthology.orchestra.run.smithi154.stdout:Updating Subscription Management repositories. 2024-03-14T08:48:24.431 INFO:teuthology.orchestra.run.smithi154.stdout:Unable to read consumer identity 2024-03-14T08:48:24.661 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:24.662 INFO:teuthology.orchestra.run.smithi154.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-14T08:48:24.662 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:48:24.965 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:03 ago on Thu 14 Mar 2024 08:48:21 AM UTC. 2024-03-14T08:48:25.027 INFO:teuthology.orchestra.run.smithi154.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-14T08:48:25.027 INFO:teuthology.orchestra.run.smithi154.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-14T08:48:25.071 INFO:teuthology.orchestra.run.smithi154.stderr:Error: No packages marked for reinstall. 2024-03-14T08:48:25.123 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-03-14T08:48:25.284 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:25.285 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-03-14T08:48:25.285 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-03-14T08:48:25.285 DEBUG:teuthology.parallel:result is None 2024-03-14T08:48:25.285 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-14T08:48:25.291 INFO:teuthology.task.internal:Creating test directory... 2024-03-14T08:48:25.292 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T08:48:25.294 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-14T08:48:25.314 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-14T08:48:25.321 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-14T08:48:25.326 INFO:teuthology.task.internal:Creating archive directory... 2024-03-14T08:48:25.327 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T08:48:25.351 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-14T08:48:25.390 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-14T08:48:25.396 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-14T08:48:25.397 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T08:48:25.425 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-14T08:48:25.459 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T08:48:25.460 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T08:48:25.475 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T08:48:25.476 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-14T08:48:25.478 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-14T08:48:25.541 INFO:teuthology.task.internal:Configuring sudo... 2024-03-14T08:48:25.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T08:48:25.544 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-14T08:48:25.574 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-14T08:48:25.584 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-14T08:48:25.584 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T08:48:25.611 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-14T08:48:25.632 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T08:48:25.707 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T08:48:25.779 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T08:48:25.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T08:48:25.864 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T08:48:25.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T08:48:25.931 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T08:48:25.978 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-14T08:48:26.034 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T08:48:26.121 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-14T08:48:26.164 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T08:48:26.164 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-14T08:48:26.232 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-03-14T08:48:26.235 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-03-14T08:48:26.279 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T08:48:26.319 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T08:48:26.769 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-14T08:48:26.777 INFO:teuthology.task.internal:Starting timer... 2024-03-14T08:48:26.777 INFO:teuthology.run_tasks:Running task pcp... 2024-03-14T08:48:26.787 INFO:teuthology.run_tasks:Running task selinux... 2024-03-14T08:48:26.796 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-03-14T08:48:26.797 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-03-14T08:48:26.874 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-03-14T08:48:26.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-03-14T08:48:26.926 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-03-14T08:48:26.927 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-14T08:48:26.927 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-03-14T08:48:26.951 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-03-14T08:48:26.951 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-03-14T08:48:26.976 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-03-14T08:48:26.976 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-03-14T08:48:26.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T08:48:27.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T08:48:27.020 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T08:48:27.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T08:48:27.049 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-14T08:48:27.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-03-14T08:48:27.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-03-14T08:48:27.117 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-14T08:48:27.127 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-14T08:48:27.133 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-14T08:48:27.262 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-14T08:48:27.278 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-14T08:48:27.279 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi154.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-14T08:57:42.162 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-03-14T08:57:42.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-14T08:57:42.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T08:57:42.258 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-03-14T08:57:42.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-03-14T08:57:42.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-14T08:57:42.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T08:57:42.404 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-03-14T08:57:42.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-14T08:57:42.472 INFO:teuthology.run_tasks:Running task clock... 2024-03-14T08:57:42.485 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-14T08:57:42.486 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T08:57:42.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T08:57:42.489 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-14T08:57:42.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T08:57:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T08:57:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T08:57:42.544 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-14T08:57:42.559 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-14T08:57:42.609 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-03-14T08:57:42.621 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-03-14T08:57:42.625 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-03-14T08:57:42.636 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T08:57:42.637 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-03-14T08:57:42.650 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T08:57:42.653 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-14T08:57:42.667 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-14T08:57:42.729 INFO:teuthology.orchestra.run.smithi154.stderr:bash: ntpq: command not found 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.733 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.747 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-03-14T08:57:42.750 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T08:57:42.751 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-03-14T08:57:42.751 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.751 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.751 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.751 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T08:57:42.752 INFO:teuthology.run_tasks:Running task install... 2024-03-14T08:57:42.760 DEBUG:teuthology.task.install:project ceph 2024-03-14T08:57:42.760 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'}} 2024-03-14T08:57:42.760 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'} 2024-03-14T08:57:42.760 INFO:teuthology.task.install:Using flavor: default 2024-03-14T08:57:42.766 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-14T08:57:42.766 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-14T08:57:42.767 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0', 'tag': None, 'wait_for_package': False} 2024-03-14T08:57:42.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f465319e3c83d48f999325459bcdfd0346449c0 2024-03-14T08:57:42.769 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0', 'tag': None, 'wait_for_package': False} 2024-03-14T08:57:42.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f465319e3c83d48f999325459bcdfd0346449c0 2024-03-14T08:57:43.024 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/ 2024-03-14T08:57:43.025 INFO:teuthology.task.install.rpm:Package version is 18.2.2-825.g2f465319 2024-03-14T08:57:43.031 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/ 2024-03-14T08:57:43.031 INFO:teuthology.task.install.rpm:Package version is 18.2.2-825.g2f465319 2024-03-14T08:57:43.140 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T08:57:43.140 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T08:57:43.141 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T08:57:43.170 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T08:57:43.170 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f465319e3c83d48f999325459bcdfd0346449c0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T08:57:43.237 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri-testing-2024-03-12-1240-reef/2f465319e3c83d48f999325459bcdfd0346449c0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-14T08:57:43.237 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T08:57:43.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-14T08:57:43.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T08:57:43.267 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-14T08:57:43.267 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f465319e3c83d48f999325459bcdfd0346449c0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-14T08:57:43.317 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T08:57:43.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-14T08:57:43.384 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-03-14T08:57:43.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-03-14T08:57:43.414 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-14T08:57:43.481 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-03-14T08:57:43.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-03-14T08:57:43.764 INFO:teuthology.orchestra.run.smithi154.stdout:Updating Subscription Management repositories. 2024-03-14T08:57:43.863 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2024-03-14T08:57:45.139 INFO:teuthology.orchestra.run.smithi154.stdout:61 files removed 2024-03-14T08:57:45.190 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T08:57:45.240 INFO:teuthology.orchestra.run.smithi191.stdout:61 files removed 2024-03-14T08:57:45.299 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-14T08:57:45.529 INFO:teuthology.orchestra.run.smithi154.stdout:Updating Subscription Management repositories. 2024-03-14T08:57:45.639 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2024-03-14T08:57:47.240 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 186 kB/s | 80 kB 00:00 2024-03-14T08:57:47.294 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 322 kB/s | 80 kB 00:00 2024-03-14T08:57:47.670 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-14T08:57:47.766 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 20 kB/s | 9.9 kB 00:00 2024-03-14T08:57:48.029 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-03-14T08:57:48.275 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2024-03-14T08:57:48.296 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-03-14T08:57:48.517 INFO:teuthology.orchestra.run.smithi154.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-03-14T08:57:49.265 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2024-03-14T08:57:50.118 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-03-14T08:57:52.873 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 388 kB/s | 24 kB 00:00 2024-03-14T08:57:53.634 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-14T08:57:53.676 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 345 kB/s | 24 kB 00:00 2024-03-14T08:57:55.095 INFO:teuthology.orchestra.run.smithi154.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 59 MB 00:01 2024-03-14T08:57:56.494 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 59 MB 00:01 2024-03-14T08:58:05.678 INFO:teuthology.orchestra.run.smithi154.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-14T08:58:07.812 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 66 MB 00:01 2024-03-14T08:58:08.746 INFO:teuthology.orchestra.run.smithi154.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 66 MB 00:01 2024-03-14T08:58:22.748 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-14T08:58:22.749 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-14T08:58:22.767 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-14T08:58:22.768 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-14T08:58:22.770 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-14T08:58:22.771 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-14T08:58:22.774 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-14T08:58:22.778 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-14T08:58:22.779 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-14T08:58:22.999 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-03-14T08:58:23.013 INFO:teuthology.orchestra.run.smithi191.stdout:=================================================================================================================================== 2024-03-14T08:58:23.013 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout:=================================================================================================================================== 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:18.2.2-825.g2f465319.el8 ceph 6.4 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:18.2.2-825.g2f465319.el8 ceph 5.4 M 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:18.2.2-825.g2f465319.el8 ceph 857 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-825.g2f465319.el8 ceph 165 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:18.2.2-825.g2f465319.el8 ceph 1.6 M 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 137 k 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 1.7 M 2024-03-14T08:58:23.014 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 7.4 M 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 49 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:18.2.2-825.g2f465319.el8 ceph 13 M 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:18.2.2-825.g2f465319.el8 ceph 47 M 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 263 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 170 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:18.2.2-825.g2f465319.el8 ceph 261 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 18 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:18.2.2-825.g2f465319.el8 ceph 31 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 779 k 2024-03-14T08:58:23.015 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:18.2.2-825.g2f465319.el8 ceph 139 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:18.2.2-825.g2f465319.el8 ceph 197 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:18.2.2-825.g2f465319.el8 ceph 372 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:18.2.2-825.g2f465319.el8 ceph 365 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:18.2.2-825.g2f465319.el8 ceph 107 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:18.2.2-825.g2f465319.el8 ceph 95 k 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.5 M 2024-03-14T08:58:23.016 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:18.2.2-825.g2f465319.el8 ceph 182 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.7 M 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.6 M 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-14T08:58:23.017 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:18.2.2-825.g2f465319.el8 ceph 22 M 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 23 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:18.2.2-825.g2f465319.el8 ceph 2.3 M 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 245 k 2024-03-14T08:58:23.018 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:18.2.2-825.g2f465319.el8 ceph 4.6 M 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:18.2.2-825.g2f465319.el8 ceph 18 M 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 15 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:18.2.2-825.g2f465319.el8 ceph 24 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:18.2.2-825.g2f465319.el8 ceph 142 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-14T08:58:23.019 INFO:teuthology.orchestra.run.smithi191.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:18.2.2-825.g2f465319.el8 ceph 474 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 6.5 M 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-14T08:58:23.020 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:18.2.2-825.g2f465319.el8 ceph 45 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:18.2.2-825.g2f465319.el8 ceph 125 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-14T08:58:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-14T08:58:23.022 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-14T08:58:23.023 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-14T08:58:23.024 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout:Enabling module streams: 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql 10 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout:=================================================================================================================================== 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout:Install 130 Packages 2024-03-14T08:58:23.025 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 16 Packages 2024-03-14T08:58:23.026 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:58:23.026 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 212 M 2024-03-14T08:58:23.026 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-03-14T08:58:23.204 INFO:teuthology.orchestra.run.smithi191.stdout:(1/146): ceph-18.2.2-825.g2f465319.el8.x86_64.r 54 kB/s | 6.4 kB 00:00 2024-03-14T08:58:23.404 INFO:teuthology.orchestra.run.smithi191.stdout:(2/146): ceph-fuse-18.2.2-825.g2f465319.el8.x86 4.2 MB/s | 857 kB 00:00 2024-03-14T08:58:23.463 INFO:teuthology.orchestra.run.smithi191.stdout:(3/146): ceph-immutable-object-cache-18.2.2-825 2.8 MB/s | 165 kB 00:00 2024-03-14T08:58:23.662 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-14T08:58:23.663 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-14T08:58:23.683 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-14T08:58:23.685 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-14T08:58:23.686 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-14T08:58:23.688 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-14T08:58:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-14T08:58:23.695 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-14T08:58:23.697 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-14T08:58:23.755 INFO:teuthology.orchestra.run.smithi191.stdout:(4/146): ceph-base-18.2.2-825.g2f465319.el8.x86 8.1 MB/s | 5.4 MB 00:00 2024-03-14T08:58:23.805 INFO:teuthology.orchestra.run.smithi191.stdout:(5/146): ceph-mds-18.2.2-825.g2f465319.el8.x86_ 6.6 MB/s | 2.3 MB 00:00 2024-03-14T08:58:23.918 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-03-14T08:58:23.922 INFO:teuthology.orchestra.run.smithi191.stdout:(6/146): ceph-mgr-18.2.2-825.g2f465319.el8.x86_ 9.3 MB/s | 1.6 MB 00:00 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout:=================================================================================================================================== 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout:=================================================================================================================================== 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:18.2.2-825.g2f465319.el8 ceph 6.4 k 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:18.2.2-825.g2f465319.el8 ceph 5.4 M 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:18.2.2-825.g2f465319.el8 ceph 857 k 2024-03-14T08:58:23.933 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-825.g2f465319.el8 ceph 165 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:18.2.2-825.g2f465319.el8 ceph 1.6 M 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 137 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 1.7 M 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 7.4 M 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 49 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:18.2.2-825.g2f465319.el8 ceph 13 M 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:18.2.2-825.g2f465319.el8 ceph 47 M 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 263 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 170 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:18.2.2-825.g2f465319.el8 ceph 261 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 18 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-14T08:58:23.934 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:18.2.2-825.g2f465319.el8 ceph 31 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 779 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:18.2.2-825.g2f465319.el8 ceph 139 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:18.2.2-825.g2f465319.el8 ceph 197 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:18.2.2-825.g2f465319.el8 ceph 372 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:18.2.2-825.g2f465319.el8 ceph 365 k 2024-03-14T08:58:23.935 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:18.2.2-825.g2f465319.el8 ceph 107 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:18.2.2-825.g2f465319.el8 ceph 95 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.5 M 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:18.2.2-825.g2f465319.el8 ceph 182 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.7 M 2024-03-14T08:58:23.936 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:18.2.2-825.g2f465319.el8 ceph 3.6 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:18.2.2-825.g2f465319.el8 ceph 22 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 23 k 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:18.2.2-825.g2f465319.el8 ceph 2.3 M 2024-03-14T08:58:23.937 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 245 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:18.2.2-825.g2f465319.el8 ceph 4.6 M 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:18.2.2-825.g2f465319.el8 ceph 18 M 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:18.2.2-825.g2f465319.el8 ceph-noarch 15 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:18.2.2-825.g2f465319.el8 ceph 24 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:18.2.2-825.g2f465319.el8 ceph 142 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-14T08:58:23.938 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:18.2.2-825.g2f465319.el8 ceph 474 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:18.2.2-825.g2f465319.el8 ceph 6.5 M 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-14T08:58:23.939 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:18.2.2-825.g2f465319.el8 ceph 45 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:18.2.2-825.g2f465319.el8 ceph 125 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-14T08:58:23.940 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-14T08:58:23.941 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-14T08:58:23.942 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-14T08:58:23.943 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:Enabling module streams: 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql 10 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:=================================================================================================================================== 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:Install 130 Packages 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 16 Packages 2024-03-14T08:58:23.944 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:58:23.945 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 212 M 2024-03-14T08:58:23.945 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-03-14T08:58:24.406 INFO:teuthology.orchestra.run.smithi191.stdout:(7/146): ceph-mon-18.2.2-825.g2f465319.el8.x86_ 7.7 MB/s | 4.6 MB 00:00 2024-03-14T08:58:24.529 INFO:teuthology.orchestra.run.smithi154.stdout:(1/146): ceph-18.2.2-825.g2f465319.el8.x86_64.r 28 kB/s | 6.4 kB 00:00 2024-03-14T08:58:25.362 INFO:teuthology.orchestra.run.smithi154.stdout:(2/146): ceph-fuse-18.2.2-825.g2f465319.el8.x86 1.0 MB/s | 857 kB 00:00 2024-03-14T08:58:25.446 INFO:teuthology.orchestra.run.smithi154.stdout:(3/146): ceph-immutable-object-cache-18.2.2-825 1.9 MB/s | 165 kB 00:00 2024-03-14T08:58:25.464 INFO:teuthology.orchestra.run.smithi191.stdout:(8/146): ceph-osd-18.2.2-825.g2f465319.el8.x86_ 12 MB/s | 18 MB 00:01 2024-03-14T08:58:25.506 INFO:teuthology.orchestra.run.smithi191.stdout:(9/146): ceph-selinux-18.2.2-825.g2f465319.el8. 585 kB/s | 24 kB 00:00 2024-03-14T08:58:26.474 INFO:teuthology.orchestra.run.smithi154.stdout:(4/146): ceph-mds-18.2.2-825.g2f465319.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2024-03-14T08:58:26.532 INFO:teuthology.orchestra.run.smithi191.stdout:(10/146): ceph-common-18.2.2-825.g2f465319.el8. 6.3 MB/s | 22 MB 00:03 2024-03-14T08:58:26.615 INFO:teuthology.orchestra.run.smithi191.stdout:(11/146): cephfs-mirror-18.2.2-825.g2f465319.el 3.1 MB/s | 261 kB 00:00 2024-03-14T08:58:26.758 INFO:teuthology.orchestra.run.smithi191.stdout:(12/146): ceph-radosgw-18.2.2-825.g2f465319.el8 5.6 MB/s | 13 MB 00:02 2024-03-14T08:58:26.783 INFO:teuthology.orchestra.run.smithi191.stdout:(13/146): libcephfs-devel-18.2.2-825.g2f465319. 185 kB/s | 31 kB 00:00 2024-03-14T08:58:26.796 INFO:teuthology.orchestra.run.smithi154.stdout:(5/146): ceph-mgr-18.2.2-825.g2f465319.el8.x86_ 4.8 MB/s | 1.6 MB 00:00 2024-03-14T08:58:26.891 INFO:teuthology.orchestra.run.smithi191.stdout:(14/146): libcephsqlite-18.2.2-825.g2f465319.el 1.3 MB/s | 142 kB 00:00 2024-03-14T08:58:26.992 INFO:teuthology.orchestra.run.smithi191.stdout:(15/146): libcephfs2-18.2.2-825.g2f465319.el8.x 3.3 MB/s | 779 kB 00:00 2024-03-14T08:58:27.018 INFO:teuthology.orchestra.run.smithi191.stdout:(16/146): librados-devel-18.2.2-825.g2f465319.e 1.1 MB/s | 139 kB 00:00 2024-03-14T08:58:27.163 INFO:teuthology.orchestra.run.smithi154.stdout:(6/146): ceph-base-18.2.2-825.g2f465319.el8.x86 1.9 MB/s | 5.4 MB 00:02 2024-03-14T08:58:27.176 INFO:teuthology.orchestra.run.smithi191.stdout:(17/146): libradosstriper1-18.2.2-825.g2f465319 2.5 MB/s | 474 kB 00:00 2024-03-14T08:58:27.260 INFO:teuthology.orchestra.run.smithi191.stdout:(18/146): python3-ceph-argparse-18.2.2-825.g2f4 535 kB/s | 45 kB 00:00 2024-03-14T08:58:27.352 INFO:teuthology.orchestra.run.smithi191.stdout:(19/146): python3-ceph-common-18.2.2-825.g2f465 1.3 MB/s | 125 kB 00:00 2024-03-14T08:58:27.484 INFO:teuthology.orchestra.run.smithi191.stdout:(20/146): python3-cephfs-18.2.2-825.g2f465319.e 1.5 MB/s | 197 kB 00:00 2024-03-14T08:58:27.719 INFO:teuthology.orchestra.run.smithi191.stdout:(21/146): python3-rados-18.2.2-825.g2f465319.el 1.5 MB/s | 372 kB 00:00 2024-03-14T08:58:27.722 INFO:teuthology.orchestra.run.smithi154.stdout:(7/146): ceph-mon-18.2.2-825.g2f465319.el8.x86_ 5.0 MB/s | 4.6 MB 00:00 2024-03-14T08:58:27.861 INFO:teuthology.orchestra.run.smithi191.stdout:(22/146): python3-rbd-18.2.2-825.g2f465319.el8. 2.5 MB/s | 365 kB 00:00 2024-03-14T08:58:27.962 INFO:teuthology.orchestra.run.smithi191.stdout:(23/146): python3-rgw-18.2.2-825.g2f465319.el8. 1.0 MB/s | 107 kB 00:00 2024-03-14T08:58:28.062 INFO:teuthology.orchestra.run.smithi191.stdout:(24/146): rbd-fuse-18.2.2-825.g2f465319.el8.x86 952 kB/s | 95 kB 00:00 2024-03-14T08:58:28.662 INFO:teuthology.orchestra.run.smithi191.stdout:(25/146): librgw2-18.2.2-825.g2f465319.el8.x86_ 4.0 MB/s | 6.5 MB 00:01 2024-03-14T08:58:28.804 INFO:teuthology.orchestra.run.smithi191.stdout:(26/146): rbd-nbd-18.2.2-825.g2f465319.el8.x86_ 1.3 MB/s | 182 kB 00:00 2024-03-14T08:58:28.930 INFO:teuthology.orchestra.run.smithi191.stdout:(27/146): ceph-grafana-dashboards-18.2.2-825.g2 186 kB/s | 23 kB 00:00 2024-03-14T08:58:29.063 INFO:teuthology.orchestra.run.smithi191.stdout:(28/146): ceph-mgr-cephadm-18.2.2-825.g2f465319 1.0 MB/s | 137 kB 00:00 2024-03-14T08:58:29.122 INFO:teuthology.orchestra.run.smithi191.stdout:(29/146): rbd-mirror-18.2.2-825.g2f465319.el8.x 3.3 MB/s | 3.5 MB 00:01 2024-03-14T08:58:29.572 INFO:teuthology.orchestra.run.smithi154.stdout:(8/146): ceph-radosgw-18.2.2-825.g2f465319.el8. 7.1 MB/s | 13 MB 00:01 2024-03-14T08:58:29.581 INFO:teuthology.orchestra.run.smithi191.stdout:(30/146): ceph-mgr-dashboard-18.2.2-825.g2f4653 3.4 MB/s | 1.7 MB 00:00 2024-03-14T08:58:29.653 INFO:teuthology.orchestra.run.smithi154.stdout:(9/146): ceph-selinux-18.2.2-825.g2f465319.el8. 302 kB/s | 24 kB 00:00 2024-03-14T08:58:29.690 INFO:teuthology.orchestra.run.smithi191.stdout:(31/146): ceph-mgr-modules-core-18.2.2-825.g2f4 2.2 MB/s | 245 kB 00:00 2024-03-14T08:58:29.782 INFO:teuthology.orchestra.run.smithi191.stdout:(32/146): ceph-mgr-rook-18.2.2-825.g2f465319.el 537 kB/s | 49 kB 00:00 2024-03-14T08:58:29.865 INFO:teuthology.orchestra.run.smithi191.stdout:(33/146): ceph-prometheus-alerts-18.2.2-825.g2f 177 kB/s | 15 kB 00:00 2024-03-14T08:58:30.016 INFO:teuthology.orchestra.run.smithi191.stdout:(34/146): ceph-volume-18.2.2-825.g2f465319.el8. 1.7 MB/s | 263 kB 00:00 2024-03-14T08:58:30.108 INFO:teuthology.orchestra.run.smithi191.stdout:(35/146): cephadm-18.2.2-825.g2f465319.el8.noar 1.8 MB/s | 170 kB 00:00 2024-03-14T08:58:30.184 INFO:teuthology.orchestra.run.smithi191.stdout:(36/146): cephfs-top-18.2.2-825.g2f465319.el8.n 238 kB/s | 18 kB 00:00 2024-03-14T08:58:30.297 INFO:teuthology.orchestra.run.smithi154.stdout:(10/146): ceph-osd-18.2.2-825.g2f465319.el8.x86 5.7 MB/s | 18 MB 00:03 2024-03-14T08:58:30.398 INFO:teuthology.orchestra.run.smithi154.stdout:(11/146): cephfs-mirror-18.2.2-825.g2f465319.el 2.6 MB/s | 261 kB 00:00 2024-03-14T08:58:30.481 INFO:teuthology.orchestra.run.smithi154.stdout:(12/146): libcephfs-devel-18.2.2-825.g2f465319. 371 kB/s | 31 kB 00:00 2024-03-14T08:58:30.640 INFO:teuthology.orchestra.run.smithi154.stdout:(13/146): libcephfs2-18.2.2-825.g2f465319.el8.x 4.8 MB/s | 779 kB 00:00 2024-03-14T08:58:30.740 INFO:teuthology.orchestra.run.smithi154.stdout:(14/146): libcephsqlite-18.2.2-825.g2f465319.el 1.4 MB/s | 142 kB 00:00 2024-03-14T08:58:30.816 INFO:teuthology.orchestra.run.smithi154.stdout:(15/146): librados-devel-18.2.2-825.g2f465319.e 1.8 MB/s | 139 kB 00:00 2024-03-14T08:58:30.876 INFO:teuthology.orchestra.run.smithi191.stdout:(37/146): ceph-test-18.2.2-825.g2f465319.el8.x8 8.7 MB/s | 47 MB 00:05 2024-03-14T08:58:30.899 INFO:teuthology.orchestra.run.smithi154.stdout:(16/146): libradosstriper1-18.2.2-825.g2f465319 5.6 MB/s | 474 kB 00:00 2024-03-14T08:58:31.001 INFO:teuthology.orchestra.run.smithi191.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 375 kB/s | 306 kB 00:00 2024-03-14T08:58:31.093 INFO:teuthology.orchestra.run.smithi191.stdout:(39/146): ceph-mgr-diskprediction-local-18.2.2- 3.7 MB/s | 7.4 MB 00:01 2024-03-14T08:58:31.226 INFO:teuthology.orchestra.run.smithi191.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2024-03-14T08:58:31.368 INFO:teuthology.orchestra.run.smithi191.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 204 kB/s | 75 kB 00:00 2024-03-14T08:58:31.577 INFO:teuthology.orchestra.run.smithi191.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 124 kB/s | 44 kB 00:00 2024-03-14T08:58:31.674 INFO:teuthology.orchestra.run.smithi154.stdout:(17/146): librgw2-18.2.2-825.g2f465319.el8.x86_ 8.4 MB/s | 6.5 MB 00:00 2024-03-14T08:58:31.686 INFO:teuthology.orchestra.run.smithi191.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 803 kB/s | 476 kB 00:00 2024-03-14T08:58:31.785 INFO:teuthology.orchestra.run.smithi154.stdout:(18/146): python3-ceph-argparse-18.2.2-825.g2f4 406 kB/s | 45 kB 00:00 2024-03-14T08:58:31.858 INFO:teuthology.orchestra.run.smithi154.stdout:(19/146): python3-ceph-common-18.2.2-825.g2f465 1.7 MB/s | 125 kB 00:00 2024-03-14T08:58:31.895 INFO:teuthology.orchestra.run.smithi191.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 352 kB/s | 185 kB 00:00 2024-03-14T08:58:31.920 INFO:teuthology.orchestra.run.smithi191.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 96 kB/s | 33 kB 00:00 2024-03-14T08:58:31.942 INFO:teuthology.orchestra.run.smithi154.stdout:(20/146): python3-cephfs-18.2.2-825.g2f465319.e 2.3 MB/s | 197 kB 00:00 2024-03-14T08:58:31.945 INFO:teuthology.orchestra.run.smithi191.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 63 kB/s | 16 kB 00:00 2024-03-14T08:58:32.067 INFO:teuthology.orchestra.run.smithi154.stdout:(21/146): python3-rados-18.2.2-825.g2f465319.el 2.9 MB/s | 372 kB 00:00 2024-03-14T08:58:32.143 INFO:teuthology.orchestra.run.smithi154.stdout:(22/146): python3-rbd-18.2.2-825.g2f465319.el8. 4.8 MB/s | 365 kB 00:00 2024-03-14T08:58:32.226 INFO:teuthology.orchestra.run.smithi154.stdout:(23/146): python3-rgw-18.2.2-825.g2f465319.el8. 1.3 MB/s | 107 kB 00:00 2024-03-14T08:58:32.287 INFO:teuthology.orchestra.run.smithi191.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 238 kB/s | 81 kB 00:00 2024-03-14T08:58:32.321 INFO:teuthology.orchestra.run.smithi191.stdout:(48/146): python3-cheroot-8.5.2-1.el8.noarch.rp 405 kB/s | 173 kB 00:00 2024-03-14T08:58:32.335 INFO:teuthology.orchestra.run.smithi154.stdout:(24/146): rbd-fuse-18.2.2-825.g2f465319.el8.x86 877 kB/s | 95 kB 00:00 2024-03-14T08:58:32.471 INFO:teuthology.orchestra.run.smithi191.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 60 kB/s | 11 kB 00:00 2024-03-14T08:58:32.505 INFO:teuthology.orchestra.run.smithi191.stdout:(50/146): python3-cherrypy-18.4.0-1.el8.noarch. 657 kB/s | 384 kB 00:00 2024-03-14T08:58:32.581 INFO:teuthology.orchestra.run.smithi191.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 68 kB/s | 18 kB 00:00 2024-03-14T08:58:32.644 INFO:teuthology.orchestra.run.smithi154.stdout:(25/146): ceph-common-18.2.2-825.g2f465319.el8. 2.6 MB/s | 22 MB 00:08 2024-03-14T08:58:32.794 INFO:teuthology.orchestra.run.smithi154.stdout:(26/146): rbd-mirror-18.2.2-825.g2f465319.el8.x 7.5 MB/s | 3.5 MB 00:00 2024-03-14T08:58:32.819 INFO:teuthology.orchestra.run.smithi154.stdout:(27/146): rbd-nbd-18.2.2-825.g2f465319.el8.x86_ 1.0 MB/s | 182 kB 00:00 2024-03-14T08:58:32.839 INFO:teuthology.orchestra.run.smithi191.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 146 kB/s | 49 kB 00:00 2024-03-14T08:58:32.853 INFO:teuthology.orchestra.run.smithi154.stdout:(28/146): ceph-grafana-dashboards-18.2.2-825.g2 398 kB/s | 23 kB 00:00 2024-03-14T08:58:32.911 INFO:teuthology.orchestra.run.smithi154.stdout:(29/146): ceph-mgr-cephadm-18.2.2-825.g2f465319 1.5 MB/s | 137 kB 00:00 2024-03-14T08:58:32.923 INFO:teuthology.orchestra.run.smithi191.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 172 kB/s | 59 kB 00:00 2024-03-14T08:58:33.148 INFO:teuthology.orchestra.run.smithi191.stdout:(54/146): python3-kubernetes-11.0.0-6.el8.noarc 1.7 MB/s | 1.2 MB 00:00 2024-03-14T08:58:33.182 INFO:teuthology.orchestra.run.smithi191.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 176 kB/s | 60 kB 00:00 2024-03-14T08:58:33.353 INFO:teuthology.orchestra.run.smithi154.stdout:(30/146): ceph-test-18.2.2-825.g2f465319.el8.x8 13 MB/s | 47 MB 00:03 2024-03-14T08:58:33.395 INFO:teuthology.orchestra.run.smithi154.stdout:(31/146): ceph-mgr-dashboard-18.2.2-825.g2f4653 3.2 MB/s | 1.7 MB 00:00 2024-03-14T08:58:33.412 INFO:teuthology.orchestra.run.smithi154.stdout:(32/146): ceph-mgr-modules-core-18.2.2-825.g2f4 4.1 MB/s | 245 kB 00:00 2024-03-14T08:58:33.415 INFO:teuthology.orchestra.run.smithi191.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 58 kB/s | 16 kB 00:00 2024-03-14T08:58:33.441 INFO:teuthology.orchestra.run.smithi191.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 546 kB/s | 283 kB 00:00 2024-03-14T08:58:33.446 INFO:teuthology.orchestra.run.smithi154.stdout:(33/146): ceph-prometheus-alerts-18.2.2-825.g2f 444 kB/s | 15 kB 00:00 2024-03-14T08:58:33.466 INFO:teuthology.orchestra.run.smithi191.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 118 kB/s | 34 kB 00:00 2024-03-14T08:58:33.742 INFO:teuthology.orchestra.run.smithi191.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 68 kB/s | 19 kB 00:00 2024-03-14T08:58:33.754 INFO:teuthology.orchestra.run.smithi154.stdout:(34/146): ceph-mgr-rook-18.2.2-825.g2f465319.el 137 kB/s | 49 kB 00:00 2024-03-14T08:58:33.784 INFO:teuthology.orchestra.run.smithi191.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 188 kB/s | 64 kB 00:00 2024-03-14T08:58:33.805 INFO:teuthology.orchestra.run.smithi154.stdout:(35/146): ceph-volume-18.2.2-825.g2f465319.el8. 733 kB/s | 263 kB 00:00 2024-03-14T08:58:33.838 INFO:teuthology.orchestra.run.smithi154.stdout:(36/146): cephfs-top-18.2.2-825.g2f465319.el8.n 544 kB/s | 18 kB 00:00 2024-03-14T08:58:33.843 INFO:teuthology.orchestra.run.smithi191.stdout:(61/146): python3-routes-2.4.1-12.el8.noarch.rp 460 kB/s | 196 kB 00:00 2024-03-14T08:58:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:(37/146): cephadm-18.2.2-825.g2f465319.el8.noar 1.2 MB/s | 170 kB 00:00 2024-03-14T08:58:34.018 INFO:teuthology.orchestra.run.smithi191.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 87 kB/s | 24 kB 00:00 2024-03-14T08:58:34.052 INFO:teuthology.orchestra.run.smithi191.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 109 kB/s | 29 kB 00:00 2024-03-14T08:58:34.102 INFO:teuthology.orchestra.run.smithi191.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 106 kB/s | 27 kB 00:00 2024-03-14T08:58:34.239 INFO:teuthology.orchestra.run.smithi154.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2024-03-14T08:58:34.422 INFO:teuthology.orchestra.run.smithi154.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 410 kB/s | 75 kB 00:00 2024-03-14T08:58:34.447 INFO:teuthology.orchestra.run.smithi154.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 503 kB/s | 306 kB 00:00 2024-03-14T08:58:34.461 INFO:teuthology.orchestra.run.smithi191.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 171 kB/s | 61 kB 00:00 2024-03-14T08:58:34.528 INFO:teuthology.orchestra.run.smithi191.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 501 kB/s | 255 kB 00:00 2024-03-14T08:58:34.553 INFO:teuthology.orchestra.run.smithi191.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 501 kB/s | 251 kB 00:00 2024-03-14T08:58:34.556 INFO:teuthology.orchestra.run.smithi154.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 402 kB/s | 44 kB 00:00 2024-03-14T08:58:34.706 INFO:teuthology.orchestra.run.smithi154.stdout:(42/146): ceph-mgr-diskprediction-local-18.2.2- 4.1 MB/s | 7.4 MB 00:01 2024-03-14T08:58:34.732 INFO:teuthology.orchestra.run.smithi154.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2024-03-14T08:58:34.757 INFO:teuthology.orchestra.run.smithi154.stdout:(44/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2024-03-14T08:58:34.795 INFO:teuthology.orchestra.run.smithi191.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 86 kB/s | 23 kB 00:00 2024-03-14T08:58:34.820 INFO:teuthology.orchestra.run.smithi191.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 237 kB/s | 85 kB 00:00 2024-03-14T08:58:34.832 INFO:teuthology.orchestra.run.smithi154.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-03-14T08:58:34.846 INFO:teuthology.orchestra.run.smithi191.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-03-14T08:58:34.874 INFO:teuthology.orchestra.run.smithi154.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2024-03-14T08:58:34.980 INFO:teuthology.orchestra.run.smithi191.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 303 kB/s | 48 kB 00:00 2024-03-14T08:58:34.983 INFO:teuthology.orchestra.run.smithi154.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2024-03-14T08:58:35.008 INFO:teuthology.orchestra.run.smithi154.stdout:(48/146): python3-cachetools-3.1.1-4.el8.noarch 109 kB/s | 33 kB 00:00 2024-03-14T08:58:35.013 INFO:teuthology.orchestra.run.smithi191.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 124 kB/s | 20 kB 00:00 2024-03-14T08:58:35.033 INFO:teuthology.orchestra.run.smithi154.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2024-03-14T08:58:35.084 INFO:teuthology.orchestra.run.smithi154.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2024-03-14T08:58:35.134 INFO:teuthology.orchestra.run.smithi154.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2024-03-14T08:58:35.147 INFO:teuthology.orchestra.run.smithi191.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 263 kB/s | 44 kB 00:00 2024-03-14T08:58:35.172 INFO:teuthology.orchestra.run.smithi191.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 693 kB/s | 110 kB 00:00 2024-03-14T08:58:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2024-03-14T08:58:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2024-03-14T08:58:35.323 INFO:teuthology.orchestra.run.smithi191.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 151 kB/s | 27 kB 00:00 2024-03-14T08:58:35.326 INFO:teuthology.orchestra.run.smithi154.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 424 kB/s | 60 kB 00:00 2024-03-14T08:58:35.352 INFO:teuthology.orchestra.run.smithi154.stdout:(55/146): python3-more-itertools-7.2.0-3.el8.no 270 kB/s | 59 kB 00:00 2024-03-14T08:58:35.356 INFO:teuthology.orchestra.run.smithi191.stdout:(76/146): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2024-03-14T08:58:35.382 INFO:teuthology.orchestra.run.smithi191.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 604 kB/s | 126 kB 00:00 2024-03-14T08:58:35.410 INFO:teuthology.orchestra.run.smithi154.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-03-14T08:58:35.436 INFO:teuthology.orchestra.run.smithi154.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2024-03-14T08:58:35.483 INFO:teuthology.orchestra.run.smithi191.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2024-03-14T08:58:35.511 INFO:teuthology.orchestra.run.smithi154.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 212 kB/s | 34 kB 00:00 2024-03-14T08:58:35.545 INFO:teuthology.orchestra.run.smithi154.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2024-03-14T08:58:35.570 INFO:teuthology.orchestra.run.smithi154.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 479 kB/s | 64 kB 00:00 2024-03-14T08:58:35.583 INFO:teuthology.orchestra.run.smithi191.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2024-03-14T08:58:35.617 INFO:teuthology.orchestra.run.smithi191.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2024-03-14T08:58:35.620 INFO:teuthology.orchestra.run.smithi154.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-03-14T08:58:35.645 INFO:teuthology.orchestra.run.smithi154.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-03-14T08:58:35.650 INFO:teuthology.orchestra.run.smithi191.stdout:(81/146): bison-3.0.4-10.el8.x86_64.rpm 2.3 MB/s | 688 kB 00:00 2024-03-14T08:58:35.671 INFO:teuthology.orchestra.run.smithi154.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2024-03-14T08:58:35.726 INFO:teuthology.orchestra.run.smithi191.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 272 kB/s | 39 kB 00:00 2024-03-14T08:58:35.763 INFO:teuthology.orchestra.run.smithi154.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 193 kB/s | 27 kB 00:00 2024-03-14T08:58:35.776 INFO:teuthology.orchestra.run.smithi191.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2024-03-14T08:58:35.779 INFO:teuthology.orchestra.run.smithi154.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2024-03-14T08:58:35.801 INFO:teuthology.orchestra.run.smithi191.stdout:(84/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-03-14T08:58:35.805 INFO:teuthology.orchestra.run.smithi154.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-03-14T08:58:35.877 INFO:teuthology.orchestra.run.smithi191.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 686 kB/s | 103 kB 00:00 2024-03-14T08:58:35.880 INFO:teuthology.orchestra.run.smithi154.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 850 kB/s | 85 kB 00:00 2024-03-14T08:58:35.905 INFO:teuthology.orchestra.run.smithi154.stdout:(68/146): python3-websocket-client-0.56.0-5.el8 430 kB/s | 61 kB 00:00 2024-03-14T08:58:35.930 INFO:teuthology.orchestra.run.smithi154.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2024-03-14T08:58:35.956 INFO:teuthology.orchestra.run.smithi154.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-03-14T08:58:36.002 INFO:teuthology.orchestra.run.smithi191.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.6 MB/s | 538 kB 00:00 2024-03-14T08:58:36.064 INFO:teuthology.orchestra.run.smithi154.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2024-03-14T08:58:36.086 INFO:teuthology.orchestra.run.smithi191.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2024-03-14T08:58:36.090 INFO:teuthology.orchestra.run.smithi154.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 126 kB/s | 20 kB 00:00 2024-03-14T08:58:36.140 INFO:teuthology.orchestra.run.smithi154.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 247 kB/s | 44 kB 00:00 2024-03-14T08:58:36.180 INFO:teuthology.orchestra.run.smithi191.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-03-14T08:58:36.212 INFO:teuthology.orchestra.run.smithi191.stdout:(89/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-03-14T08:58:36.215 INFO:teuthology.orchestra.run.smithi154.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 222 kB/s | 27 kB 00:00 2024-03-14T08:58:36.240 INFO:teuthology.orchestra.run.smithi154.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 626 kB/s | 110 kB 00:00 2024-03-14T08:58:36.245 INFO:teuthology.orchestra.run.smithi191.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.4 MB/s | 225 kB 00:00 2024-03-14T08:58:36.316 INFO:teuthology.orchestra.run.smithi154.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 719 kB/s | 126 kB 00:00 2024-03-14T08:58:36.354 INFO:teuthology.orchestra.run.smithi191.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2024-03-14T08:58:36.383 INFO:teuthology.orchestra.run.smithi154.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2024-03-14T08:58:36.387 INFO:teuthology.orchestra.run.smithi191.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-03-14T08:58:36.433 INFO:teuthology.orchestra.run.smithi154.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.1 MB/s | 457 kB 00:00 2024-03-14T08:58:36.500 INFO:teuthology.orchestra.run.smithi154.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 1.7 MB/s | 320 kB 00:00 2024-03-14T08:58:36.542 INFO:teuthology.orchestra.run.smithi154.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2024-03-14T08:58:36.555 INFO:teuthology.orchestra.run.smithi191.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 15 MB/s | 5.2 MB 00:00 2024-03-14T08:58:36.576 INFO:teuthology.orchestra.run.smithi154.stdout:(81/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2024-03-14T08:58:36.588 INFO:teuthology.orchestra.run.smithi191.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-03-14T08:58:36.593 INFO:teuthology.orchestra.run.smithi154.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-03-14T08:58:36.614 INFO:teuthology.orchestra.run.smithi191.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2024-03-14T08:58:36.618 INFO:teuthology.orchestra.run.smithi154.stdout:(83/146): libtool-2.4.6-25.el8.x86_64.rpm 3.0 MB/s | 709 kB 00:00 2024-03-14T08:58:36.643 INFO:teuthology.orchestra.run.smithi154.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2024-03-14T08:58:36.647 INFO:teuthology.orchestra.run.smithi191.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2024-03-14T08:58:36.685 INFO:teuthology.orchestra.run.smithi154.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2024-03-14T08:58:36.723 INFO:teuthology.orchestra.run.smithi191.stdout:(97/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-03-14T08:58:36.735 INFO:teuthology.orchestra.run.smithi154.stdout:(86/146): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-03-14T08:58:36.748 INFO:teuthology.orchestra.run.smithi191.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 405 kB/s | 54 kB 00:00 2024-03-14T08:58:36.774 INFO:teuthology.orchestra.run.smithi191.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 3.6 MB/s | 188 kB 00:00 2024-03-14T08:58:36.811 INFO:teuthology.orchestra.run.smithi154.stdout:(87/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.9 MB/s | 225 kB 00:00 2024-03-14T08:58:36.878 INFO:teuthology.orchestra.run.smithi154.stdout:(88/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-03-14T08:58:36.932 INFO:teuthology.orchestra.run.smithi191.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 8.2 MB/s | 1.5 MB 00:00 2024-03-14T08:58:36.936 INFO:teuthology.orchestra.run.smithi154.stdout:(89/146): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2024-03-14T08:58:36.961 INFO:teuthology.orchestra.run.smithi154.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2024-03-14T08:58:36.987 INFO:teuthology.orchestra.run.smithi154.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2024-03-14T08:58:37.137 INFO:teuthology.orchestra.run.smithi154.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 20 MB/s | 5.2 MB 00:00 2024-03-14T08:58:37.212 INFO:teuthology.orchestra.run.smithi154.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-03-14T08:58:37.246 INFO:teuthology.orchestra.run.smithi154.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 2.7 MB/s | 714 kB 00:00 2024-03-14T08:58:37.271 INFO:teuthology.orchestra.run.smithi154.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-03-14T08:58:37.291 INFO:teuthology.orchestra.run.smithi191.stdout:(101/146): postgresql-contrib-10.23-4.module+el 1.5 MB/s | 812 kB 00:00 2024-03-14T08:58:37.355 INFO:teuthology.orchestra.run.smithi154.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2024-03-14T08:58:37.388 INFO:teuthology.orchestra.run.smithi154.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 381 kB/s | 54 kB 00:00 2024-03-14T08:58:37.442 INFO:teuthology.orchestra.run.smithi191.stdout:(102/146): postgresql-server-10.23-4.module+el8 9.9 MB/s | 5.1 MB 00:00 2024-03-14T08:58:37.459 INFO:teuthology.orchestra.run.smithi191.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 107 kB/s | 18 kB 00:00 2024-03-14T08:58:37.489 INFO:teuthology.orchestra.run.smithi154.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-03-14T08:58:37.539 INFO:teuthology.orchestra.run.smithi154.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 10 MB/s | 1.5 MB 00:00 2024-03-14T08:58:37.567 INFO:teuthology.orchestra.run.smithi191.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 340 kB/s | 43 kB 00:00 2024-03-14T08:58:37.593 INFO:teuthology.orchestra.run.smithi191.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2024-03-14T08:58:37.706 INFO:teuthology.orchestra.run.smithi154.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 32 MB/s | 14 MB 00:00 2024-03-14T08:58:37.743 INFO:teuthology.orchestra.run.smithi191.stdout:(106/146): python3-scipy-1.0.0-21.module+el8.9. 13 MB/s | 14 MB 00:01 2024-03-14T08:58:37.748 INFO:teuthology.orchestra.run.smithi154.stdout:(101/146): postgresql-contrib-10.23-4.module+el 3.1 MB/s | 812 kB 00:00 2024-03-14T08:58:37.760 INFO:teuthology.orchestra.run.smithi191.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 256 kB/s | 43 kB 00:00 2024-03-14T08:58:37.785 INFO:teuthology.orchestra.run.smithi191.stdout:(108/146): python3-pycparser-2.14-14.el8.noarch 501 kB/s | 109 kB 00:00 2024-03-14T08:58:37.881 INFO:teuthology.orchestra.run.smithi154.stdout:(102/146): postgresql-server-10.23-4.module+el8 15 MB/s | 5.1 MB 00:00 2024-03-14T08:58:37.906 INFO:teuthology.orchestra.run.smithi154.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 113 kB/s | 18 kB 00:00 2024-03-14T08:58:37.919 INFO:teuthology.orchestra.run.smithi191.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 106 kB/s | 19 kB 00:00 2024-03-14T08:58:37.932 INFO:teuthology.orchestra.run.smithi154.stdout:(104/146): python3-packaging-16.8-9.el8.noarch. 214 kB/s | 48 kB 00:00 2024-03-14T08:58:37.944 INFO:teuthology.orchestra.run.smithi191.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 238 kB/s | 44 kB 00:00 2024-03-14T08:58:38.007 INFO:teuthology.orchestra.run.smithi154.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 340 kB/s | 43 kB 00:00 2024-03-14T08:58:38.028 INFO:teuthology.orchestra.run.smithi191.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.4 MB/s | 605 kB 00:00 2024-03-14T08:58:38.066 INFO:teuthology.orchestra.run.smithi154.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 974 kB/s | 155 kB 00:00 2024-03-14T08:58:38.162 INFO:teuthology.orchestra.run.smithi191.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 944 kB/s | 229 kB 00:00 2024-03-14T08:58:38.191 INFO:teuthology.orchestra.run.smithi154.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 421 kB/s | 109 kB 00:00 2024-03-14T08:58:38.246 INFO:teuthology.orchestra.run.smithi191.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 732 kB/s | 220 kB 00:00 2024-03-14T08:58:38.262 INFO:teuthology.orchestra.run.smithi191.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 853 kB/s | 200 kB 00:00 2024-03-14T08:58:38.266 INFO:teuthology.orchestra.run.smithi154.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 165 kB/s | 43 kB 00:00 2024-03-14T08:58:38.283 INFO:teuthology.orchestra.run.smithi154.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 86 kB/s | 19 kB 00:00 2024-03-14T08:58:38.288 INFO:teuthology.orchestra.run.smithi191.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 382 kB/s | 48 kB 00:00 2024-03-14T08:58:38.317 INFO:teuthology.orchestra.run.smithi154.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 348 kB/s | 44 kB 00:00 2024-03-14T08:58:38.380 INFO:teuthology.orchestra.run.smithi191.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 649 kB/s | 87 kB 00:00 2024-03-14T08:58:38.405 INFO:teuthology.orchestra.run.smithi191.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 405 kB/s | 58 kB 00:00 2024-03-14T08:58:38.417 INFO:teuthology.orchestra.run.smithi154.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-03-14T08:58:38.443 INFO:teuthology.orchestra.run.smithi154.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-03-14T08:58:38.464 INFO:teuthology.orchestra.run.smithi191.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 219 kB/s | 39 kB 00:00 2024-03-14T08:58:38.539 INFO:teuthology.orchestra.run.smithi191.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 522 kB/s | 83 kB 00:00 2024-03-14T08:58:38.543 INFO:teuthology.orchestra.run.smithi154.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 977 kB/s | 220 kB 00:00 2024-03-14T08:58:38.582 INFO:teuthology.orchestra.run.smithi191.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-03-14T08:58:38.611 INFO:teuthology.orchestra.run.smithi154.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 288 kB/s | 48 kB 00:00 2024-03-14T08:58:38.665 INFO:teuthology.orchestra.run.smithi154.stdout:(115/146): libselinux-devel-2.9-5.el8.x86_64.rp 807 kB/s | 200 kB 00:00 2024-03-14T08:58:38.666 INFO:teuthology.orchestra.run.smithi191.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 795 kB/s | 100 kB 00:00 2024-03-14T08:58:38.691 INFO:teuthology.orchestra.run.smithi191.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 762 kB/s | 172 kB 00:00 2024-03-14T08:58:38.710 INFO:teuthology.orchestra.run.smithi154.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 520 kB/s | 87 kB 00:00 2024-03-14T08:58:38.783 INFO:teuthology.orchestra.run.smithi191.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2024-03-14T08:58:38.800 INFO:teuthology.orchestra.run.smithi191.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 541 kB/s | 73 kB 00:00 2024-03-14T08:58:38.819 INFO:teuthology.orchestra.run.smithi154.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 276 kB/s | 58 kB 00:00 2024-03-14T08:58:38.834 INFO:teuthology.orchestra.run.smithi191.stdout:(125/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 144 kB 00:00 2024-03-14T08:58:38.844 INFO:teuthology.orchestra.run.smithi154.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 216 kB/s | 39 kB 00:00 2024-03-14T08:58:38.859 INFO:teuthology.orchestra.run.smithi191.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 814 kB/s | 62 kB 00:00 2024-03-14T08:58:38.894 INFO:teuthology.orchestra.run.smithi154.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 452 kB/s | 83 kB 00:00 2024-03-14T08:58:38.951 INFO:teuthology.orchestra.run.smithi191.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.6 MB/s | 562 kB 00:00 2024-03-14T08:58:38.978 INFO:teuthology.orchestra.run.smithi154.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-03-14T08:58:38.985 INFO:teuthology.orchestra.run.smithi191.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-03-14T08:58:39.003 INFO:teuthology.orchestra.run.smithi154.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-03-14T08:58:39.028 INFO:teuthology.orchestra.run.smithi154.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 745 kB/s | 100 kB 00:00 2024-03-14T08:58:39.079 INFO:teuthology.orchestra.run.smithi154.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2024-03-14T08:58:39.085 INFO:teuthology.orchestra.run.smithi191.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 4.1 MB/s | 559 kB 00:00 2024-03-14T08:58:39.144 INFO:teuthology.orchestra.run.smithi191.stdout:(130/146): openssl-devel-1.1.1k-12.el8_9.x86_64 7.5 MB/s | 2.3 MB 00:00 2024-03-14T08:58:39.254 INFO:teuthology.orchestra.run.smithi154.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 290 kB/s | 73 kB 00:00 2024-03-14T08:58:39.279 INFO:teuthology.orchestra.run.smithi154.stdout:(125/146): elfutils-libelf-devel-0.189-3.el8.x8 308 kB/s | 62 kB 00:00 2024-03-14T08:58:39.302 INFO:teuthology.orchestra.run.smithi191.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-03-14T08:58:39.304 INFO:teuthology.orchestra.run.smithi154.stdout:(126/146): NetworkManager-initscripts-updown-1. 522 kB/s | 144 kB 00:00 2024-03-14T08:58:39.355 INFO:teuthology.orchestra.run.smithi154.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 5.5 MB/s | 562 kB 00:00 2024-03-14T08:58:39.430 INFO:teuthology.orchestra.run.smithi154.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-03-14T08:58:39.444 INFO:teuthology.orchestra.run.smithi191.stdout:(132/146): librbd1-18.2.2-825.g2f465319.el8.x86 9.9 MB/s | 3.6 MB 00:00 2024-03-14T08:58:39.470 INFO:teuthology.orchestra.run.smithi191.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-03-14T08:58:39.514 INFO:teuthology.orchestra.run.smithi154.stdout:(129/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.9 MB/s | 2.3 MB 00:00 2024-03-14T08:58:39.547 INFO:teuthology.orchestra.run.smithi154.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 2.8 MB/s | 559 kB 00:00 2024-03-14T08:58:39.578 INFO:teuthology.orchestra.run.smithi191.stdout:(134/146): librados2-18.2.2-825.g2f465319.el8.x 6.2 MB/s | 3.7 MB 00:00 2024-03-14T08:58:39.604 INFO:teuthology.orchestra.run.smithi191.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2024-03-14T08:58:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2024-03-14T08:58:39.713 INFO:teuthology.orchestra.run.smithi191.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 738 kB/s | 99 kB 00:00 2024-03-14T08:58:39.755 INFO:teuthology.orchestra.run.smithi191.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 819 kB/s | 103 kB 00:00 2024-03-14T08:58:39.756 INFO:teuthology.orchestra.run.smithi154.stdout:(131/146): librados2-18.2.2-825.g2f465319.el8.x 11 MB/s | 3.7 MB 00:00 2024-03-14T08:58:39.790 INFO:teuthology.orchestra.run.smithi154.stdout:(132/146): elfutils-libelf-0.189-3.el8.x86_64.r 959 kB/s | 232 kB 00:00 2024-03-14T08:58:39.805 INFO:teuthology.orchestra.run.smithi191.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2024-03-14T08:58:39.848 INFO:teuthology.orchestra.run.smithi154.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 4.0 MB/s | 237 kB 00:00 2024-03-14T08:58:39.855 INFO:teuthology.orchestra.run.smithi191.stdout:(140/146): libfdisk-2.32.1-43.el8.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-14T08:58:39.889 INFO:teuthology.orchestra.run.smithi191.stdout:(141/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.7 MB/s | 854 kB 00:00 2024-03-14T08:58:39.915 INFO:teuthology.orchestra.run.smithi154.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-03-14T08:58:40.023 INFO:teuthology.orchestra.run.smithi191.stdout:(142/146): libblkid-2.32.1-43.el8.x86_64.rpm 824 kB/s | 220 kB 00:00 2024-03-14T08:58:40.057 INFO:teuthology.orchestra.run.smithi191.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.5 MB/s | 711 kB 00:00 2024-03-14T08:58:40.065 INFO:teuthology.orchestra.run.smithi154.stdout:(135/146): librbd1-18.2.2-825.g2f465319.el8.x86 6.5 MB/s | 3.6 MB 00:00 2024-03-14T08:58:40.091 INFO:teuthology.orchestra.run.smithi154.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 565 kB/s | 99 kB 00:00 2024-03-14T08:58:40.107 INFO:teuthology.orchestra.run.smithi191.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.8 MB/s | 1.5 MB 00:00 2024-03-14T08:58:40.116 INFO:teuthology.orchestra.run.smithi154.stdout:(137/146): libsmartcols-2.32.1-43.el8.x86_64.rp 671 kB/s | 179 kB 00:00 2024-03-14T08:58:40.141 INFO:teuthology.orchestra.run.smithi191.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-03-14T08:58:40.174 INFO:teuthology.orchestra.run.smithi191.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 9.1 MB/s | 1.1 MB 00:00 2024-03-14T08:58:40.178 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-03-14T08:58:40.178 INFO:teuthology.orchestra.run.smithi191.stdout:Total 12 MB/s | 212 MB 00:17 2024-03-14T08:58:40.191 INFO:teuthology.orchestra.run.smithi154.stdout:(138/146): iproute-6.2.0-5.el8_9.x86_64.rpm 11 MB/s | 854 kB 00:00 2024-03-14T08:58:40.233 INFO:teuthology.orchestra.run.smithi154.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 15 MB/s | 2.5 MB 00:00 2024-03-14T08:58:40.280 INFO:teuthology.orchestra.run.smithi154.stdout:(140/146): util-linux-user-2.32.1-43.el8.x86_64 543 kB/s | 103 kB 00:00 2024-03-14T08:58:40.326 INFO:teuthology.orchestra.run.smithi154.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2024-03-14T08:58:40.418 INFO:teuthology.orchestra.run.smithi154.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 254 kB 00:00 2024-03-14T08:58:40.451 INFO:teuthology.orchestra.run.smithi154.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.1 MB/s | 711 kB 00:00 2024-03-14T08:58:40.568 INFO:teuthology.orchestra.run.smithi154.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.1 MB/s | 1.5 MB 00:00 2024-03-14T08:58:40.593 INFO:teuthology.orchestra.run.smithi154.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-03-14T08:58:40.635 INFO:teuthology.orchestra.run.smithi154.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-14T08:58:40.638 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-03-14T08:58:40.638 INFO:teuthology.orchestra.run.smithi154.stdout:Total 13 MB/s | 212 MB 00:16 2024-03-14T08:58:41.279 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-03-14T08:58:41.672 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-03-14T08:58:41.672 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-03-14T08:58:41.707 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-03-14T08:58:42.105 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-03-14T08:58:42.105 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-03-14T08:58:42.860 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-03-14T08:58:42.871 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-03-14T08:58:43.297 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-03-14T08:58:43.307 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-03-14T08:58:48.579 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-03-14T08:58:48.713 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-14T08:58:48.992 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-14T08:58:49.189 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-14T08:58:49.303 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-03-14T08:58:49.378 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-14T08:58:49.572 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-14T08:58:49.644 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-14T08:58:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-14T08:58:49.847 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-14T08:58:49.921 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-14T08:58:49.935 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-14T08:58:50.128 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-14T08:58:50.150 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-14T08:58:50.317 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-14T08:58:50.322 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-14T08:58:50.389 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-14T08:58:50.500 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-14T08:58:50.569 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-14T08:58:50.623 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-14T08:58:50.758 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-14T08:58:50.830 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-14T08:58:50.856 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-14T08:58:51.064 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-14T08:58:51.259 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-14T08:58:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-14T08:58:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-14T08:58:51.472 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T08:58:51.526 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-14T08:58:51.775 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-14T08:58:51.852 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T08:58:52.064 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T08:58:52.176 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-14T08:58:52.192 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T08:58:52.372 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T08:58:52.383 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T08:58:52.471 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T08:58:52.740 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 12/162 2024-03-14T08:58:52.775 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T08:58:52.966 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T08:58:53.005 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-14T08:58:53.094 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-14T08:58:53.095 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T08:58:53.241 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T08:58:53.269 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-14T08:58:53.343 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T08:58:53.500 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-14T08:58:53.594 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 12/162 2024-03-14T08:58:53.716 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-14T08:58:53.833 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-14T08:58:53.948 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-14T08:58:54.068 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-14T08:58:54.097 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-14T08:58:54.329 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-14T08:58:54.342 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-14T08:58:54.455 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T08:58:54.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-14T08:58:54.561 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T08:58:54.705 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 20/162 2024-03-14T08:58:54.767 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 20/162 2024-03-14T08:58:54.881 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-14T08:58:54.988 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-14T08:58:55.129 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-14T08:58:55.136 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-14T08:58:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T08:58:55.321 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-14T08:58:55.391 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T08:58:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 20/162 2024-03-14T08:58:55.637 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 20/162 2024-03-14T08:58:55.849 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-14T08:58:55.941 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-14T08:58:55.970 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-14T08:58:56.155 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-14T08:58:56.166 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-14T08:58:56.757 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-14T08:58:56.792 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-14T08:58:56.963 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-14T08:58:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-14T08:58:57.597 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-14T08:58:57.644 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-14T08:58:57.824 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-14T08:58:57.936 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-14T08:58:58.127 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-14T08:58:58.292 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-14T08:58:58.519 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-14T08:58:58.598 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-14T08:58:58.840 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-14T08:58:58.910 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-14T08:58:58.996 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-14T08:58:59.172 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-14T08:58:59.203 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-14T08:58:59.310 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:18.2.2-825.g2f465319.el8.x 31/162 2024-03-14T08:58:59.465 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:18.2.2-825.g2f465319.el8 32/162 2024-03-14T08:58:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-14T08:58:59.755 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 33/162 2024-03-14T08:58:59.798 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-14T08:59:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-14T08:59:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:18.2.2-825.g2f465319.el8.x 31/162 2024-03-14T08:59:00.669 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:18.2.2-825.g2f465319.el8 32/162 2024-03-14T08:59:00.727 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-14T08:59:00.919 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 33/162 2024-03-14T08:59:01.078 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-14T08:59:01.374 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-14T08:59:01.630 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-14T08:59:01.901 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-14T08:59:02.238 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-14T08:59:02.503 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-14T08:59:02.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-14T08:59:03.367 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-14T08:59:04.100 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-14T08:59:04.352 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-14T08:59:04.468 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-14T08:59:04.574 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-14T08:59:04.753 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 41/162 2024-03-14T08:59:04.816 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-14T08:59:04.984 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 42/162 2024-03-14T08:59:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-14T08:59:05.186 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-14T08:59:05.255 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-14T08:59:05.258 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-14T08:59:05.453 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 41/162 2024-03-14T08:59:05.486 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-14T08:59:05.539 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-14T08:59:05.677 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 42/162 2024-03-14T08:59:05.742 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-14T08:59:05.889 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-14T08:59:05.949 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-14T08:59:06.022 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-14T08:59:06.155 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-14T08:59:06.215 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-14T08:59:06.261 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-14T08:59:06.460 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-14T08:59:06.720 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-14T08:59:06.724 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-14T08:59:06.964 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-14T08:59:07.070 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-14T08:59:07.087 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-14T08:59:07.307 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-14T08:59:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-14T08:59:07.532 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-14T08:59:07.724 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-14T08:59:07.789 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-14T08:59:07.806 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-14T08:59:07.914 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-14T08:59:08.026 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-14T08:59:08.140 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-14T08:59:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-14T08:59:08.409 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-14T08:59:08.466 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-14T08:59:08.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-14T08:59:08.749 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-14T08:59:08.858 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-14T08:59:08.974 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-14T08:59:09.123 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-14T08:59:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-14T08:59:09.364 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-14T08:59:09.478 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-14T08:59:09.727 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-14T08:59:09.753 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-14T08:59:09.909 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-14T08:59:10.080 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-14T08:59:10.165 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-14T08:59:10.401 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-14T08:59:10.572 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-14T08:59:10.618 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-14T08:59:10.795 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-14T08:59:10.966 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-14T08:59:11.059 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-14T08:59:11.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-14T08:59:11.307 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-14T08:59:11.528 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-14T08:59:11.554 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-14T08:59:11.713 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-14T08:59:11.766 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-14T08:59:11.941 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-14T08:59:12.020 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-14T08:59:12.222 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-14T08:59:12.284 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-14T08:59:12.402 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-14T08:59:12.481 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-14T08:59:12.509 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-14T08:59:12.710 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-14T08:59:12.794 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-14T08:59:12.939 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-14T08:59:12.994 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-14T08:59:13.224 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-14T08:59:13.227 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-14T08:59:13.302 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-14T08:59:13.450 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-03-14T08:59:13.484 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-14T08:59:13.626 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-14T08:59:13.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-14T08:59:13.726 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-14T08:59:13.813 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-14T08:59:13.987 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-14T08:59:14.057 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-03-14T08:59:14.185 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-14T08:59:14.298 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-03-14T08:59:14.457 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-14T08:59:14.496 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-03-14T08:59:14.570 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-03-14T08:59:14.690 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-14T08:59:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-03-14T08:59:14.900 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-14T08:59:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-03-14T08:59:15.106 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-03-14T08:59:15.131 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-14T08:59:15.288 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-14T08:59:15.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-14T08:59:15.491 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-14T08:59:15.535 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-14T08:59:15.745 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-14T08:59:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-14T08:59:15.953 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-14T08:59:16.029 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-14T08:59:16.196 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-14T08:59:16.209 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-14T08:59:16.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-14T08:59:16.391 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-14T08:59:16.606 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-14T08:59:16.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-14T08:59:16.846 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-14T08:59:16.867 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-14T08:59:17.081 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-14T08:59:17.120 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-14T08:59:17.266 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-14T08:59:17.375 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-14T08:59:17.435 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-14T08:59:17.634 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-14T08:59:17.651 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-14T08:59:17.808 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-14T08:59:17.865 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-14T08:59:18.050 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-14T08:59:18.107 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-14T08:59:18.400 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-14T08:59:18.628 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-14T08:59:18.828 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-14T08:59:19.070 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-14T08:59:19.117 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-14T08:59:19.564 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-14T08:59:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-14T08:59:20.142 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-14T08:59:20.158 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-14T08:59:20.376 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-14T08:59:20.500 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-14T08:59:20.592 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-14T08:59:20.745 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-14T08:59:21.123 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-14T08:59:21.389 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-14T08:59:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-14T08:59:21.583 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:18.2.2-825.g2f465319.el8.x86_64 103/162 2024-03-14T08:59:21.656 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:18.2.2-825.g2f465319.el8.x86_64 103/162 2024-03-14T08:59:21.711 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-14T08:59:21.884 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 104/162 2024-03-14T08:59:22.533 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-14T08:59:22.661 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:18.2.2-825.g2f465319.el8.x86_64 103/162 2024-03-14T08:59:22.771 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:18.2.2-825.g2f465319.el8.x86_64 103/162 2024-03-14T08:59:23.013 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 104/162 2024-03-14T08:59:25.669 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:25.925 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:26.502 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:26.777 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:26.778 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 106/162 2024-03-14T08:59:26.888 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 106/162 2024-03-14T08:59:27.041 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 107/162 2024-03-14T08:59:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:27.748 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 105/162 2024-03-14T08:59:28.062 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 106/162 2024-03-14T08:59:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 106/162 2024-03-14T08:59:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 107/162 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 107/162 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-03-14T08:59:46.479 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-03-14T08:59:46.480 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-03-14T08:59:46.480 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-03-14T08:59:46.480 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-03-14T08:59:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T08:59:46.863 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 108/162 2024-03-14T08:59:47.717 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 107/162 2024-03-14T08:59:47.717 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /dev 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /run 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /dev 2024-03-14T08:59:47.718 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T08:59:48.107 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 108/162 2024-03-14T08:59:48.436 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 108/162 2024-03-14T08:59:48.648 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 109/162 2024-03-14T08:59:48.866 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 109/162 2024-03-14T08:59:49.062 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 110/162 2024-03-14T08:59:49.519 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 110/162 2024-03-14T08:59:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 108/162 2024-03-14T08:59:49.704 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 111/162 2024-03-14T08:59:49.722 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 111/162 2024-03-14T08:59:49.925 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 109/162 2024-03-14T08:59:50.143 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 109/162 2024-03-14T08:59:50.323 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 110/162 2024-03-14T08:59:50.781 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 110/162 2024-03-14T08:59:50.973 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 111/162 2024-03-14T08:59:50.991 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 111/162 2024-03-14T08:59:51.512 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:18.2.2-825.g2f465319.el8.noarch 112/162 2024-03-14T08:59:51.684 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:18.2.2-825.g2f465319.el8.noarch 112/162 2024-03-14T08:59:51.810 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el 113/162 2024-03-14T08:59:51.962 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-825.g2f465319.e 114/162 2024-03-14T08:59:52.211 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 115/162 2024-03-14T08:59:52.446 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 115/162 2024-03-14T08:59:52.756 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 116/162 2024-03-14T08:59:52.778 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:18.2.2-825.g2f465319.el8.noarch 112/162 2024-03-14T08:59:52.969 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:18.2.2-825.g2f465319.el8.noarch 112/162 2024-03-14T08:59:53.096 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el 113/162 2024-03-14T08:59:53.240 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-825.g2f465319.e 114/162 2024-03-14T08:59:53.489 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 115/162 2024-03-14T08:59:53.713 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 115/162 2024-03-14T08:59:53.719 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 116/162 2024-03-14T08:59:53.878 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 117/162 2024-03-14T08:59:53.970 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 117/162 2024-03-14T08:59:54.065 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 116/162 2024-03-14T08:59:54.293 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8 118/162 2024-03-14T08:59:54.465 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 119/162 2024-03-14T08:59:54.502 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 119/162 2024-03-14T08:59:54.677 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 120/162 2024-03-14T08:59:54.690 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 120/162 2024-03-14T08:59:55.034 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 116/162 2024-03-14T08:59:55.206 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 117/162 2024-03-14T08:59:55.300 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 117/162 2024-03-14T08:59:55.636 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8 118/162 2024-03-14T08:59:55.799 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 119/162 2024-03-14T08:59:55.836 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 119/162 2024-03-14T08:59:55.934 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:18.2.2-825.g2f465319.el8.x86_64 121/162 2024-03-14T08:59:55.989 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 120/162 2024-03-14T08:59:56.001 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 120/162 2024-03-14T08:59:56.138 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 122/162 2024-03-14T08:59:56.173 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 122/162 2024-03-14T08:59:56.328 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-825.g2f4653 123/162 2024-03-14T08:59:56.371 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-825.g2f4653 123/162 2024-03-14T08:59:56.553 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 124/162 2024-03-14T08:59:56.899 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 124/162 2024-03-14T08:59:57.053 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 125/162 2024-03-14T08:59:57.280 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:18.2.2-825.g2f465319.el8.x86_64 121/162 2024-03-14T08:59:57.498 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 122/162 2024-03-14T08:59:57.533 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 122/162 2024-03-14T08:59:57.714 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-825.g2f4653 123/162 2024-03-14T08:59:57.756 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-825.g2f4653 123/162 2024-03-14T08:59:57.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 124/162 2024-03-14T08:59:58.253 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 124/162 2024-03-14T08:59:58.406 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 125/162 2024-03-14T09:00:01.474 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 125/162 2024-03-14T09:00:01.952 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 126/162 2024-03-14T09:00:02.295 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-14T09:00:02.501 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-14T09:00:02.826 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 125/162 2024-03-14T09:00:03.184 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-14T09:00:03.317 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 126/162 2024-03-14T09:00:03.506 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-14T09:00:03.648 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-14T09:00:03.816 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-14T09:00:03.846 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-14T09:00:04.016 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 131/162 2024-03-14T09:00:04.270 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-14T09:00:04.464 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-14T09:00:04.541 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-14T09:00:04.635 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_6 134/162 2024-03-14T09:00:04.900 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-14T09:00:05.178 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-14T09:00:05.403 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 131/162 2024-03-14T09:00:05.606 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-14T09:00:05.808 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-14T09:00:06.021 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_6 134/162 2024-03-14T09:00:06.958 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:07.276 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:07.315 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:07.766 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-14T09:00:08.156 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-14T09:00:08.284 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:08.379 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-14T09:00:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:08.601 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 139/162 2024-03-14T09:00:08.602 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-14T09:00:08.823 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 140/162 2024-03-14T09:00:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-14T09:00:09.088 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 141/162 2024-03-14T09:00:09.377 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-14T09:00:09.412 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-14T09:00:09.583 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-14T09:00:09.595 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-14T09:00:09.699 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-14T09:00:09.754 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-14T09:00:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 139/162 2024-03-14T09:00:09.958 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-14T09:00:10.039 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-14T09:00:10.145 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 140/162 2024-03-14T09:00:10.292 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-14T09:00:10.306 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-14T09:00:10.446 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 141/162 2024-03-14T09:00:10.532 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-14T09:00:10.705 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-14T09:00:10.760 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-14T09:00:10.904 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-14T09:00:10.904 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-14T09:00:10.920 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-14T09:00:10.929 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-14T09:00:11.139 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-14T09:00:11.169 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-14T09:00:11.170 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-14T09:00:11.369 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-14T09:00:11.420 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-14T09:00:11.420 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-14T09:00:11.446 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-14T09:00:11.647 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-14T09:00:11.678 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-14T09:00:11.690 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-14T09:00:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-14T09:00:11.851 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-14T09:00:12.054 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-14T09:00:12.068 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-14T09:00:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-14T09:00:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-14T09:00:12.294 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-14T09:00:12.294 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-14T09:00:12.539 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-14T09:00:12.539 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-14T09:00:12.544 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-14T09:00:12.544 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-14T09:00:12.747 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-14T09:00:12.748 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-14T09:00:12.794 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-14T09:00:12.794 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-14T09:00:12.991 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-14T09:00:13.027 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-14T09:00:13.027 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-14T09:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-14T09:00:13.218 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-14T09:00:13.237 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-14T09:00:13.411 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-14T09:00:13.477 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-14T09:00:13.643 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-14T09:00:13.644 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-14T09:00:13.654 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-14T09:00:13.654 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-14T09:00:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-14T09:00:13.930 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-14T09:00:14.180 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-14T09:00:14.180 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-14T09:00:14.447 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-14T09:00:14.447 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:18.2.2-825.g2f465319.el8.x86_64 1/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 2/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 3/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 4/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-825.g2f4653 5/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 6/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 7/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 8/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T09:00:14.571 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 12/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 13/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_6 14/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 15/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 16/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 17/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 18/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:18.2.2-825.g2f465319.el8 20/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:18.2.2-825.g2f465319.el8.x 21/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 22/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 23/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 24/162 2024-03-14T09:00:14.572 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 25/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 26/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 27/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 28/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-825.g2f465319.e 29/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 30/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 31/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 32/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8 33/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 34/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el 35/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 36/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:18.2.2-825.g2f465319.el8.noarch 37/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 38/162 2024-03-14T09:00:14.573 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-14T09:00:14.574 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-14T09:00:14.575 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-14T09:00:14.576 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-14T09:00:14.577 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-14T09:00:14.578 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-14T09:00:14.579 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:18.2.2-825.g2f465319.el8.x86_64 131/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-14T09:00:14.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:18.2.2-825.g2f465319.el8.x86_64 133/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-14T09:00:14.581 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-14T09:00:14.582 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-14T09:00:14.705 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-14T09:00:14.705 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-14T09:00:14.971 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-14T09:00:15.155 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-14T09:00:15.155 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-14T09:00:16.091 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:18.2.2-825.g2f465319.el8.x86_64 1/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 2/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 3/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 4/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-825.g2f4653 5/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 6/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 7/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 8/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 9/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 10/162 2024-03-14T09:00:16.092 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 11/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 12/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 13/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_6 14/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 15/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 16/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 17/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_ 18/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:18.2.2-825.g2f465319.el8.x86_64 19/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:18.2.2-825.g2f465319.el8 20/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:18.2.2-825.g2f465319.el8.x 21/162 2024-03-14T09:00:16.093 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 22/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 23/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 24/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 25/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 26/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 27/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 28/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-825.g2f465319.e 29/162 2024-03-14T09:00:16.094 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noar 30/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.no 31/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-825.g2f46 32/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8 33/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 34/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el 35/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 36/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:18.2.2-825.g2f465319.el8.noarch 37/162 2024-03-14T09:00:16.107 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 38/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-14T09:00:16.108 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-14T09:00:16.109 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-03-14T09:00:16.110 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-03-14T09:00:16.111 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-03-14T09:00:16.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-03-14T09:00:16.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-03-14T09:00:16.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-03-14T09:00:16.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-03-14T09:00:16.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-14T09:00:16.113 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-14T09:00:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:18.2.2-825.g2f465319.el8.x86_64 131/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-14T09:00:16.115 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:18.2.2-825.g2f465319.el8.x86_64 133/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-14T09:00:16.116 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-14T09:00:16.117 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-14T09:00:20.809 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-14T09:00:20.809 INFO:teuthology.orchestra.run.smithi191.stdout:Installed products updated. 2024-03-14T09:00:22.198 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:22.198 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-03-14T09:00:22.198 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-14T09:00:22.198 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-14T09:00:22.198 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-14T09:00:22.199 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.1-8.el8.noarch 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.0.4-10.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.200 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-14T09:00:22.201 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.1-9.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-7.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.202 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libpq-13.11-1.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-14T09:00:22.203 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:22.204 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-14T09:00:22.205 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-14T09:00:22.206 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.207 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-14T09:00:22.208 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-14T09:00:22.209 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-14T09:00:22.210 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-14T09:00:22.210 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:22.210 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-03-14T09:00:22.421 DEBUG:teuthology.parallel:result is None 2024-03-14T09:00:22.520 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-14T09:00:22.520 INFO:teuthology.orchestra.run.smithi154.stdout:Installed products updated. 2024-03-14T09:00:23.652 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:23.653 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.1-8.el8.noarch 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.0.4-10.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.654 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.655 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:18.2.2-825.g2f465319.el8.noarch 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.1-9.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-7.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-14T09:00:23.656 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libpq-13.11-1.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-14T09:00:23.657 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-14T09:00:23.658 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.659 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-14T09:00:23.660 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-2.6-1.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-14T09:00:23.661 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.662 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-14T09:00:23.663 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:18.2.2-825.g2f465319.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-14T09:00:23.664 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:23.665 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-03-14T09:00:23.902 DEBUG:teuthology.parallel:result is None 2024-03-14T09:00:23.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f465319e3c83d48f999325459bcdfd0346449c0 2024-03-14T09:00:24.045 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T09:00:24.069 INFO:teuthology.orchestra.run.smithi154.stdout:18.2.2-825.g2f465319.el8 2024-03-14T09:00:24.070 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-825.g2f465319.el8 2024-03-14T09:00:24.070 INFO:teuthology.task.install:The correct ceph version 18.2.2-825.g2f465319 is installed. 2024-03-14T09:00:24.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f465319e3c83d48f999325459bcdfd0346449c0 2024-03-14T09:00:24.212 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-14T09:00:24.241 INFO:teuthology.orchestra.run.smithi191.stdout:18.2.2-825.g2f465319.el8 2024-03-14T09:00:24.241 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-825.g2f465319.el8 2024-03-14T09:00:24.241 INFO:teuthology.task.install:The correct ceph version 18.2.2-825.g2f465319 is installed. 2024-03-14T09:00:24.243 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-14T09:00:24.243 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:24.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T09:00:24.285 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:24.285 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-14T09:00:24.332 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-14T09:00:24.333 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:24.333 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T09:00:24.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T09:00:24.428 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:24.428 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-03-14T09:00:24.458 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-14T09:00:24.528 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-14T09:00:24.528 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:24.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T09:00:24.555 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T09:00:24.622 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:24.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-14T09:00:24.650 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-14T09:00:24.717 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-14T09:00:24.717 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:24.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T09:00:24.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T09:00:24.811 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:24.811 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-03-14T09:00:24.839 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-14T09:00:24.905 INFO:teuthology.run_tasks:Running task ceph... 2024-03-14T09:00:24.973 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-14T09:00:24.973 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-03-14T09:00:24.975 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2024-03-14T09:00:25.002 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-14T09:00:25.002 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T09:00:25.043 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-14T09:00:25.069 INFO:tasks.ceph:Creating extra log directories... 2024-03-14T09:00:25.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T09:00:25.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-14T09:00:25.144 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-14T09:00:25.144 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-14T09:00:25.144 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-14T09:00:25.144 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:25.144 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T09:00:25.191 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:25.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:25.333 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:25.333 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-14T09:00:25.350 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:25.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:25.492 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-14T09:00:25.492 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-14T09:00:25.492 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661', 'branch': 'wip-yuri-testing-2024-03-12-1240-reef', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7598661', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2024-03-12-1240-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2024-03-12-1240-reef', 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 8503, 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2024-03-12-1240-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '2f465319e3c83d48f999325459bcdfd0346449c0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAh63oA83n0cc4tnDfIENESbeLZfvffjLhjv3u9kdLTpFVdZjgr4jWfTSVyFijT5tRxuy1fSLG+09VqENeQ8GGI=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAatpitHNt+TFVbP3UicmJSYlTf30fDW9oPsOZKeenJamK3ncp0pbi6YADcnolpGBU9msxgnPutue05jBsV9C0s='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '2f465319e3c83d48f999325459bcdfd0346449c0', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_multifs_auth']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-13_19:31:43', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-14T09:00:25.493 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T09:00:25.496 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-14T09:00:25.544 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T09:00:25.552 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2024-03-14T09:00:25.610 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:25.610 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T09:00:25.634 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T09:00:25.634 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 179538 Links: 1 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-14 09:00:15.721052690 +0000 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-14 08:56:56.699809721 +0000 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-14 08:56:56.699809721 +0000 2024-03-14T09:00:25.692 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-03-14T09:00:25.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T09:00:25.757 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-14T09:00:25.758 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-14T09:00:25.758 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000259144 s, 2.0 MB/s 2024-03-14T09:00:25.759 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T09:00:25.817 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-03-14T09:00:25.874 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 180485 Links: 1 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-14 09:00:15.721052690 +0000 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-14 08:56:57.052800240 +0000 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-14 08:56:57.052800240 +0000 2024-03-14T09:00:25.875 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-03-14T09:00:25.875 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T09:00:25.939 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-14T09:00:25.939 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-14T09:00:25.939 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000230435 s, 2.2 MB/s 2024-03-14T09:00:25.941 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T09:00:25.998 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-03-14T09:00:26.055 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 179619 Links: 1 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-14 09:00:15.721052690 +0000 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-14 08:56:57.405790760 +0000 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-14 08:56:57.405790760 +0000 2024-03-14T09:00:26.056 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-03-14T09:00:26.056 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T09:00:26.121 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-14T09:00:26.121 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-14T09:00:26.121 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000187644 s, 2.7 MB/s 2024-03-14T09:00:26.122 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T09:00:26.180 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-03-14T09:00:26.237 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T09:00:26.237 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.237 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 175010 Links: 1 2024-03-14T09:00:26.237 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.238 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.238 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-03-14 09:00:15.721052690 +0000 2024-03-14T09:00:26.238 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-03-14 08:56:57.770780957 +0000 2024-03-14T09:00:26.238 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-03-14 08:56:57.770780957 +0000 2024-03-14T09:00:26.238 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2024-03-14T09:00:26.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T09:00:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-03-14T09:00:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-03-14T09:00:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000224721 s, 2.3 MB/s 2024-03-14T09:00:26.304 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T09:00:26.362 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:26.362 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:26.363 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-03-14T09:00:26.382 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-14T09:00:26.382 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 181379 Links: 1 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.440 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-14 09:00:14.175854085 +0000 2024-03-14T09:00:26.441 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-14 08:56:58.998836946 +0000 2024-03-14T09:00:26.441 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-14 08:56:58.998836946 +0000 2024-03-14T09:00:26.441 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-03-14T09:00:26.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-14T09:00:26.508 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-14T09:00:26.508 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-14T09:00:26.508 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000214857 s, 2.4 MB/s 2024-03-14T09:00:26.509 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-14T09:00:26.569 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-03-14T09:00:26.626 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 178979 Links: 1 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-14 09:00:14.175854085 +0000 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-14 08:56:59.361830942 +0000 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-14 08:56:59.361830942 +0000 2024-03-14T09:00:26.627 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-03-14T09:00:26.627 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-14T09:00:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-14T09:00:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-14T09:00:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000221853 s, 2.3 MB/s 2024-03-14T09:00:26.694 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-14T09:00:26.752 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-03-14T09:00:26.810 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-14T09:00:26.810 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.810 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 180538 Links: 1 2024-03-14T09:00:26.810 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.810 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.811 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-14 09:00:14.175854085 +0000 2024-03-14T09:00:26.811 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-14 08:56:59.708825203 +0000 2024-03-14T09:00:26.811 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-14 08:56:59.708825203 +0000 2024-03-14T09:00:26.811 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-03-14T09:00:26.811 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-14T09:00:26.876 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-14T09:00:26.876 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-14T09:00:26.876 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000181559 s, 2.8 MB/s 2024-03-14T09:00:26.877 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-14T09:00:26.935 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 180611 Links: 1 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-14 09:00:14.175854085 +0000 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-14 08:57:00.053819498 +0000 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-14 08:57:00.053819498 +0000 2024-03-14T09:00:26.993 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-03-14T09:00:26.993 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-14T09:00:27.058 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-14T09:00:27.058 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-14T09:00:27.058 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000232506 s, 2.2 MB/s 2024-03-14T09:00:27.060 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-14T09:00:27.117 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:27.117 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T09:00:27.118 INFO:tasks.ceph:Generating config... 2024-03-14T09:00:27.119 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-14T09:00:27.119 INFO:tasks.ceph:[client] debug client = 20 2024-03-14T09:00:27.119 INFO:tasks.ceph:[client] debug ms = 1 2024-03-14T09:00:27.119 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-14T09:00:27.119 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-14T09:00:27.119 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-14T09:00:27.119 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-14T09:00:27.119 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-14T09:00:27.119 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-14T09:00:27.120 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-14T09:00:27.121 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-14T09:00:27.121 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-14T09:00:27.122 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-14T09:00:27.122 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-14T09:00:27.122 INFO:tasks.ceph:Setting up mon.a... 2024-03-14T09:00:27.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-14T09:00:27.182 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.keyring 2024-03-14T09:00:27.185 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-14T09:00:27.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T09:00:27.260 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.154'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2024-03-14T09:00:27.261 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.154,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-14T09:00:27.261 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:27.262 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T09:00:27.318 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.154 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:27.405 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: generated fsid 661f6023-01e4-4d6e-b4b3-5b5701ce7cc5 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:setting min_mon_release = pacific 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:epoch 0 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:fsid 661f6023-01e4-4d6e-b4b3-5b5701ce7cc5 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:last_changed 2024-03-14T09:00:27.407367+0000 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:created 2024-03-14T09:00:27.407367+0000 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:min_mon_release 16 (pacific) 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:election_strategy: 1 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:0: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.a 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2024-03-14T09:00:27.406 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-14T09:00:27.408 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-14T09:00:27.424 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 661f6023-01e4-4d6e-b4b3-5b5701ce7cc5... 2024-03-14T09:00:27.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T09:00:27.467 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on skipped message = False 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 661f6023-01e4-4d6e-b4b3-5b5701ce7cc5 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-03-14T09:00:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2024-03-14T09:00:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-03-14T09:00:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-03-14T09:00:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-03-14T09:00:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-03-14T09:00:27.518 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2024-03-14T09:00:27.519 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2024-03-14T09:00:27.520 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.521 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-14T09:00:27.522 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.154,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on skipped message = False 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 661f6023-01e4-4d6e-b4b3-5b5701ce7cc5 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.523 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.524 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2024-03-14T09:00:27.525 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-03-14T09:00:27.526 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.527 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2024-03-14T09:00:27.528 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-03-14T09:00:27.529 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds balancer = 20 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2024-03-14T09:00:27.530 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2024-03-14T09:00:27.532 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-14T09:00:27.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-14T09:00:27.629 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-14T09:00:27.630 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:27.630 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-14T09:00:27.647 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:27.647 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-14T09:00:27.704 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:00:27.704 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:27.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T09:00:27.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T09:00:27.781 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:27.781 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:27.838 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:00:27.839 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:27.839 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-14T09:00:27.839 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-14T09:00:27.883 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:27.883 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:27.942 INFO:tasks.ceph:Setting up mon nodes... 2024-03-14T09:00:27.942 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-14T09:00:27.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T09:00:27.996 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-14T09:00:27.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T09:00:28.128 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-14T09:00:28.131 INFO:tasks.ceph:Setting up mds nodes... 2024-03-14T09:00:28.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T09:00:28.186 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-14T09:00:28.189 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-14T09:00:28.215 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T09:00:28.309 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-14T09:00:28.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-14T09:00:28.338 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T09:00:28.402 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-14T09:00:28.405 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-14T09:00:28.433 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-14T09:00:28.529 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-14T09:00:28.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-14T09:00:28.558 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-14T09:00:28.558 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-14T09:00:28.602 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-14T09:00:28.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-14T09:00:28.658 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-14T09:00:28.671 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-14T09:00:28.672 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-14T09:00:28.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-14T09:00:28.698 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:28.698 INFO:tasks.ceph:role: osd.0 2024-03-14T09:00:28.698 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:00:28.698 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T09:00:28.772 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:28.772 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:28.772 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:28.772 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:28.773 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:29.204 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-14T09:00:29.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-14T09:00:29.207 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-14T09:00:29.394 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-14T09:00:29.430 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-14T09:00:29.500 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:29.500 INFO:tasks.ceph:role: osd.1 2024-03-14T09:00:29.500 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:00:29.500 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:29.576 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:29.993 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-14T09:00:29.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-14T09:00:29.995 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-14T09:00:30.123 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-14T09:00:30.158 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-14T09:00:30.226 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:30.226 INFO:tasks.ceph:role: osd.2 2024-03-14T09:00:30.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:00:30.227 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T09:00:30.302 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:30.302 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:30.302 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:30.303 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:30.741 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-14T09:00:30.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-14T09:00:30.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-14T09:00:30.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-14T09:00:30.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-14T09:00:31.009 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:31.009 INFO:tasks.ceph:role: osd.3 2024-03-14T09:00:31.010 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:00:31.010 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:31.086 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:31.087 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:31.087 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:31.530 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-03-14T09:00:31.531 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-03-14T09:00:31.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-14T09:00:31.709 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-14T09:00:31.743 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:31.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.834+0000 7fb2d2a9b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-14T09:00:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.834+0000 7fb2d2a9b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-14T09:00:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.834+0000 7fb2d2a9b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T09:00:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.834+0000 7fb2d2a9b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T09:00:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.834+0000 7fb2d2a9b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-14T09:00:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:31.836+0000 7fb2d2a9b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-14T09:00:34.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-14T09:00:34.723 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.815+0000 7f48b8d92640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.815+0000 7f48b8d92640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.815+0000 7f48b8d92640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.815+0000 7f48b8d92640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.815+0000 7f48b8d92640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-14T09:00:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:34.816+0000 7f48b8d92640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-14T09:00:37.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-14T09:00:37.705 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.797+0000 7fb5ac84e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.797+0000 7fb5ac84e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.797+0000 7fb5ac84e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.797+0000 7fb5ac84e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.797+0000 7fb5ac84e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-14T09:00:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:37.798+0000 7fb5ac84e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-14T09:00:40.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-14T09:00:40.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:40.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.774+0000 7fb7c9342640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-14T09:00:40.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.775+0000 7fb7c9342640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-14T09:00:40.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.775+0000 7fb7c9342640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T09:00:40.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.775+0000 7fb7c9342640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T09:00:40.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.775+0000 7fb7c9342640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-14T09:00:40.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:00:40.775+0000 7fb7c9342640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-14T09:00:43.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-14T09:00:43.661 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-14T09:00:43.695 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:43.696 INFO:tasks.ceph:role: osd.4 2024-03-14T09:00:43.696 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:00:43.696 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-14T09:00:43.774 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:43.775 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:43.776 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:43.776 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:43.776 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:44.205 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-14T09:00:44.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-14T09:00:44.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-14T09:00:44.359 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-14T09:00:44.398 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-14T09:00:44.468 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:44.468 INFO:tasks.ceph:role: osd.5 2024-03-14T09:00:44.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:00:44.469 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-14T09:00:44.545 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:44.545 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:44.545 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:44.545 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:44.545 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:44.546 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:44.546 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:44.546 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:44.546 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:44.546 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:44.971 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-14T09:00:44.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-14T09:00:44.973 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-14T09:00:45.090 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-14T09:00:45.124 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-14T09:00:45.195 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:45.195 INFO:tasks.ceph:role: osd.6 2024-03-14T09:00:45.195 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:00:45.195 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-14T09:00:45.272 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:45.272 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:45.273 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:45.720 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-14T09:00:45.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-14T09:00:45.722 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-14T09:00:45.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-14T09:00:45.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-14T09:00:45.977 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-14T09:00:45.978 INFO:tasks.ceph:role: osd.7 2024-03-14T09:00:45.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:00:45.978 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-14T09:00:46.054 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-14T09:00:46.055 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-14T09:00:46.511 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-03-14T09:00:46.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-03-14T09:00:46.514 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-14T09:00:46.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-14T09:00:46.733 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:46.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.827+0000 7f44d579b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-14T09:00:46.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.828+0000 7f44d579b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-14T09:00:46.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.828+0000 7f44d579b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T09:00:46.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.828+0000 7f44d579b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T09:00:46.827 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.828+0000 7f44d579b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-14T09:00:46.828 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:46.828+0000 7f44d579b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-14T09:00:49.688 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-14T09:00:49.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.812+0000 7fbcc728e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.813+0000 7fbcc728e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.813+0000 7fbcc728e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.813+0000 7fbcc728e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.813+0000 7fbcc728e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-14T09:00:49.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:49.813+0000 7fbcc728e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-14T09:00:52.667 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-14T09:00:52.699 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.792+0000 7f51aa58d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.792+0000 7f51aa58d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.792+0000 7f51aa58d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.792+0000 7f51aa58d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.792+0000 7f51aa58d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-14T09:00:52.792 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:52.793+0000 7f51aa58d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-14T09:00:55.494 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:55.497 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:00:55.659 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-14T09:00:55.691 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:00:55.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.783+0000 7f08f39a0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-14T09:00:55.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.783+0000 7f08f39a0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-14T09:00:55.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.783+0000 7f08f39a0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T09:00:55.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.783+0000 7f08f39a0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T09:00:55.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.783+0000 7f08f39a0640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-14T09:00:55.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:00:55.784+0000 7f08f39a0640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-14T09:00:58.635 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-14T09:00:58.667 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-14T09:00:58.668 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-14T09:00:58.700 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-14T09:00:58.772 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.772 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-14T09:00:58.844 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-14T09:00:58.916 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-14T09:00:58.988 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:58.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-14T09:00:59.058 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:59.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-14T09:00:59.129 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-14T09:00:59.162 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.162 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-14T09:00:59.233 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-14T09:00:59.303 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-14T09:00:59.371 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-14T09:00:59.440 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.440 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-14T09:00:59.508 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.508 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-14T09:00:59.574 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:00:59.574 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-14T09:00:59.594 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:00:59.594 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-14T09:00:59.631 INFO:tasks.ceph:Adding keys to all mons... 2024-03-14T09:00:59.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T09:00:59.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCrvPJlQsV4OxAADoB6pH163AX2+5II4EtdFw== 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlnCs5CxAA/mf9z7eSw0Fh5yqCxmvhww== 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlvIuREhAAO1gOyqDP2tTJcb1IuOdQ0A== 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCvvPJlDzrPMRAAEAP7kHfAaxZGM+W0rY0DIA== 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-03-14T09:00:59.675 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCyvPJlgS2rMBAAoPtjrTbc7bMLMJAIUKD+vg== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC1vPJltO6ULxAAvkKiSoRBoKxFz8zPjGCOnA== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC4vPJlWIs/LhAAi5LaiHbS1svlQ0UesC9CLg== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlSRLFBxAAkIg2TMRbiwcdMggjl+vMWw== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlq18XGBAA9Nr/cXAtRbeaKmJDCLUz6w== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlAAuqHxAAy5SQz2lJHPwRad7HCIjjSg== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC+vPJl5TleMRAAjIGbvgOjoV3mJYIK1fQoYA== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDBvPJlhiV5MBAAe7XI8s0kWQRAhvWWqMInrg== 2024-03-14T09:00:59.676 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDEvPJlbghELxAASO+DuPJa7zXCH/KkhPi1WA== 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDHvPJlRba5LhAASpiL/U05lgta2f4xKra5ZQ== 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJlhDgFJBAA47spIEFGnChaoxmMQkaA+w== 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2024-03-14T09:00:59.677 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCsvPJl58xgJxAAfqdSCJED8MwHCScl1fqdFA== 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCrvPJlQsV4OxAADoB6pH163AX2+5II4EtdFw== 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlnCs5CxAA/mf9z7eSw0Fh5yqCxmvhww== 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlvIuREhAAO1gOyqDP2tTJcb1IuOdQ0A== 2024-03-14T09:00:59.697 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCvvPJlDzrPMRAAEAP7kHfAaxZGM+W0rY0DIA== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCyvPJlgS2rMBAAoPtjrTbc7bMLMJAIUKD+vg== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQC1vPJltO6ULxAAvkKiSoRBoKxFz8zPjGCOnA== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQC4vPJlWIs/LhAAi5LaiHbS1svlQ0UesC9CLg== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlSRLFBxAAkIg2TMRbiwcdMggjl+vMWw== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlq18XGBAA9Nr/cXAtRbeaKmJDCLUz6w== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlAAuqHxAAy5SQz2lJHPwRad7HCIjjSg== 2024-03-14T09:00:59.698 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQC+vPJl5TleMRAAjIGbvgOjoV3mJYIK1fQoYA== 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDBvPJlhiV5MBAAe7XI8s0kWQRAhvWWqMInrg== 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDEvPJlbghELxAASO+DuPJa7zXCH/KkhPi1WA== 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDHvPJlRba5LhAASpiL/U05lgta2f4xKra5ZQ== 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJlhDgFJBAA47spIEFGnChaoxmMQkaA+w== 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2024-03-14T09:00:59.699 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCsvPJl58xgJxAAfqdSCJED8MwHCScl1fqdFA== 2024-03-14T09:00:59.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T09:00:59.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T09:00:59.794 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:00:59.796 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:00:59.854 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:00:59.856 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:00:59.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:00:59.914 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:00:59.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:00:59.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.027 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.082 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.135 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T09:01:00.137 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-14T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:01:00.193 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:01:00.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:01:00.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-14T09:01:00.308 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.363 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.365 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.473 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.475 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-14T09:01:00.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T09:01:00.530 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T09:01:00.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T09:01:00.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-14T09:01:00.645 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-14T09:01:00.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-14T09:01:00.672 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T09:01:00.771 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-14T09:01:00.798 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-14T09:01:00.825 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T09:01:00.924 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-14T09:01:00.951 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-14T09:01:01.017 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-14T09:01:01.117 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-14T09:01:01.143 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:01:01.145 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-14T09:01:01.200 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-14T09:01:01.200 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-14T09:01:01.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-14T09:01:01.203 INFO:tasks.ceph.mon.a:Started 2024-03-14T09:01:01.203 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-14T09:01:01.203 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-14T09:01:01.243 INFO:tasks.ceph.mon.b:Started 2024-03-14T09:01:01.243 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-14T09:01:01.243 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-14T09:01:01.245 INFO:tasks.ceph.mon.c:Started 2024-03-14T09:01:01.245 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-14T09:01:01.245 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-14T09:01:01.245 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-14T09:01:01.247 INFO:tasks.ceph.mgr.y:Started 2024-03-14T09:01:01.248 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-14T09:01:01.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-14T09:01:01.250 INFO:tasks.ceph.mgr.x:Started 2024-03-14T09:01:01.250 INFO:tasks.ceph:Setting crush tunables to default 2024-03-14T09:01:01.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd crush tunables default 2024-03-14T09:01:01.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.375+0000 7f0627a97700 1 Processor -- start 2024-03-14T09:01:01.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.375+0000 7f0627a97700 1 -- start start 2024-03-14T09:01:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.375+0000 7f0627a97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f0620158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0627a97700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0627a97700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0620154f40 con 0x7f06200c1d50 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0627a97700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06200c16a0 con 0x7f06200bf9c0 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0627a97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0620158d80 con 0x7f06201553d0 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55466/0 (socket says 172.21.15.154:55466) 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 -- 172.21.15.154:0/2188608995 learned_addr learned my addr 172.21.15.154:0/2188608995 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 -- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 msgr2=0x7f06200bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 -- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 msgr2=0x7f06200bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 --2- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0626034700 1 --2- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f0620158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0625833700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 msgr2=0x7f0620158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-14T09:01:01.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0625833700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 msgr2=0x7f0620158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-14T09:01:01.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f0620158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-14T09:01:01.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.376+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f0620158840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-14T09:01:01.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015252322 0 0) 0x7f0620154f40 con 0x7f06200c1d50 2024-03-14T09:01:01.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060c0035b0 con 0x7f06200c1d50 2024-03-14T09:01:01.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206476227 0 0) 0x7f060c0035b0 con 0x7f06200c1d50 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 msgr2=0x7f0620158840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f0620158840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 msgr2=0x7f06200bfe20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 --2- 172.21.15.154:0/2188608995 >> v2:172.21.15.154:3300/0 conn(0x7f06200bf9c0 0x7f06200bfe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620154930 con 0x7f06200c1d50 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0614001670 con 0x7f06200c1d50 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 >> v1:172.21.15.191:6789/0 conn(0x7f06200c1d50 legacy=0x7f06200c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c003a00 0x7f060c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f060c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f060c004d90 0x7f060c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200c16a0 con 0x7f060c003a00 2024-03-14T09:01:01.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0620158d80 con 0x7f060c004d90 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.562+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c00bac0 con 0x7f06201553d0 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c003a00 0x7f060c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0626034700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f060c004d90 0x7f060c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625032700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f060c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 msgr2=0x7f060c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f060c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f060c004d90 msgr2=0x7f060c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f060c004d90 0x7f060c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 -- 172.21.15.154:0/2188608995 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0035b0 con 0x7f060c003a00 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625032700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06201553d0 0x7f060c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0625833700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c003a00 0x7f060c003e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f061c0098c0 tx=0x7f060c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ff620acd00aca server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00c070 con 0x7f060c003a00 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c004020 con 0x7f060c003a00 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.563+0000 7f0624831700 1 -- 172.21.15.154:0/2188608995 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0043d0 con 0x7f060c003a00 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 -- 172.21.15.154:0/2188608995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c003a00 msgr2=0x7f060c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:01.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 --2- 172.21.15.154:0/2188608995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f060c003a00 0x7f060c003e20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f061c0098c0 tx=0x7f060c005880 comp rx=0 tx=0).stop 2024-03-14T09:01:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0626034700 1 -- 172.21.15.154:0/2188608995 reap_dead start 2024-03-14T09:01:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 -- 172.21.15.154:0/2188608995 shutdown_connections 2024-03-14T09:01:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 -- 172.21.15.154:0/2188608995 >> 172.21.15.154:0/2188608995 conn(0x7f062006a770 msgr2=0x7f062006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 -- 172.21.15.154:0/2188608995 shutdown_connections 2024-03-14T09:01:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.564+0000 7f0627a97700 1 -- 172.21.15.154:0/2188608995 wait complete. 2024-03-14T09:01:01.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.566+0000 7f0627a97700 1 Processor -- start 2024-03-14T09:01:01.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.566+0000 7f0627a97700 1 -- start start 2024-03-14T09:01:01.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 0x7f06200a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 0x7f0620150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200aae10 con 0x7f062014ec10 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200aa800 con 0x7f06201df990 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0627a97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06201de680 con 0x7f060c003a00 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625833700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 0x7f06200a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0626034700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 0x7f0620150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625833700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 0x7f06200a9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:44174/0 (socket says 172.21.15.154:44174) 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0626034700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55506/0 (socket says 172.21.15.154:55506) 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625833700 1 -- 172.21.15.154:0/388983320 learned_addr learned my addr 172.21.15.154:0/388983320 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 -- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 msgr2=0x7f06200a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 0x7f06200a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 -- 172.21.15.154:0/388983320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 msgr2=0x7f062014f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 -- 172.21.15.154:0/388983320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06201de960 con 0x7f06201df990 2024-03-14T09:01:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0626034700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.567+0000 7f0625032700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 0x7f0620150ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f061400e780 tx=0x7f061400ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140052e0 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614005440 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06200aaaa0 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140038c0 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0620151420 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0614003d20 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.570+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f061401e780 con 0x7f06201df990 2024-03-14T09:01:01.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.571+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06200aaf80 con 0x7f06201df990 2024-03-14T09:01:01.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.573+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0614023060 con 0x7f06201df990 2024-03-14T09:01:01.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.668+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f06200697d0 con 0x7f06201df990 2024-03-14T09:01:01.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.672+0000 7f06127fc700 1 -- 172.21.15.154:0/388983320 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0614019070 con 0x7f06201df990 2024-03-14T09:01:01.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 msgr2=0x7f0620150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 0x7f0620150ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f061400e780 tx=0x7f061400ea90 comp rx=0 tx=0).stop 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 shutdown_connections 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f062014ec10 0x7f062014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f060c003a00 0x7f06200a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 --2- 172.21.15.154:0/388983320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06201df990 0x7f0620150ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 >> 172.21.15.154:0/388983320 conn(0x7f062006a770 msgr2=0x7f0620152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 shutdown_connections 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:01.673+0000 7f0627a97700 1 -- 172.21.15.154:0/388983320 wait complete. 2024-03-14T09:01:01.673 INFO:teuthology.orchestra.run.smithi154.stderr:adjusted tunables profile to default 2024-03-14T09:01:01.684 INFO:tasks.ceph:check_enable_crimson: False 2024-03-14T09:01:01.684 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-14T09:01:01.685 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:01.685 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-14T09:01:01.712 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:01.713 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-14T09:01:01.780 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:01.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-14T09:01:01.848 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:01.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-14T09:01:01.915 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:01.915 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-14T09:01:01.942 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:01.943 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-14T09:01:01.977 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:01.978+0000 7fb036387200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T09:01:02.000 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.000+0000 7ffa5c989200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-14T09:01:02.002 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:02.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-14T09:01:02.059 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.060+0000 7fb036387200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T09:01:02.070 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:02.070 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-14T09:01:02.079 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.079+0000 7ffa5c989200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-14T09:01:02.136 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 64076ce9-7a03-49dd-b455-6dd27cedc7cc 0 2024-03-14T09:01:02.136 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.137+0000 7fb036387200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T09:01:02.153 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.153+0000 7ffa5c989200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-14T09:01:02.209 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.210+0000 7fb036387200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T09:01:02.223 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.224+0000 7ffa5c989200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-14T09:01:02.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.300+0000 7f77bbac9700 1 Processor -- start 2024-03-14T09:01:02.300 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.301+0000 7f77bbac9700 1 -- start start 2024-03-14T09:01:02.301 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.301+0000 7f77bbac9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.301+0000 7f77bbac9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b413ade0 con 0x7f77b40c3250 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.301+0000 7f77bbac9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b406fb30 con 0x7f77b40a8540 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.301+0000 7f77bbac9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b406ec30 con 0x7f77b40c1e90 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b9865700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f77b40a8540 0x7f77b4154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35090/0 (socket says 172.21.15.191:35090) 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77ba066700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b9865700 1 -- 172.21.15.191:0/2811618377 learned_addr learned my addr 172.21.15.191:0/2811618377 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b8863700 1 -- 172.21.15.191:0/2811618377 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521976244 0 0) 0x7f77b406fb30 con 0x7f77b40a8540 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b8863700 1 -- 172.21.15.191:0/2811618377 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a00035b0 con 0x7f77b40a8540 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77ba066700 1 -- 172.21.15.191:0/2811618377 >> v1:172.21.15.191:6789/0 conn(0x7f77b40a8540 legacy=0x7f77b4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77ba066700 1 -- 172.21.15.191:0/2811618377 >> v1:172.21.15.154:6789/0 conn(0x7f77b40c3250 legacy=0x7f77b40c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77ba066700 1 -- 172.21.15.191:0/2811618377 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b406ef10 con 0x7f77b40c1e90 2024-03-14T09:01:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77ba066700 1 --2- 172.21.15.191:0/2811618377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b413a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009dc0 tx=0x7f77ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9134191ea1ec3ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b8863700 1 -- 172.21.15.191:0/2811618377 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00e070 con 0x7f77b40c1e90 2024-03-14T09:01:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b8863700 1 -- 172.21.15.191:0/2811618377 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac00bb20 con 0x7f77b40c1e90 2024-03-14T09:01:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.302+0000 7f77b8863700 1 -- 172.21.15.191:0/2811618377 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00be90 con 0x7f77b40c1e90 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 -- 172.21.15.191:0/2811618377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 msgr2=0x7f77b413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 --2- 172.21.15.191:0/2811618377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b413a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009dc0 tx=0x7f77ac00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 -- 172.21.15.191:0/2811618377 shutdown_connections 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 --2- 172.21.15.191:0/2811618377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b413a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 -- 172.21.15.191:0/2811618377 >> 172.21.15.191:0/2811618377 conn(0x7f77b40c3880 msgr2=0x7f77b40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 -- 172.21.15.191:0/2811618377 shutdown_connections 2024-03-14T09:01:02.304 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.303+0000 7f77bbac9700 1 -- 172.21.15.191:0/2811618377 wait complete. 2024-03-14T09:01:02.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.305+0000 7f77bbac9700 1 Processor -- start 2024-03-14T09:01:02.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.305+0000 7f77bbac9700 1 -- start start 2024-03-14T09:01:02.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b41e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77b40c3250 0x7f77b40aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41e8470 con 0x7f77b40c3250 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41ebeb0 con 0x7f77b40a8540 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77bbac9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41ec3e0 con 0x7f77b40c1e90 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77b9064700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b41e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77b9865700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77b9865700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38788/0 (socket says 172.21.15.191:38788) 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77b9064700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b41e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:57418/0 (socket says 172.21.15.191:57418) 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77b9865700 1 -- 172.21.15.191:0/662459125 learned_addr learned my addr 172.21.15.191:0/662459125 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:02.308 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.306+0000 7f77ba066700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77b40c3250 0x7f77b40aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 -- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 msgr2=0x7f77b41e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b41e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 -- 172.21.15.191:0/662459125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77b40c3250 msgr2=0x7f77b40aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77b40c3250 0x7f77b40aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 -- 172.21.15.191:0/662459125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac009a70 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77b9865700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c560 tx=0x7f77b000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0005d70 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b00070d0 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b40c5860 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0007480 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40ac8b0 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77b001c440 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.307+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77b0012740 con 0x7f77b40a8540 2024-03-14T09:01:02.309 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.308+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b40ac710 con 0x7f77b40a8540 2024-03-14T09:01:02.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.310+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f77b40ac710 con 0x7f77b40a8540 2024-03-14T09:01:02.391 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.392+0000 7fb036387200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T09:01:02.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.393+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "64076ce9-7a03-49dd-b455-6dd27cedc7cc", "id": 0} v 0) v1 -- 0x7f77b4155240 con 0x7f77b40a8540 2024-03-14T09:01:02.396 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.397+0000 7f77aa7fc700 1 -- 172.21.15.191:0/662459125 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64076ce9-7a03-49dd-b455-6dd27cedc7cc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77b0017070 con 0x7f77b40a8540 2024-03-14T09:01:02.396 INFO:teuthology.orchestra.run.smithi191.stdout:0 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 msgr2=0x7f77b414e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c560 tx=0x7f77b000c870 comp rx=0 tx=0).stop 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 shutdown_connections 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77b40c3250 0x7f77b40aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77b40a8540 0x7f77b414e870 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 --2- 172.21.15.191:0/662459125 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77b40c1e90 0x7f77b41e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.397 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 >> 172.21.15.191:0/662459125 conn(0x7f77b40c3880 msgr2=0x7f77b40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 shutdown_connections 2024-03-14T09:01:02.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.398+0000 7f77bbac9700 1 -- 172.21.15.191:0/662459125 wait complete. 2024-03-14T09:01:02.398 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.398+0000 7ffa5c989200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-14T09:01:02.408 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 310ba386-e95e-42f1-a244-e5ff86d37353 1 2024-03-14T09:01:02.562 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.563+0000 7ffa5c989200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T09:01:02.563 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.564+0000 7fb036387200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-14T09:01:02.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.573+0000 7ff49356b700 1 Processor -- start 2024-03-14T09:01:02.572 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.573+0000 7ff49356b700 1 -- start start 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff49356b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff48c14eaf0 0x7ff48c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff49356b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff48c156500 0x7ff48c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff49356b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0ac8e0 con 0x7ff48c06acb0 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff49356b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c06b310 con 0x7ff48c14eaf0 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff49356b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c06de30 con 0x7ff48c156500 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff48c156500 0x7ff48c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff490b06700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff48c06acb0 0x7ff48c155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35100/0 (socket says 172.21.15.191:35100) 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff490b06700 1 -- 172.21.15.191:0/1049616912 learned_addr learned my addr 172.21.15.191:0/1049616912 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491307700 1 --2- 172.21.15.191:0/1049616912 >> v2:172.21.15.154:3300/0 conn(0x7ff48c14eaf0 0x7ff48c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 -- 172.21.15.191:0/1049616912 >> v2:172.21.15.154:3300/0 conn(0x7ff48c14eaf0 msgr2=0x7ff48c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 --2- 172.21.15.191:0/1049616912 >> v2:172.21.15.154:3300/0 conn(0x7ff48c14eaf0 0x7ff48c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.575 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 -- 172.21.15.191:0/1049616912 >> v1:172.21.15.191:6789/0 conn(0x7ff48c06acb0 legacy=0x7ff48c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff483fff700 1 -- 172.21.15.191:0/1049616912 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966179288 0 0) 0x7ff48c0ac8e0 con 0x7ff48c06acb0 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 -- 172.21.15.191:0/1049616912 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c156910 con 0x7ff48c156500 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff491b08700 1 --2- 172.21.15.191:0/1049616912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c15f280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0c4920 tx=0x7ff48400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991903c00e28be15 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.574+0000 7ff483fff700 1 -- 172.21.15.191:0/1049616912 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400e070 con 0x7ff48c156500 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff483fff700 1 -- 172.21.15.191:0/1049616912 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48400bb20 con 0x7ff48c156500 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff483fff700 1 -- 172.21.15.191:0/1049616912 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484012430 con 0x7ff48c156500 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 -- 172.21.15.191:0/1049616912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 msgr2=0x7ff48c15f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 --2- 172.21.15.191:0/1049616912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c15f280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0c4920 tx=0x7ff48400b330 comp rx=0 tx=0).stop 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 -- 172.21.15.191:0/1049616912 shutdown_connections 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 --2- 172.21.15.191:0/1049616912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c15f280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 --2- 172.21.15.191:0/1049616912 >> v2:172.21.15.154:3300/0 conn(0x7ff48c14eaf0 0x7ff48c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 -- 172.21.15.191:0/1049616912 >> 172.21.15.191:0/1049616912 conn(0x7ff48c0c3880 msgr2=0x7ff48c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 -- 172.21.15.191:0/1049616912 shutdown_connections 2024-03-14T09:01:02.576 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.575+0000 7ff49356b700 1 -- 172.21.15.191:0/1049616912 wait complete. 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.576+0000 7ff49356b700 1 Processor -- start 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.576+0000 7ff49356b700 1 -- start start 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c06acb0 0x7ff48c1e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 0x7ff48c1de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c158e60 con 0x7ff48c06acb0 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c1e07a0 con 0x7ff48c156500 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff49356b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c1ee8c0 con 0x7ff48c14eaf0 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.579 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff490b06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 0x7ff48c1de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491307700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c06acb0 0x7ff48c1e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38804/0 (socket says 172.21.15.191:38804) 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 -- 172.21.15.191:0/3519803032 learned_addr learned my addr 172.21.15.191:0/3519803032 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 -- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 msgr2=0x7ff48c1de8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 0x7ff48c1de8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 -- 172.21.15.191:0/3519803032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c06acb0 msgr2=0x7ff48c1e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c06acb0 0x7ff48c1e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff491b08700 1 -- 172.21.15.191:0/3519803032 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484009a70 con 0x7ff48c156500 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.577+0000 7ff490b06700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 0x7ff48c1de8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff491b08700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff48c06df70 tx=0x7ff484004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484012430 con 0x7ff48c156500 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484020070 con 0x7ff48c156500 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c1588a0 con 0x7ff48c156500 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484004280 con 0x7ff48c156500 2024-03-14T09:01:02.580 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c1dee00 con 0x7ff48c156500 2024-03-14T09:01:02.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff4840046e0 con 0x7ff48c156500 2024-03-14T09:01:02.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.578+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff484022070 con 0x7ff48c156500 2024-03-14T09:01:02.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.579+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c1e0550 con 0x7ff48c156500 2024-03-14T09:01:02.581 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.581+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff48402c060 con 0x7ff48c156500 2024-03-14T09:01:02.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.665+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "310ba386-e95e-42f1-a244-e5ff86d37353", "id": 1} v 0) v1 -- 0x7ff48c1e0910 con 0x7ff48c156500 2024-03-14T09:01:02.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.673+0000 7ff481ffb700 1 -- 172.21.15.191:0/3519803032 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "310ba386-e95e-42f1-a244-e5ff86d37353", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff484012e20 con 0x7ff48c156500 2024-03-14T09:01:02.672 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 msgr2=0x7ff48c1e0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff48c06df70 tx=0x7ff484004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 shutdown_connections 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c156500 0x7ff48c1e0c30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c06acb0 0x7ff48c1e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 --2- 172.21.15.191:0/3519803032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff48c14eaf0 0x7ff48c1de8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 >> 172.21.15.191:0/3519803032 conn(0x7ff48c0c3880 msgr2=0x7ff48c0c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 shutdown_connections 2024-03-14T09:01:02.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.674+0000 7ff49356b700 1 -- 172.21.15.191:0/3519803032 wait complete. 2024-03-14T09:01:02.675 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.676+0000 7ffa5c989200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T09:01:02.680 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.681+0000 7fb036387200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-14T09:01:02.685 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 5c46843b-b6e8-43ac-8dcf-8c0f72917016 2 2024-03-14T09:01:02.769 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.770+0000 7ffa5c989200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T09:01:02.780 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.781+0000 7fb036387200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-14T09:01:02.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.808+0000 7f3d621a8700 1 Processor -- start 2024-03-14T09:01:02.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.808+0000 7f3d621a8700 1 -- start start 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d621a8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d621a8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 0x7f3d5c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d621a8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0acd10 con 0x7f3d5c0702f0 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d621a8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c150110 con 0x7f3d5c0c5d60 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d621a8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c66b0 con 0x7f3d5c0c4380 2024-03-14T09:01:02.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d5affd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d5affd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38810/0 (socket says 172.21.15.191:38810) 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d5affd700 1 -- 172.21.15.191:0/3316172070 learned_addr learned my addr 172.21.15.191:0/3316172070 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.809+0000 7f3d5bfff700 1 --2- 172.21.15.191:0/3316172070 >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 0x7f3d5c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5affd700 1 -- 172.21.15.191:0/3316172070 >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 msgr2=0x7f3d5c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5affd700 1 --2- 172.21.15.191:0/3316172070 >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 0x7f3d5c0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5affd700 1 -- 172.21.15.191:0/3316172070 >> v1:172.21.15.154:6789/0 conn(0x7f3d5c0702f0 legacy=0x7f3d5c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5affd700 1 -- 172.21.15.191:0/3316172070 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0ac9c0 con 0x7f3d5c0c4380 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5bfff700 1 --2- 172.21.15.191:0/3316172070 >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 0x7f3d5c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5affd700 1 --2- 172.21.15.191:0/3316172070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0c5980 tx=0x7f3d44009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3ff9b69d473181 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5a7fc700 1 -- 172.21.15.191:0/3316172070 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44011830 con 0x7f3d5c0c4380 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5a7fc700 1 -- 172.21.15.191:0/3316172070 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44011990 con 0x7f3d5c0c4380 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.810+0000 7f3d5a7fc700 1 -- 172.21.15.191:0/3316172070 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44011d20 con 0x7f3d5c0c4380 2024-03-14T09:01:02.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 -- 172.21.15.191:0/3316172070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0c4380 msgr2=0x7f3d5c14fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 --2- 172.21.15.191:0/3316172070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0c5980 tx=0x7f3d44009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 -- 172.21.15.191:0/3316172070 shutdown_connections 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 --2- 172.21.15.191:0/3316172070 >> v2:172.21.15.154:3300/0 conn(0x7f3d5c0c5d60 0x7f3d5c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 --2- 172.21.15.191:0/3316172070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0c4380 0x7f3d5c14fbd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 -- 172.21.15.191:0/3316172070 >> 172.21.15.191:0/3316172070 conn(0x7f3d5c0a9bf0 msgr2=0x7f3d5c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 -- 172.21.15.191:0/3316172070 shutdown_connections 2024-03-14T09:01:02.812 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.811+0000 7f3d621a8700 1 -- 172.21.15.191:0/3316172070 wait complete. 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.812+0000 7f3d621a8700 1 Processor -- start 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.812+0000 7f3d621a8700 1 -- start start 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c0c5d60 0x7f3d5c1eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1553e0 con 0x7f3d5c0c5d60 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1559a0 con 0x7f3d5c0702f0 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c151430 con 0x7f3d5c0c4380 2024-03-14T09:01:02.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38816/0 (socket says 172.21.15.191:38816) 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:57432/0 (socket says 172.21.15.191:57432) 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 -- 172.21.15.191:0/1922662817 learned_addr learned my addr 172.21.15.191:0/1922662817 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5bfff700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c0c5d60 0x7f3d5c1eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 -- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 msgr2=0x7f3d5c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 -- 172.21.15.191:0/1922662817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c0c5d60 msgr2=0x7f3d5c1eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c0c5d60 0x7f3d5c1eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 -- 172.21.15.191:0/1922662817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009a70 con 0x7f3d5c0702f0 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5affd700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d5b7fe700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c510 tx=0x7f3d5400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:02.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d540052d0 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54005430 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d44009550 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.813+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d540038c0 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.814+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c1ec3e0 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.814+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3d54003d20 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.814+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f3d54011d60 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.814+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c155c80 con 0x7f3d5c0702f0 2024-03-14T09:01:02.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.816+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f3d54017070 con 0x7f3d5c0702f0 2024-03-14T09:01:02.846 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:02.847+0000 7ffa5c989200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T09:01:02.860 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:02.861+0000 7fb036387200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-14T09:01:02.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.900+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5c46843b-b6e8-43ac-8dcf-8c0f72917016", "id": 2} v 0) v1 -- 0x7f3d5c0697d0 con 0x7f3d5c0702f0 2024-03-14T09:01:02.907 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.908+0000 7f3d53fff700 1 -- 172.21.15.191:0/1922662817 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c46843b-b6e8-43ac-8dcf-8c0f72917016", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d54011590 con 0x7f3d5c0702f0 2024-03-14T09:01:02.907 INFO:teuthology.orchestra.run.smithi191.stdout:2 2024-03-14T09:01:02.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 msgr2=0x7f3d5c150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:02.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c510 tx=0x7f3d5400c820 comp rx=0 tx=0).stop 2024-03-14T09:01:02.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 shutdown_connections 2024-03-14T09:01:02.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d5c0c5d60 0x7f3d5c1eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.908 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5c0702f0 0x7f3d5c150ef0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 --2- 172.21.15.191:0/1922662817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5c0c4380 0x7f3d5c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:02.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 >> 172.21.15.191:0/1922662817 conn(0x7f3d5c0a9bf0 msgr2=0x7f3d5c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:02.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 shutdown_connections 2024-03-14T09:01:02.909 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:02.909+0000 7f3d621a8700 1 -- 172.21.15.191:0/1922662817 wait complete. 2024-03-14T09:01:02.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new d5110594-4610-435f-a8b3-cd63e4717693 3 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.061+0000 7f9983756700 1 Processor -- start 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.061+0000 7f9983756700 1 -- start start 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.062+0000 7f9983756700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f997c15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.062+0000 7f9983756700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c071030 con 0x7f997c0708f0 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.062+0000 7f9983756700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c072520 con 0x7f997c06c1d0 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.062+0000 7f9983756700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c06e0d0 con 0x7f997c161d70 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.063+0000 7f99814f2700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f997c06c1d0 0x7f997c0701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35116/0 (socket says 172.21.15.191:35116) 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.063+0000 7f99814f2700 1 -- 172.21.15.191:0/2825985096 learned_addr learned my addr 172.21.15.191:0/2825985096 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-14T09:01:03.064 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.063+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099205818 0 0) 0x7f997c072520 con 0x7f997c06c1d0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9968003a00 con 0x7f997c06c1d0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9981cf3700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f997c15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545816571 0 0) 0x7f997c071030 con 0x7f997c0708f0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f997c072520 con 0x7f997c0708f0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 724225668 0 0) 0x7f9968003a00 con 0x7f997c06c1d0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 msgr2=0x7f997c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f997c15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 >> v1:172.21.15.154:6789/0 conn(0x7f997c0708f0 legacy=0x7f997c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c070d90 con 0x7f997c06c1d0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f99740019f0 con 0x7f997c06c1d0 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 >> v1:172.21.15.191:6789/0 conn(0x7f997c06c1d0 legacy=0x7f997c0701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f9968004ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680051e0 0x7f9968005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.065 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968003c90 0x7f996800be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c06e0d0 con 0x7f9968003c90 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680043f0 con 0x7f99680051e0 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680045b0 con 0x7f997c161d70 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f9968004ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9981cf3700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968003c90 0x7f996800be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f9980cf1700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680051e0 0x7f9968005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 -- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680051e0 msgr2=0x7f9968005660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680051e0 0x7f9968005660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 -- 172.21.15.191:0/2825985096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968003c90 msgr2=0x7f996800be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968003c90 0x7f996800be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.064+0000 7f99814f2700 1 -- 172.21.15.191:0/2825985096 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968003a00 con 0x7f997c161d70 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.065+0000 7f9980cf1700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680051e0 0x7f9968005660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.065+0000 7f9981cf3700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968003c90 0x7f996800be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:03.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.065+0000 7f99814f2700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f9968004ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f997400adb0 tx=0x7f9968005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca34e84786330e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.065+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400a850 con 0x7f997c161d70 2024-03-14T09:01:03.067 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.065+0000 7f9973fff700 1 -- 172.21.15.191:0/2825985096 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400cc60 con 0x7f997c161d70 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 -- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 msgr2=0x7f9968004ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 --2- 172.21.15.191:0/2825985096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c161d70 0x7f9968004ca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f997400adb0 tx=0x7f9968005cd0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9981cf3700 1 -- 172.21.15.191:0/2825985096 reap_dead start 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 -- 172.21.15.191:0/2825985096 shutdown_connections 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 -- 172.21.15.191:0/2825985096 >> 172.21.15.191:0/2825985096 conn(0x7f997c0c2970 msgr2=0x7f997c0c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 -- 172.21.15.191:0/2825985096 shutdown_connections 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.066+0000 7f9983756700 1 -- 172.21.15.191:0/2825985096 wait complete. 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.068+0000 7f9983756700 1 Processor -- start 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.068+0000 7f9983756700 1 -- start start 2024-03-14T09:01:03.068 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 0x7f997c1f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f997c1fb3f0 0x7f997c1f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0aac70 con 0x7f997c161d70 2024-03-14T09:01:03.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0aa990 con 0x7f997c1fb3f0 2024-03-14T09:01:03.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9983756700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c1fa910 con 0x7f997c0ab1a0 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.191:50580/0 (socket says 172.21.15.191:50580) 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 -- 172.21.15.191:0/1390199470 learned_addr learned my addr 172.21.15.191:0/1390199470 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9981cf3700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f997c1fb3f0 0x7f997c1f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9980cf1700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 0x7f997c1f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 -- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 msgr2=0x7f997c1f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 0x7f997c1f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 -- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f997c1fb3f0 msgr2=0x7f997c1f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f997c1fb3f0 0x7f997c1f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f99814f2700 1 -- 172.21.15.191:0/1390199470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c1fabf0 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.069+0000 7f9980cf1700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 0x7f997c1f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f99814f2700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9974014000 tx=0x7f99740020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997401c070 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997401d040 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c1faed0 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740189a0 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c158fd0 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9974022460 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.070+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f997400cdd0 con 0x7f997c161d70 2024-03-14T09:01:03.071 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.071+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c04ea80 con 0x7f997c161d70 2024-03-14T09:01:03.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.074+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f9974027070 con 0x7f997c161d70 2024-03-14T09:01:03.075 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:03.075+0000 7ffa5c989200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T09:01:03.087 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:03.088+0000 7fb036387200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-14T09:01:03.146 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:03.147+0000 7ffa5c989200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T09:01:03.161 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:03.162+0000 7fb036387200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-14T09:01:03.164 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.166+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d5110594-4610-435f-a8b3-cd63e4717693", "id": 3} v 0) v1 -- 0x7f997c071fe0 con 0x7f997c161d70 2024-03-14T09:01:03.172 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.173+0000 7f9971ffb700 1 -- 172.21.15.191:0/1390199470 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5110594-4610-435f-a8b3-cd63e4717693", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f997c071fe0 con 0x7f997c161d70 2024-03-14T09:01:03.172 INFO:teuthology.orchestra.run.smithi191.stdout:3 2024-03-14T09:01:03.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 msgr2=0x7f997c0c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.173 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9974014000 tx=0x7f99740020e0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 shutdown_connections 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f997c1fb3f0 0x7f997c1f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f997c161d70 0x7f997c0c5150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 --2- 172.21.15.191:0/1390199470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f997c0ab1a0 0x7f997c1f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 >> 172.21.15.191:0/1390199470 conn(0x7f997c0c2970 msgr2=0x7f997c0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 shutdown_connections 2024-03-14T09:01:03.174 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.174+0000 7f9983756700 1 -- 172.21.15.191:0/1390199470 wait complete. 2024-03-14T09:01:03.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 7cdbd767-c933-41d9-87ed-746a04c6a187 4 2024-03-14T09:01:03.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.308+0000 7fbfec4b9700 1 Processor -- start 2024-03-14T09:01:03.307 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.308+0000 7fbfec4b9700 1 -- start start 2024-03-14T09:01:03.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfe4156330 0x7fbfe4157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe406b310 con 0x7fbfe4156330 2024-03-14T09:01:03.310 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe40ac8e0 con 0x7fbfe414eaf0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfec4b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe406de30 con 0x7fbfe406acb0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfea255700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfea255700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38836/0 (socket says 172.21.15.191:38836) 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:57466/0 (socket says 172.21.15.191:57466) 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfea255700 1 -- 172.21.15.191:0/597362050 learned_addr learned my addr 172.21.15.191:0/597362050 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfeaa56700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.154:3300/0 conn(0x7fbfe4156330 0x7fbfe4157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 -- 172.21.15.191:0/597362050 >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 msgr2=0x7fbfe406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 -- 172.21.15.191:0/597362050 >> v2:172.21.15.154:3300/0 conn(0x7fbfe4156330 msgr2=0x7fbfe4157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.309+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.154:3300/0 conn(0x7fbfe4156330 0x7fbfe4157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfe9a54700 1 -- 172.21.15.191:0/597362050 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40acb30 con 0x7fbfe406acb0 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfea255700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:03.311 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/597362050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4070da0 tx=0x7fbfd4009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21643a0d70ae144e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfe9253700 1 -- 172.21.15.191:0/597362050 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4011830 con 0x7fbfe406acb0 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfe9253700 1 -- 172.21.15.191:0/597362050 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4011990 con 0x7fbfe406acb0 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfe9253700 1 -- 172.21.15.191:0/597362050 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4011d20 con 0x7fbfe406acb0 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfec4b9700 1 -- 172.21.15.191:0/597362050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 msgr2=0x7fbfe4155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.310+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/597362050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4070da0 tx=0x7fbfd4009b20 comp rx=0 tx=0).stop 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 -- 172.21.15.191:0/597362050 shutdown_connections 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.154:3300/0 conn(0x7fbfe4156330 0x7fbfe4157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/597362050 >> v2:172.21.15.191:3300/0 conn(0x7fbfe414eaf0 0x7fbfe406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/597362050 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe406acb0 0x7fbfe4155df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 -- 172.21.15.191:0/597362050 >> 172.21.15.191:0/597362050 conn(0x7fbfe40c3880 msgr2=0x7fbfe40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 -- 172.21.15.191:0/597362050 shutdown_connections 2024-03-14T09:01:03.312 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.311+0000 7fbfec4b9700 1 -- 172.21.15.191:0/597362050 wait complete. 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.312+0000 7fbfec4b9700 1 Processor -- start 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.312+0000 7fbfec4b9700 1 -- start start 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 0x7fbfe41eb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe4156330 0x7fbfe4151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe4142520 con 0x7fbfe4156330 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41eb570 con 0x7fbfe406acb0 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfec4b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c2990 con 0x7fbfe414eaf0 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfea255700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 0x7fbfe41eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:57470/0 (socket says 172.21.15.191:57470) 2024-03-14T09:01:03.315 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 -- 172.21.15.191:0/1982676557 learned_addr learned my addr 172.21.15.191:0/1982676557 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfeaa56700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe4156330 0x7fbfe4151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 -- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 msgr2=0x7fbfe41eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 0x7fbfe41eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 -- 172.21.15.191:0/1982676557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe4156330 msgr2=0x7fbfe4151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe4156330 0x7fbfe4151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfe9a54700 1 -- 172.21.15.191:0/1982676557 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0009550 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.313+0000 7fbfea255700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 0x7fbfe41eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfe9a54700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4000c00 tx=0x7fbfd4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4004450 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd401e070 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40038e0 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40094a0 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40c2ec0 con 0x7fbfe414eaf0 2024-03-14T09:01:03.316 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbfd4003d40 con 0x7fbfe414eaf0 2024-03-14T09:01:03.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.314+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbfd401a420 con 0x7fbfe414eaf0 2024-03-14T09:01:03.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.315+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe40697d0 con 0x7fbfe414eaf0 2024-03-14T09:01:03.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.317+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fbfd4020070 con 0x7fbfe414eaf0 2024-03-14T09:01:03.398 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.399+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7cdbd767-c933-41d9-87ed-746a04c6a187", "id": 4} v 0) v1 -- 0x7fbfe406b470 con 0x7fbfe414eaf0 2024-03-14T09:01:03.405 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.406+0000 7fbfdaffd700 1 -- 172.21.15.191:0/1982676557 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cdbd767-c933-41d9-87ed-746a04c6a187", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbfd4033020 con 0x7fbfe414eaf0 2024-03-14T09:01:03.405 INFO:teuthology.orchestra.run.smithi191.stdout:4 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 msgr2=0x7fbfe414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4000c00 tx=0x7fbfd4004750 comp rx=0 tx=0).stop 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 shutdown_connections 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe4156330 0x7fbfe4151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe406acb0 0x7fbfe41eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 --2- 172.21.15.191:0/1982676557 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe414eaf0 0x7fbfe414a150 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 >> 172.21.15.191:0/1982676557 conn(0x7fbfe40c3880 msgr2=0x7fbfe4157090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.407+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 shutdown_connections 2024-03-14T09:01:03.407 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.408+0000 7fbfec4b9700 1 -- 172.21.15.191:0/1982676557 wait complete. 2024-03-14T09:01:03.418 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new f85d1595-d283-4331-bf87-cf7c03fdad6a 5 2024-03-14T09:01:03.527 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:03.527+0000 7ffa5c989200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T09:01:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.542+0000 7f6f58854700 1 Processor -- start 2024-03-14T09:01:03.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 -- start start 2024-03-14T09:01:03.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6f54158840 0x7f6f540c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6f540bf9c0 0x7f6f540bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f54154f40 con 0x7f6f541553d0 2024-03-14T09:01:03.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f54154930 con 0x7f6f540bf9c0 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.543+0000 7f6f58854700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540c57e0 con 0x7f6f54158840 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6f54158840 0x7f6f540c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5259c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f6f541553d0 0x7f6f540c1d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35124/0 (socket says 172.21.15.191:35124) 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6f54158840 0x7f6f540c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38862/0 (socket says 172.21.15.191:38862) 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5259c700 1 -- 172.21.15.191:0/2522649772 learned_addr learned my addr 172.21.15.191:0/2522649772 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f52d9d700 1 --2- 172.21.15.191:0/2522649772 >> v2:172.21.15.154:3300/0 conn(0x7f6f540bf9c0 0x7f6f540bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 -- 172.21.15.191:0/2522649772 >> v2:172.21.15.154:3300/0 conn(0x7f6f540bf9c0 msgr2=0x7f6f540bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 --2- 172.21.15.191:0/2522649772 >> v2:172.21.15.154:3300/0 conn(0x7f6f540bf9c0 0x7f6f540bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 -- 172.21.15.191:0/2522649772 >> v1:172.21.15.191:6789/0 conn(0x7f6f541553d0 legacy=0x7f6f540c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 -- 172.21.15.191:0/2522649772 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540c5ac0 con 0x7f6f54158840 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5159a700 1 -- 172.21.15.191:0/2522649772 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899386479 0 0) 0x7f6f54154f40 con 0x7f6f541553d0 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f51d9b700 1 --2- 172.21.15.191:0/2522649772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f540c0f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f54062450 tx=0x7f6f48009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a23c060f9fd862 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5159a700 1 -- 172.21.15.191:0/2522649772 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48011830 con 0x7f6f54158840 2024-03-14T09:01:03.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5159a700 1 -- 172.21.15.191:0/2522649772 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48011990 con 0x7f6f54158840 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.544+0000 7f6f5159a700 1 -- 172.21.15.191:0/2522649772 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48011d20 con 0x7f6f54158840 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 -- 172.21.15.191:0/2522649772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 msgr2=0x7f6f540c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 --2- 172.21.15.191:0/2522649772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f540c0f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6f54062450 tx=0x7f6f48009b20 comp rx=0 tx=0).stop 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 -- 172.21.15.191:0/2522649772 shutdown_connections 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 --2- 172.21.15.191:0/2522649772 >> v2:172.21.15.154:3300/0 conn(0x7f6f540bf9c0 0x7f6f540bfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 --2- 172.21.15.191:0/2522649772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f540c0f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 -- 172.21.15.191:0/2522649772 >> 172.21.15.191:0/2522649772 conn(0x7f6f5406a770 msgr2=0x7f6f5406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 -- 172.21.15.191:0/2522649772 shutdown_connections 2024-03-14T09:01:03.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.545+0000 7f6f58854700 1 -- 172.21.15.191:0/2522649772 wait complete. 2024-03-14T09:01:03.548 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.546+0000 7f6f58854700 1 Processor -- start 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.546+0000 7f6f58854700 1 -- start start 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f540bf9c0 0x7f6f54157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 0x7f6f54099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541e9af0 con 0x7f6f541553d0 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54157d00 con 0x7f6f54158840 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f58854700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541e9d90 con 0x7f6f540bf9c0 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f5259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f540bf9c0 0x7f6f54157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38870/0 (socket says 172.21.15.191:38870) 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 -- 172.21.15.191:0/3844586580 learned_addr learned my addr 172.21.15.191:0/3844586580 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f51d9b700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 0x7f6f54099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.549 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 -- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f540bf9c0 msgr2=0x7f6f54157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f540bf9c0 0x7f6f54157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 -- 172.21.15.191:0/3844586580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 msgr2=0x7f6f54099ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 0x7f6f54099ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 -- 172.21.15.191:0/3844586580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480094a0 con 0x7f6f54158840 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f51d9b700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 0x7f6f54099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.547+0000 7f6f52d9d700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00efc0 tx=0x7f6f4c00c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00cad0 con 0x7f6f54158840 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c00f040 con 0x7f6f54158840 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f541ea070 con 0x7f6f54158840 2024-03-14T09:01:03.550 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00ce70 con 0x7f6f54158840 2024-03-14T09:01:03.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f54071e90 con 0x7f6f54158840 2024-03-14T09:01:03.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f4c014780 con 0x7f6f54158840 2024-03-14T09:01:03.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.548+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6f4c013960 con 0x7f6f54158840 2024-03-14T09:01:03.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.549+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f541ea590 con 0x7f6f54158840 2024-03-14T09:01:03.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.551+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6f541ea590 con 0x7f6f54158840 2024-03-14T09:01:03.556 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:03.558+0000 7fb036387200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-14T09:01:03.635 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.636+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f85d1595-d283-4331-bf87-cf7c03fdad6a", "id": 5} v 0) v1 -- 0x7f6f540697d0 con 0x7f6f54158840 2024-03-14T09:01:03.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.643+0000 7f6f42ffd700 1 -- 172.21.15.191:0/3844586580 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f85d1595-d283-4331-bf87-cf7c03fdad6a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6f4c018020 con 0x7f6f54158840 2024-03-14T09:01:03.642 INFO:teuthology.orchestra.run.smithi191.stdout:5 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 msgr2=0x7f6f5414e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00efc0 tx=0x7f6f4c00c0c0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 shutdown_connections 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f54158840 0x7f6f5414e5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f540bf9c0 0x7f6f54157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 --2- 172.21.15.191:0/3844586580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f541553d0 0x7f6f54099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 >> 172.21.15.191:0/3844586580 conn(0x7f6f5406a770 msgr2=0x7f6f540c0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 shutdown_connections 2024-03-14T09:01:03.645 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.644+0000 7f6f58854700 1 -- 172.21.15.191:0/3844586580 wait complete. 2024-03-14T09:01:03.655 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new da7d5636-8b31-4679-9d32-793b2cf2451a 6 2024-03-14T09:01:03.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.779+0000 7fea1d997700 1 Processor -- start 2024-03-14T09:01:03.779 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea1d997700 1 -- start start 2024-03-14T09:01:03.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea1d997700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea18151fd0 0x7fea1815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea1d997700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea18155a40 con 0x7fea180c26c0 2024-03-14T09:01:03.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea1d997700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea18072a90 con 0x7fea1814edf0 2024-03-14T09:01:03.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea1d997700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea180c2f10 con 0x7fea18151fd0 2024-03-14T09:01:03.781 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea167fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fea1814edf0 0x7fea181518c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35136/0 (socket says 172.21.15.191:35136) 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.780+0000 7fea167fc700 1 -- 172.21.15.191:0/3352997884 learned_addr learned my addr 172.21.15.191:0/3352997884 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea177fe700 1 --2- 172.21.15.191:0/3352997884 >> v2:172.21.15.154:3300/0 conn(0x7fea18151fd0 0x7fea1815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027716524 0 0) 0x7fea18072a90 con 0x7fea1814edf0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea000035b0 con 0x7fea1814edf0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357206495 0 0) 0x7fea18155a40 con 0x7fea180c26c0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea18072a90 con 0x7fea180c26c0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea177fe700 1 -- 172.21.15.191:0/3352997884 >> v1:172.21.15.191:6789/0 conn(0x7fea1814edf0 legacy=0x7fea181518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1406718513 0 0) 0x7fea000035b0 con 0x7fea1814edf0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea177fe700 1 -- 172.21.15.191:0/3352997884 >> v1:172.21.15.154:6789/0 conn(0x7fea180c26c0 legacy=0x7fea180c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea177fe700 1 -- 172.21.15.191:0/3352997884 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea181556d0 con 0x7fea18151fd0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea177fe700 1 --2- 172.21.15.191:0/3352997884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea18151fd0 0x7fea1815ce60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea10009d90 tx=0x7fea1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1577ae2472b55bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1000e070 con 0x7fea18151fd0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea10003a20 con 0x7fea18151fd0 2024-03-14T09:01:03.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.781+0000 7fea15ffb700 1 -- 172.21.15.191:0/3352997884 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08001670 con 0x7fea18151fd0 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 -- 172.21.15.191:0/3352997884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea18151fd0 msgr2=0x7fea1815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 --2- 172.21.15.191:0/3352997884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea18151fd0 0x7fea1815ce60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea10009d90 tx=0x7fea1000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 -- 172.21.15.191:0/3352997884 shutdown_connections 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 --2- 172.21.15.191:0/3352997884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea18151fd0 0x7fea1815ce60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 -- 172.21.15.191:0/3352997884 >> 172.21.15.191:0/3352997884 conn(0x7fea1806a770 msgr2=0x7fea1806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 -- 172.21.15.191:0/3352997884 shutdown_connections 2024-03-14T09:01:03.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.782+0000 7fea1d997700 1 -- 172.21.15.191:0/3352997884 wait complete. 2024-03-14T09:01:03.785 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.783+0000 7fea1d997700 1 Processor -- start 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.783+0000 7fea1d997700 1 -- start start 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 0x7fea181505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 0x7fea180a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea181ea0c0 con 0x7fea180c26c0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea181ea2f0 con 0x7fea18151fd0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea1d997700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea181ea5d0 con 0x7fea1814edf0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea177fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea167fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 0x7fea180a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea177fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38872/0 (socket says 172.21.15.191:38872) 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea177fe700 1 -- 172.21.15.191:0/2560377498 learned_addr learned my addr 172.21.15.191:0/2560377498 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.784+0000 7fea16ffd700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 0x7fea181505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:03.786 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 -- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 msgr2=0x7fea180a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 0x7fea180a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 -- 172.21.15.191:0/2560377498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 msgr2=0x7fea181505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 0x7fea181505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 -- 172.21.15.191:0/2560377498 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10009a70 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea167fc700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 0x7fea180a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea16ffd700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 0x7fea181505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea177fe700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b370 tx=0x7fea10004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10004440 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea100045a0 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10004950 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea181ea8b0 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea181eae70 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fea1001d770 con 0x7fea18151fd0 2024-03-14T09:01:03.787 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.785+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fea10012d90 con 0x7fea18151fd0 2024-03-14T09:01:03.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.786+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea181eb030 con 0x7fea18151fd0 2024-03-14T09:01:03.788 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.789+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fea10045b50 con 0x7fea18151fd0 2024-03-14T09:01:03.850 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:03.851+0000 7ffa5c989200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T09:01:03.872 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.873+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da7d5636-8b31-4679-9d32-793b2cf2451a", "id": 6} v 0) v1 -- 0x7fea180697d0 con 0x7fea18151fd0 2024-03-14T09:01:03.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.884+0000 7fe9fffff700 1 -- 172.21.15.191:0/2560377498 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da7d5636-8b31-4679-9d32-793b2cf2451a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea10034020 con 0x7fea18151fd0 2024-03-14T09:01:03.884 INFO:teuthology.orchestra.run.smithi191.stdout:6 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.885+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 msgr2=0x7fea1814b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.885+0000 7fea1d997700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea1000b370 tx=0x7fea10004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.885+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 shutdown_connections 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.885+0000 7fea1d997700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea18151fd0 0x7fea1814b720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.886+0000 7fea1d997700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea180c26c0 0x7fea181505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.886+0000 7fea1d997700 1 --2- 172.21.15.191:0/2560377498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea1814edf0 0x7fea180a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.886+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 >> 172.21.15.191:0/2560377498 conn(0x7fea1806a770 msgr2=0x7fea18071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.886+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 shutdown_connections 2024-03-14T09:01:03.885 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:03.886+0000 7fea1d997700 1 -- 172.21.15.191:0/2560377498 wait complete. 2024-03-14T09:01:03.896 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:03.897+0000 7fb036387200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-14T09:01:03.896 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new e0b2fa32-8a5e-418c-8d1a-e8c7520519c6 7 2024-03-14T09:01:04.009 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:04.010+0000 7ffa5c989200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T09:01:04.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.019+0000 7f0ec8146700 1 Processor -- start 2024-03-14T09:01:04.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.019+0000 7f0ec8146700 1 -- start start 2024-03-14T09:01:04.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec8146700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ec0155df0 0x7f0ec0156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec8146700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec00ac8e0 con 0x7f0ec015d270 2024-03-14T09:01:04.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec8146700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec006de30 con 0x7f0ec014eaf0 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec8146700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec006b310 con 0x7f0ec0155df0 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec56e1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ec0155df0 0x7f0ec0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec56e1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ec0155df0 0x7f0ec0156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38876/0 (socket says 172.21.15.191:38876) 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec5ee2700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f0ec014eaf0 0x7f0ec006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:35138/0 (socket says 172.21.15.191:35138) 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.020+0000 7f0ec56e1700 1 -- 172.21.15.191:0/2943416332 learned_addr learned my addr 172.21.15.191:0/2943416332 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec56e1700 1 -- 172.21.15.191:0/2943416332 >> v1:172.21.15.191:6789/0 conn(0x7f0ec014eaf0 legacy=0x7f0ec006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec56e1700 1 -- 172.21.15.191:0/2943416332 >> v1:172.21.15.154:6789/0 conn(0x7f0ec015d270 legacy=0x7f0ec015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec56e1700 1 -- 172.21.15.191:0/2943416332 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00acb30 con 0x7f0ec0155df0 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec56e1700 1 --2- 172.21.15.191:0/2943416332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec0156250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00098f0 tx=0x7f0eb0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4561b8b48fc50f71 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec4ee0700 1 -- 172.21.15.191:0/2943416332 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000b8a0 con 0x7f0ec0155df0 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec4ee0700 1 -- 172.21.15.191:0/2943416332 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb000ba00 con 0x7f0ec0155df0 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec4ee0700 1 -- 172.21.15.191:0/2943416332 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000bd90 con 0x7f0ec0155df0 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec8146700 1 -- 172.21.15.191:0/2943416332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 msgr2=0x7f0ec0156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.021+0000 7f0ec8146700 1 --2- 172.21.15.191:0/2943416332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec0156250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00098f0 tx=0x7f0eb0009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.022+0000 7f0ec8146700 1 -- 172.21.15.191:0/2943416332 shutdown_connections 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.022+0000 7f0ec8146700 1 --2- 172.21.15.191:0/2943416332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec0156250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.022+0000 7f0ec8146700 1 -- 172.21.15.191:0/2943416332 >> 172.21.15.191:0/2943416332 conn(0x7f0ec00c3880 msgr2=0x7f0ec00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.022+0000 7f0ec8146700 1 -- 172.21.15.191:0/2943416332 shutdown_connections 2024-03-14T09:01:04.022 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.022+0000 7f0ec8146700 1 -- 172.21.15.191:0/2943416332 wait complete. 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.023+0000 7f0ec8146700 1 Processor -- start 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.023+0000 7f0ec8146700 1 -- start start 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ec014eaf0 0x7f0ec0143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 0x7f0ec015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01edfd0 con 0x7f0ec014eaf0 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec015a210 con 0x7f0ec0155df0 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec8146700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0143050 con 0x7f0ec015d270 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec66e3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 0x7f0ec015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.025 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec5ee2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ec014eaf0 0x7f0ec0143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38878/0 (socket says 172.21.15.191:38878) 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 -- 172.21.15.191:0/3400163147 learned_addr learned my addr 172.21.15.191:0/3400163147 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 -- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 msgr2=0x7f0ec015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 0x7f0ec015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 -- 172.21.15.191:0/3400163147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ec014eaf0 msgr2=0x7f0ec0143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ec014eaf0 0x7f0ec0143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec56e1700 1 -- 172.21.15.191:0/3400163147 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0009550 con 0x7f0ec0155df0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.024+0000 7f0ec66e3700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 0x7f0ec015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0ec56e1700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0000c00 tx=0x7f0eb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004590 con 0x7f0ec0155df0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00036a0 con 0x7f0ec0155df0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec01ed980 con 0x7f0ec0155df0 2024-03-14T09:01:04.026 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0003a50 con 0x7f0ec0155df0 2024-03-14T09:01:04.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec014ac30 con 0x7f0ec0155df0 2024-03-14T09:01:04.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0eb0021070 con 0x7f0ec0155df0 2024-03-14T09:01:04.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.025+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f0eb0024470 con 0x7f0ec0155df0 2024-03-14T09:01:04.027 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.026+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec01ee180 con 0x7f0ec0155df0 2024-03-14T09:01:04.028 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.030+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0eb0044b50 con 0x7f0ec0155df0 2024-03-14T09:01:04.066 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:04.067+0000 7fb036387200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-14T09:01:04.089 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:04.090+0000 7ffa5c989200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T09:01:04.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.114+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0b2fa32-8a5e-418c-8d1a-e8c7520519c6", "id": 7} v 0) v1 -- 0x7f0ec00697d0 con 0x7f0ec0155df0 2024-03-14T09:01:04.121 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.122+0000 7f0eb67fc700 1 -- 172.21.15.191:0/3400163147 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0b2fa32-8a5e-418c-8d1a-e8c7520519c6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0eb0033030 con 0x7f0ec0155df0 2024-03-14T09:01:04.121 INFO:teuthology.orchestra.run.smithi191.stdout:7 2024-03-14T09:01:04.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 msgr2=0x7f0ec014a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.122 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0000c00 tx=0x7f0eb0004750 comp rx=0 tx=0).stop 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 shutdown_connections 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec015d270 0x7f0ec015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ec014eaf0 0x7f0ec0143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 --2- 172.21.15.191:0/3400163147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0155df0 0x7f0ec014a6f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 >> 172.21.15.191:0/3400163147 conn(0x7f0ec00c3880 msgr2=0x7f0ec006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 shutdown_connections 2024-03-14T09:01:04.123 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-14T09:01:04.123+0000 7f0ec8146700 1 -- 172.21.15.191:0/3400163147 wait complete. 2024-03-14T09:01:04.134 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-14T09:01:04.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-14T09:01:04.136 INFO:tasks.ceph.osd.0:Started 2024-03-14T09:01:04.136 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-14T09:01:04.137 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-14T09:01:04.139 INFO:tasks.ceph.osd.1:Started 2024-03-14T09:01:04.139 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-14T09:01:04.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-14T09:01:04.141 INFO:tasks.ceph.osd.2:Started 2024-03-14T09:01:04.141 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-14T09:01:04.141 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-14T09:01:04.143 INFO:tasks.ceph.osd.3:Started 2024-03-14T09:01:04.143 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-14T09:01:04.143 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-14T09:01:04.145 INFO:tasks.ceph.osd.4:Started 2024-03-14T09:01:04.145 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-14T09:01:04.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-14T09:01:04.148 INFO:tasks.ceph.osd.5:Started 2024-03-14T09:01:04.148 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-14T09:01:04.148 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-14T09:01:04.149 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:04.149+0000 7fb036387200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-14T09:01:04.150 INFO:tasks.ceph.osd.6:Started 2024-03-14T09:01:04.150 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-14T09:01:04.150 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-14T09:01:04.152 INFO:tasks.ceph.osd.7:Started 2024-03-14T09:01:04.153 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:04.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.312+0000 7fc03fe78700 1 Processor -- start 2024-03-14T09:01:04.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.312+0000 7fc03fe78700 1 -- start start 2024-03-14T09:01:04.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.312+0000 7fc03fe78700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0380a9650 0x7fc0380c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.312+0000 7fc03fe78700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 0x7fc03806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03fe78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc038152690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03fe78700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc038156050 con 0x7fc0380a9650 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03fe78700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0380a86f0 con 0x7fc0381564e0 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03fe78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380c34b0 con 0x7fc038156960 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0380a9650 0x7fc0380c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0380a9650 0x7fc0380c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55538/0 (socket says 172.21.15.154:55538) 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 -- 172.21.15.154:0/4197774361 learned_addr learned my addr 172.21.15.154:0/4197774361 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03d413700 1 --2- 172.21.15.154:0/4197774361 >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 0x7fc03806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03e415700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc038152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 -- 172.21.15.154:0/4197774361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 msgr2=0x7fc038152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc038152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 -- 172.21.15.154:0/4197774361 >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 msgr2=0x7fc03806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 --2- 172.21.15.154:0/4197774361 >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 0x7fc03806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 -- 172.21.15.154:0/4197774361 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380a8990 con 0x7fc0380a9650 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03d413700 1 --2- 172.21.15.154:0/4197774361 >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 0x7fc03806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03dc14700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc0380c1780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0380a9fb0 tx=0x7fc034009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76876c898fe4c7d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03cc12700 1 -- 172.21.15.154:0/4197774361 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400b870 con 0x7fc0380a9650 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03cc12700 1 -- 172.21.15.154:0/4197774361 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400b9d0 con 0x7fc0380a9650 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.313+0000 7fc03cc12700 1 -- 172.21.15.154:0/4197774361 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400bd60 con 0x7fc0380a9650 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 -- 172.21.15.154:0/4197774361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 msgr2=0x7fc0380c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc0380c1780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0380a9fb0 tx=0x7fc034009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 -- 172.21.15.154:0/4197774361 shutdown_connections 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc038152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 --2- 172.21.15.154:0/4197774361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc0380c1780 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 --2- 172.21.15.154:0/4197774361 >> v2:172.21.15.191:3300/0 conn(0x7fc0381564e0 0x7fc03806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 -- 172.21.15.154:0/4197774361 >> 172.21.15.154:0/4197774361 conn(0x7fc0380c3880 msgr2=0x7fc0380c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 -- 172.21.15.154:0/4197774361 shutdown_connections 2024-03-14T09:01:04.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.314+0000 7fc03fe78700 1 -- 172.21.15.154:0/4197774361 wait complete. 2024-03-14T09:01:04.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.315+0000 7fc03fe78700 1 Processor -- start 2024-03-14T09:01:04.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.315+0000 7fc03fe78700 1 -- start start 2024-03-14T09:01:04.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0381564e0 0x7fc03813ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc0380c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:04.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0381de7d0 con 0x7fc0380a9650 2024-03-14T09:01:04.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0381de210 con 0x7fc0381564e0 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0381de4f0 con 0x7fc038156960 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55554/0 (socket says 172.21.15.154:55554) 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 -- 172.21.15.154:0/421759550 learned_addr learned my addr 172.21.15.154:0/421759550 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03d413700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0381564e0 0x7fc03813ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03e415700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc0380c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 -- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 msgr2=0x7fc0380c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc0380c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 -- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0381564e0 msgr2=0x7fc03813ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0381564e0 0x7fc03813ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 -- 172.21.15.154:0/421759550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034009550 con 0x7fc0380a9650 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03dc14700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c00 tx=0x7fc034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034004310 con 0x7fc0380a9650 2024-03-14T09:01:04.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03401e070 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0380c5a60 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340036a0 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.316+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc038147f90 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.317+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc034020030 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.317+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc03402cde0 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.317+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0381de350 con 0x7fc0380a9650 2024-03-14T09:01:04.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.319+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fc0381de350 con 0x7fc0380a9650 2024-03-14T09:01:04.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.400+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0380c5c70 con 0x7fc0380a9650 2024-03-14T09:01:04.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.401+0000 7fc02e7fc700 1 -- 172.21.15.154:0/421759550 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc034015020 con 0x7fc0380a9650 2024-03-14T09:01:04.401 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:04.401 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":10,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:04.115453+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 msgr2=0x7fc03814bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc034000c00 tx=0x7fc034004750 comp rx=0 tx=0).stop 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 shutdown_connections 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc038156960 0x7fc0380c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0380a9650 0x7fc03814bb00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 --2- 172.21.15.154:0/421759550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0381564e0 0x7fc03813ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 >> 172.21.15.154:0/421759550 conn(0x7fc0380c3880 msgr2=0x7fc038071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:04.403 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 shutdown_connections 2024-03-14T09:01:04.403 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:04.402+0000 7fc03fe78700 1 -- 172.21.15.154:0/421759550 wait complete. 2024-03-14T09:01:04.415 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-14T09:01:04.415 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-14T09:01:04.751 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-14T09:01:04.752+0000 7f2753b4c640 -1 Falling back to public interface 2024-03-14T09:01:04.757 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-14T09:01:04.758+0000 7f47b9769640 -1 Falling back to public interface 2024-03-14T09:01:04.764 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:04.764+0000 7ff9f0188640 -1 Falling back to public interface 2024-03-14T09:01:04.764 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-03-14T09:01:04.764+0000 7fd1e499c640 -1 Falling back to public interface 2024-03-14T09:01:04.764 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:04.764+0000 7ff9e7543700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-14T09:01:04.765 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:04.764+0000 7ff9e7543700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-03-14T09:01:04.768 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-03-14T09:01:04.769+0000 7f981a446640 -1 Falling back to public interface 2024-03-14T09:01:04.771 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-14T09:01:04.772+0000 7f14e4d06640 -1 Falling back to public interface 2024-03-14T09:01:04.771 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-14T09:01:04.772+0000 7fb8f9c8d640 -1 Falling back to public interface 2024-03-14T09:01:04.774 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-03-14T09:01:04.774+0000 7fbdf31a9640 -1 Falling back to public interface 2024-03-14T09:01:04.851 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:04.852+0000 7ffa5c989200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T09:01:04.876 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:04.877+0000 7fb036387200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-14T09:01:04.934 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:04.935+0000 7ffa5c989200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T09:01:04.959 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:04.960+0000 7fb036387200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-14T09:01:05.068 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:05.069+0000 7ffa5c989200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T09:01:05.092 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:05.093+0000 7fb036387200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-14T09:01:05.222 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:05.339 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.339+0000 7f7036b0d700 1 Processor -- start 2024-03-14T09:01:05.339 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.339+0000 7f7036b0d700 1 -- start start 2024-03-14T09:01:05.341 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f7036b0d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 0x7f70301602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:05.341 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f7036b0d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70300c1890 con 0x7f7030066490 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f7036b0d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703006bd10 con 0x7f7030152050 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f7036b0d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7030154a40 con 0x7f70301589a0 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f70348a9700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7030066490 0x7f703006eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40348/0 (socket says 172.21.15.154:40348) 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f70348a9700 1 -- 172.21.15.154:0/1979733510 learned_addr learned my addr 172.21.15.154:0/1979733510 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f70350aa700 1 --2- 172.21.15.154:0/1979733510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 0x7f70301602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924261971 0 0) 0x7f70300c1890 con 0x7f7030066490 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70200035b0 con 0x7f7030066490 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034644989 0 0) 0x7f703006bd10 con 0x7f7030152050 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.340+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70300c1890 con 0x7f7030152050 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f70350aa700 1 -- 172.21.15.154:0/1979733510 >> v1:172.21.15.191:6789/0 conn(0x7f7030152050 legacy=0x7f7030158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f70350aa700 1 -- 172.21.15.154:0/1979733510 >> v1:172.21.15.154:6789/0 conn(0x7f7030066490 legacy=0x7f703006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f70350aa700 1 -- 172.21.15.154:0/1979733510 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70301524d0 con 0x7f70301589a0 2024-03-14T09:01:05.342 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f70350aa700 1 --2- 172.21.15.154:0/1979733510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 0x7f70301602f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7028009dc0 tx=0x7f702800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c9bd727443b724 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800e070 con 0x7f70301589a0 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800bb20 con 0x7f70301589a0 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f702f7fe700 1 -- 172.21.15.154:0/1979733510 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028012430 con 0x7f70301589a0 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f7036b0d700 1 -- 172.21.15.154:0/1979733510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 msgr2=0x7f70301602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f7036b0d700 1 --2- 172.21.15.154:0/1979733510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 0x7f70301602f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7028009dc0 tx=0x7f702800b330 comp rx=0 tx=0).stop 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f7036b0d700 1 -- 172.21.15.154:0/1979733510 shutdown_connections 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.341+0000 7f7036b0d700 1 --2- 172.21.15.154:0/1979733510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70301589a0 0x7f70301602f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.342+0000 7f7036b0d700 1 -- 172.21.15.154:0/1979733510 >> 172.21.15.154:0/1979733510 conn(0x7f703006a770 msgr2=0x7f703006ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.342+0000 7f7036b0d700 1 -- 172.21.15.154:0/1979733510 shutdown_connections 2024-03-14T09:01:05.343 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.342+0000 7f7036b0d700 1 -- 172.21.15.154:0/1979733510 wait complete. 2024-03-14T09:01:05.345 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 Processor -- start 2024-03-14T09:01:05.345 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 -- start start 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7030066490 0x7f70301ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70301589a0 0x7f70300a9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703014eb10 con 0x7f7030152050 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703014d850 con 0x7f70301589a0 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.343+0000 7f7036b0d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703014e450 con 0x7f7030066490 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55566/0 (socket says 172.21.15.154:55566) 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 -- 172.21.15.154:0/2898618042 learned_addr learned my addr 172.21.15.154:0/2898618042 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f70348a9700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7030066490 0x7f70301ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f70350aa700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70301589a0 0x7f70300a9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 -- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7030066490 msgr2=0x7f70301ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.346 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7030066490 0x7f70301ed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 -- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70301589a0 msgr2=0x7f70300a9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70301589a0 0x7f70300a9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 -- 172.21.15.154:0/2898618042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028009a70 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702ffff700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c520 tx=0x7f701c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c0052d0 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c005430 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703014e730 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c0038c0 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.344+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70300c62f0 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.345+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f701c003d20 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.345+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f701c012af0 con 0x7f7030152050 2024-03-14T09:01:05.347 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.345+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703004ea80 con 0x7f7030152050 2024-03-14T09:01:05.348 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.349+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f701c017070 con 0x7f7030152050 2024-03-14T09:01:05.420 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.420+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70300c0b70 con 0x7f7030152050 2024-03-14T09:01:05.420 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.421+0000 7f702d7fa700 1 -- 172.21.15.154:0/2898618042 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f701c011590 con 0x7f7030152050 2024-03-14T09:01:05.420 INFO:teuthology.misc.health.smithi154.stdout: 2024-03-14T09:01:05.421 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":10,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:04.115453+0000","last_up_change":"0.000000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 msgr2=0x7f70300c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f701c00c520 tx=0x7f701c00c830 comp rx=0 tx=0).stop 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 shutdown_connections 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70301589a0 0x7f70300a9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7030066490 0x7f70301ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 --2- 172.21.15.154:0/2898618042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7030152050 0x7f70300c5920 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 >> 172.21.15.154:0/2898618042 conn(0x7f703006a770 msgr2=0x7f70301542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 shutdown_connections 2024-03-14T09:01:05.422 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:05.422+0000 7f7036b0d700 1 -- 172.21.15.154:0/2898618042 wait complete. 2024-03-14T09:01:05.432 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-14T09:01:05.634 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:05.635+0000 7ffa5c989200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T09:01:05.641 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:05.643+0000 7fb036387200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-14T09:01:05.791 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:05.791+0000 7ffa5c989200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T09:01:05.808 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:05.808+0000 7fb036387200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-14T09:01:05.958 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:05.958+0000 7ffa5c989200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T09:01:05.973 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:05.973+0000 7fb036387200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-14T09:01:06.776 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:06.776+0000 7ffa5c989200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T09:01:06.788 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:06.788+0000 7fb036387200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-14T09:01:07.018 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:07.018+0000 7ffa5c989200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T09:01:07.025 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:07.025+0000 7fb036387200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-14T09:01:07.092 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:07.093+0000 7ffa5c989200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T09:01:07.098 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:07.098+0000 7fb036387200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-14T09:01:07.490 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:07.490+0000 7fb036387200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T09:01:07.513 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-03-14T09:01:07.514+0000 7ffa5c989200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-14T09:01:08.037 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-03-14T09:01:08.037+0000 7fd1e499c640 -1 osd.3 0 log_to_monitors true 2024-03-14T09:01:08.239 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-14T09:01:08.240+0000 7f2753b4c640 -1 osd.7 0 log_to_monitors true 2024-03-14T09:01:08.241 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-03-14T09:01:08.242+0000 7f981a446640 -1 osd.0 0 log_to_monitors true 2024-03-14T09:01:08.269 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-14T09:01:08.269+0000 7f47b9769640 -1 osd.4 0 log_to_monitors true 2024-03-14T09:01:08.290 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:08.290+0000 7ff9f0188640 -1 osd.2 0 log_to_monitors true 2024-03-14T09:01:08.304 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-03-14T09:01:08.305+0000 7fbdf31a9640 -1 osd.1 0 log_to_monitors true 2024-03-14T09:01:08.321 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-14T09:01:08.322+0000 7fb8f9c8d640 -1 osd.6 0 log_to_monitors true 2024-03-14T09:01:08.333 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-14T09:01:08.334+0000 7f14e4d06640 -1 osd.5 0 log_to_monitors true 2024-03-14T09:01:08.517 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:08.517+0000 7faff163e700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-14T09:01:08.517 INFO:tasks.ceph.mgr.x.smithi191.stderr:2024-03-14T09:01:08.517+0000 7faff163e700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-14T09:01:11.482 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-14T09:01:11.482+0000 7f14d88ba700 -1 osd.5 0 waiting for initial osdmap 2024-03-14T09:01:11.488 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-14T09:01:11.488+0000 7f47ad31d700 -1 osd.4 0 waiting for initial osdmap 2024-03-14T09:01:11.490 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-03-14T09:01:11.490+0000 7fbde6d5d700 -1 osd.1 0 waiting for initial osdmap 2024-03-14T09:01:11.503 INFO:tasks.ceph.osd.5.smithi191.stderr:2024-03-14T09:01:11.503+0000 7f14d0c95700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.509 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-03-14T09:01:11.505+0000 7fbddf138700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.517 INFO:tasks.ceph.osd.4.smithi191.stderr:2024-03-14T09:01:11.518+0000 7f47a56f8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.521 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-03-14T09:01:11.522+0000 7f980dffa700 -1 osd.0 0 waiting for initial osdmap 2024-03-14T09:01:11.531 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-03-14T09:01:11.532+0000 7f98063d5700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.577 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-14T09:01:11.577+0000 7f2747700700 -1 osd.7 0 waiting for initial osdmap 2024-03-14T09:01:11.585 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:11.586+0000 7ff9e3d3c700 -1 osd.2 0 waiting for initial osdmap 2024-03-14T09:01:11.589 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-14T09:01:11.589+0000 7fb8ed841700 -1 osd.6 0 waiting for initial osdmap 2024-03-14T09:01:11.589 INFO:tasks.ceph.osd.7.smithi191.stderr:2024-03-14T09:01:11.589+0000 7f273fadb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.601 INFO:tasks.ceph.osd.6.smithi191.stderr:2024-03-14T09:01:11.602+0000 7fb8e5c1c700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.608 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-03-14T09:01:11.609+0000 7ff9dc117700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:11.633 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-03-14T09:01:11.634+0000 7fd1d8550700 -1 osd.3 0 waiting for initial osdmap 2024-03-14T09:01:11.648 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-03-14T09:01:11.649+0000 7fd1d092b700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-14T09:01:12.239 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:12.354 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.354+0000 7f0bc0237700 1 Processor -- start 2024-03-14T09:01:12.354 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.355+0000 7f0bc0237700 1 -- start start 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.355+0000 7f0bc0237700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.355+0000 7f0bc0237700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0bb806acb0 0x7f0bb80c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bc0237700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0bb806b130 0x7f0bb80c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bc0237700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb806eab0 con 0x7f0bb806b130 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bc0237700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb8154a70 con 0x7f0bb806acb0 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bc0237700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8154d50 con 0x7f0bb806e6d0 2024-03-14T09:01:12.357 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0bb806b130 0x7f0bb80c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0bb806b130 0x7f0bb80c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44848/0 (socket says 172.21.15.154:44848) 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 -- 172.21.15.154:0/172631310 learned_addr learned my addr 172.21.15.154:0/172631310 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbdfd3700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 -- 172.21.15.154:0/172631310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 msgr2=0x7f0bb806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 -- 172.21.15.154:0/172631310 >> v2:172.21.15.191:3300/0 conn(0x7f0bb806acb0 msgr2=0x7f0bb80c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 --2- 172.21.15.154:0/172631310 >> v2:172.21.15.191:3300/0 conn(0x7f0bb806acb0 0x7f0bb80c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 -- 172.21.15.154:0/172631310 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8155030 con 0x7f0bb806b130 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbe7d4700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c0a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806c860 tx=0x7f0bb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8060f55ab9cef59 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbcfd1700 1 -- 172.21.15.154:0/172631310 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000e070 con 0x7f0bb806b130 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbcfd1700 1 -- 172.21.15.154:0/172631310 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb000b9c0 con 0x7f0bb806b130 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.356+0000 7f0bbcfd1700 1 -- 172.21.15.154:0/172631310 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000bd70 con 0x7f0bb806b130 2024-03-14T09:01:12.358 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 -- 172.21.15.154:0/172631310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 msgr2=0x7f0bb80c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c0a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806c860 tx=0x7f0bb000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 -- 172.21.15.154:0/172631310 shutdown_connections 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c0a40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 --2- 172.21.15.154:0/172631310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 --2- 172.21.15.154:0/172631310 >> v2:172.21.15.191:3300/0 conn(0x7f0bb806acb0 0x7f0bb80c0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 -- 172.21.15.154:0/172631310 >> 172.21.15.154:0/172631310 conn(0x7f0bb80c3ad0 msgr2=0x7f0bb80c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 -- 172.21.15.154:0/172631310 shutdown_connections 2024-03-14T09:01:12.359 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.357+0000 7f0bc0237700 1 -- 172.21.15.154:0/172631310 wait complete. 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.358+0000 7f0bc0237700 1 Processor -- start 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.358+0000 7f0bc0237700 1 -- start start 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 0x7f0bb8144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb8151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8149ba0 con 0x7f0bb806b130 2024-03-14T09:01:12.362 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80c5490 con 0x7f0bb806acb0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8144800 con 0x7f0bb806e6d0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44862/0 (socket says 172.21.15.154:44862) 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 -- 172.21.15.154:0/1260093046 learned_addr learned my addr 172.21.15.154:0/1260093046 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbe7d4700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb8151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbdfd3700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 0x7f0bb8144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 -- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 msgr2=0x7f0bb8151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb8151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 -- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 msgr2=0x7f0bb8144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 0x7f0bb8144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 -- 172.21.15.154:0/1260093046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0009b30 con 0x7f0bb806b130 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbdfd3700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 0x7f0bb8144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bbd7d2700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400b530 tx=0x7f0bb400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.363 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4011830 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb4011990 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb8144520 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.359+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4011d40 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.360+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb81edec0 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.360+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb80749c0 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.361+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7f0bb401a400 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.361+0000 7f0bae7fc700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 0x7f0b9c06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.361+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f0bb4087e30 con 0x7f0bb806b130 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.361+0000 7f0bbdfd3700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 0x7f0b9c06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.364 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.362+0000 7f0bbdfd3700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 0x7f0b9c06cc60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007a50 tx=0x7f0ba8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.365 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.366+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0bb40540c0 con 0x7f0bb806b130 2024-03-14T09:01:12.566 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.567+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bb804ea80 con 0x7f0bb806b130 2024-03-14T09:01:12.567 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.568+0000 7f0bae7fc700 1 -- 172.21.15.154:0/1260093046 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10488 (secure 0 0 0) 0x7f0bb4015070 con 0x7f0bb806b130 2024-03-14T09:01:12.567 INFO:teuthology.misc.health.smithi154.stdout: 2024-03-14T09:01:12.567 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":13,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:12.513554+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 msgr2=0x7f0b9c06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 0x7f0b9c06cc60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8007a50 tx=0x7f0ba8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 msgr2=0x7f0bb80c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400b530 tx=0x7f0bb400b840 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 shutdown_connections 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bb806e6d0 0x7f0bb8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bb806acb0 0x7f0bb8144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0b9c06a7a0 0x7f0b9c06cc60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 --2- 172.21.15.154:0/1260093046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bb806b130 0x7f0bb80c4f50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 >> 172.21.15.154:0/1260093046 conn(0x7f0bb80c3ad0 msgr2=0x7f0bb80c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 shutdown_connections 2024-03-14T09:01:12.570 INFO:teuthology.misc.health.smithi154.stderr:2024-03-14T09:01:12.570+0000 7f0bc0237700 1 -- 172.21.15.154:0/1260093046 wait complete. 2024-03-14T09:01:12.580 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-14T09:01:12.581 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-14T09:01:12.581 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T09:01:12.581 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T09:01:12.583 INFO:tasks.ceph.mds.a:Started 2024-03-14T09:01:12.583 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T09:01:12.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T09:01:12.585 INFO:tasks.ceph.mds.c:Started 2024-03-14T09:01:12.585 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T09:01:12.585 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T09:01:12.588 INFO:tasks.ceph.mds.b:Started 2024-03-14T09:01:12.588 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T09:01:12.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T09:01:12.591 INFO:tasks.ceph.mds.d:Started 2024-03-14T09:01:12.591 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-14T09:01:12.591 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:12.668 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:01:12.668 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:01:12.678 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:01:12.680 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-14T09:01:12.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.724+0000 7f2b5e40b700 1 Processor -- start 2024-03-14T09:01:12.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.724+0000 7f2b5e40b700 1 -- start start 2024-03-14T09:01:12.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5e40b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b5814eaf0 0x7f2b5806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5e40b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b5815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5e40b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580ac8e0 con 0x7f2b5806acb0 2024-03-14T09:01:12.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5e40b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5806b310 con 0x7f2b5814eaf0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5e40b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5806de30 con 0x7f2b58156500 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b5814eaf0 0x7f2b5806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b5814eaf0 0x7f2b5806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44904/0 (socket says 172.21.15.154:44904) 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 -- 172.21.15.154:0/3817565534 learned_addr learned my addr 172.21.15.154:0/3817565534 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b5c9a8700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b5815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 -- 172.21.15.154:0/3817565534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 msgr2=0x7f2b5815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b5815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 -- 172.21.15.154:0/3817565534 >> v1:172.21.15.191:6789/0 conn(0x7f2b5806acb0 legacy=0x7f2b58155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 -- 172.21.15.154:0/3817565534 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58156930 con 0x7f2b5814eaf0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.725+0000 7f2b57fff700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b5806a770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098f0 tx=0x7f2b48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255e4232891fa3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b56ffd700 1 -- 172.21.15.154:0/3817565534 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48011830 con 0x7f2b5814eaf0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b56ffd700 1 -- 172.21.15.154:0/3817565534 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48011990 con 0x7f2b5814eaf0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b56ffd700 1 -- 172.21.15.154:0/3817565534 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48011d20 con 0x7f2b5814eaf0 2024-03-14T09:01:12.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 -- 172.21.15.154:0/3817565534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 msgr2=0x7f2b5806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b5806a770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098f0 tx=0x7f2b48009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 -- 172.21.15.154:0/3817565534 shutdown_connections 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b5815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/3817565534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b5806a770 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 -- 172.21.15.154:0/3817565534 >> 172.21.15.154:0/3817565534 conn(0x7f2b580c3880 msgr2=0x7f2b580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 -- 172.21.15.154:0/3817565534 shutdown_connections 2024-03-14T09:01:12.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.726+0000 7f2b5e40b700 1 -- 172.21.15.154:0/3817565534 wait complete. 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.727+0000 7f2b5e40b700 1 Processor -- start 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.727+0000 7f2b5e40b700 1 -- start start 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b5806acb0 0x7f2b581ea460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b58144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58145c20 con 0x7f2b5814eaf0 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58148af0 con 0x7f2b5806acb0 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5e40b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5814a0f0 con 0x7f2b58156500 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b57fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b5806acb0 0x7f2b581ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44914/0 (socket says 172.21.15.154:44914) 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 -- 172.21.15.154:0/2162656763 learned_addr learned my addr 172.21.15.154:0/2162656763 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b5c9a8700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b58144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 -- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 msgr2=0x7f2b58144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b58144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 -- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b5806acb0 msgr2=0x7f2b581ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b5806acb0 0x7f2b581ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.728+0000 7f2b577fe700 1 -- 172.21.15.154:0/2162656763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48009550 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b577fe700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c4e0 tx=0x7f2b4000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b400052d0 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40005430 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b400038c0 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b581eb410 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.729+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58149240 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.730+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b580697d0 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.730+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7f2b40003a20 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.730+0000 7f2b54ff9700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 0x7f2b3806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.731+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f2b40087a80 con 0x7f2b5814eaf0 2024-03-14T09:01:12.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.731+0000 7f2b57fff700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 0x7f2b3806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:12.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.731+0000 7f2b57fff700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 0x7f2b3806cc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009c40 tx=0x7f2b48008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:12.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.735+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2b40089e60 con 0x7f2b5814eaf0 2024-03-14T09:01:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.929+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b5806b470 con 0x7f2b5814eaf0 2024-03-14T09:01:12.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.930+0000 7f2b54ff9700 1 -- 172.21.15.154:0/2162656763 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10488 (secure 0 0 0) 0x7f2b400534e0 con 0x7f2b5814eaf0 2024-03-14T09:01:12.929 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:12.929 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":13,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:12.513554+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 msgr2=0x7f2b3806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 0x7f2b3806cc60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009c40 tx=0x7f2b48008030 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 msgr2=0x7f2b581ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000c4e0 tx=0x7f2b4000c7f0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 shutdown_connections 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b58156500 0x7f2b58144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b5806acb0 0x7f2b581ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b3806a7a0 0x7f2b3806cc60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 --2- 172.21.15.154:0/2162656763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5814eaf0 0x7f2b581ea9a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 >> 172.21.15.154:0/2162656763 conn(0x7f2b580c3880 msgr2=0x7f2b5814d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 shutdown_connections 2024-03-14T09:01:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:12.932+0000 7f2b5e40b700 1 -- 172.21.15.154:0/2162656763 wait complete. 2024-03-14T09:01:12.944 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-14T09:01:12.944 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:13.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 Processor -- start 2024-03-14T09:01:13.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 -- start start 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6240c26c0 0x7fb62414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6240c55a0 0x7fb6240c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb624155a40 con 0x7fb6241518c0 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6240c2f10 con 0x7fb6240c55a0 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.066+0000 7fb62aa6f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb624072a90 con 0x7fb6240c26c0 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6240c55a0 0x7fb6240c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb6240c55a0 0x7fb6240c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44920/0 (socket says 172.21.15.154:44920) 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 -- 172.21.15.154:0/2032114296 learned_addr learned my addr 172.21.15.154:0/2032114296 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb62900c700 1 --1- 172.21.15.154:0/2032114296 >> v1:172.21.15.154:6789/0 conn(0x7fb6241518c0 0x7fb624151d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58942/0 (socket says 172.21.15.154:58942) 2024-03-14T09:01:13.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb62880b700 1 --2- 172.21.15.154:0/2032114296 >> v2:172.21.15.191:3300/0 conn(0x7fb6240c26c0 0x7fb62414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 -- 172.21.15.154:0/2032114296 >> v2:172.21.15.191:3300/0 conn(0x7fb6240c26c0 msgr2=0x7fb62414edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 --2- 172.21.15.154:0/2032114296 >> v2:172.21.15.191:3300/0 conn(0x7fb6240c26c0 0x7fb62414edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 -- 172.21.15.154:0/2032114296 >> v1:172.21.15.154:6789/0 conn(0x7fb6241518c0 legacy=0x7fb624151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 -- 172.21.15.154:0/2032114296 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6241556d0 con 0x7fb6240c55a0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb6237fe700 1 -- 172.21.15.154:0/2032114296 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349222659 0 0) 0x7fb624155a40 con 0x7fb6241518c0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb623fff700 1 --2- 172.21.15.154:0/2032114296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6240c55a0 0x7fb6240c5a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6240c6140 tx=0x7fb61000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be124b6e4def235 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb6237fe700 1 -- 172.21.15.154:0/2032114296 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61000dd30 con 0x7fb6240c55a0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb6237fe700 1 -- 172.21.15.154:0/2032114296 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61000de90 con 0x7fb6240c55a0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.067+0000 7fb6237fe700 1 -- 172.21.15.154:0/2032114296 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610010690 con 0x7fb6240c55a0 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 -- 172.21.15.154:0/2032114296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6240c55a0 msgr2=0x7fb6240c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/2032114296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6240c55a0 0x7fb6240c5a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6240c6140 tx=0x7fb61000be70 comp rx=0 tx=0).stop 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 -- 172.21.15.154:0/2032114296 shutdown_connections 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/2032114296 >> v2:172.21.15.191:3300/0 conn(0x7fb6240c26c0 0x7fb62414edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/2032114296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6240c55a0 0x7fb6240c5a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 -- 172.21.15.154:0/2032114296 >> 172.21.15.154:0/2032114296 conn(0x7fb62406a770 msgr2=0x7fb62406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:13.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 -- 172.21.15.154:0/2032114296 shutdown_connections 2024-03-14T09:01:13.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.068+0000 7fb62aa6f700 1 -- 172.21.15.154:0/2032114296 wait complete. 2024-03-14T09:01:13.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.069+0000 7fb62aa6f700 1 Processor -- start 2024-03-14T09:01:13.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.069+0000 7fb62aa6f700 1 -- start start 2024-03-14T09:01:13.073 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6240c26c0 0x7fb6241ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.073 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6240c55a0 0x7fb6240aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241e66f0 con 0x7fb6241518c0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241e6330 con 0x7fb6240c26c0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62aa6f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240a9d40 con 0x7fb6240c55a0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62900c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62900c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44936/0 (socket says 172.21.15.154:44936) 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.070+0000 7fb62900c700 1 -- 172.21.15.154:0/213453769 learned_addr learned my addr 172.21.15.154:0/213453769 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62880b700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6240c26c0 0x7fb6241ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 -- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6240c55a0 msgr2=0x7fb6240aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb623fff700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6240c55a0 0x7fb6240aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6240c55a0 0x7fb6240aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 -- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6240c26c0 msgr2=0x7fb6241ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6240c26c0 0x7fb6241ea2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 -- 172.21.15.154:0/213453769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61000b7f0 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62900c700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00cba0 tx=0x7fb61c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00ab80 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00ad00 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6240aa020 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c007310 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.071+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240a7dd0 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.072+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6240697d0 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.072+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7fb61c007470 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.073+0000 7fb6217fa700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 0x7fb60806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.073+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb61c0881c0 con 0x7fb6241518c0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.073+0000 7fb62880b700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 0x7fb60806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:13.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.073+0000 7fb62880b700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 0x7fb60806cc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6241e7460 tx=0x7fb618008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:13.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.077+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb61c053c20 con 0x7fb6241518c0 2024-03-14T09:01:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.314+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb62406c500 con 0x7fb6241518c0 2024-03-14T09:01:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.315+0000 7fb6217fa700 1 -- 172.21.15.154:0/213453769 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fb61c018500 con 0x7fb6241518c0 2024-03-14T09:01:13.315 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:13.315 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 msgr2=0x7fb60806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 0x7fb60806cc60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6241e7460 tx=0x7fb618008040 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 msgr2=0x7fb6240a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00cba0 tx=0x7fb61c00cf60 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 shutdown_connections 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb60806a7a0 0x7fb60806cc60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6241518c0 0x7fb6240a75a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6240c26c0 0x7fb6241ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 --2- 172.21.15.154:0/213453769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6240c55a0 0x7fb6240aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:13.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 >> 172.21.15.154:0/213453769 conn(0x7fb62406a770 msgr2=0x7fb6240c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 shutdown_connections 2024-03-14T09:01:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:13.317+0000 7fb62aa6f700 1 -- 172.21.15.154:0/213453769 wait complete. 2024-03-14T09:01:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 2 2024-03-14T09:01:15.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:15.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.458+0000 7f15fef77700 1 Processor -- start 2024-03-14T09:01:15.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 -- start start 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15f80a8540 0x7f15f813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f813ade0 con 0x7f15f81550b0 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f806fb30 con 0x7f15f80a8540 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fef77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f806ec30 con 0x7f15f80c3250 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.459+0000 7f15fcd13700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15f80a8540 0x7f15f813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15f80a8540 0x7f15f813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44940/0 (socket says 172.21.15.154:44940) 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 -- 172.21.15.154:0/2520877912 learned_addr learned my addr 172.21.15.154:0/2520877912 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:15.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15f7fff700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 -- 172.21.15.154:0/2520877912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 msgr2=0x7f15f8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 -- 172.21.15.154:0/2520877912 >> v1:172.21.15.154:6789/0 conn(0x7f15f81550b0 legacy=0x7f15f80c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 -- 172.21.15.154:0/2520877912 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f806ef10 con 0x7f15f80a8540 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15f77fe700 1 -- 172.21.15.154:0/2520877912 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228075719 0 0) 0x7f15f0001490 con 0x7f15f81550b0 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15fcd13700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f80a8540 0x7f15f813a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098f0 tx=0x7f15ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9465fe4d169f4d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15f77fe700 1 -- 172.21.15.154:0/2520877912 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec011830 con 0x7f15f80a8540 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15f77fe700 1 -- 172.21.15.154:0/2520877912 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec011990 con 0x7f15f80a8540 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.460+0000 7f15f77fe700 1 -- 172.21.15.154:0/2520877912 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec011d20 con 0x7f15f80a8540 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 -- 172.21.15.154:0/2520877912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f80a8540 msgr2=0x7f15f813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f80a8540 0x7f15f813a8a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098f0 tx=0x7f15ec009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 -- 172.21.15.154:0/2520877912 shutdown_connections 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f80a8540 0x7f15f813a8a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 --2- 172.21.15.154:0/2520877912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 -- 172.21.15.154:0/2520877912 >> 172.21.15.154:0/2520877912 conn(0x7f15f80c3880 msgr2=0x7f15f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:15.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 -- 172.21.15.154:0/2520877912 shutdown_connections 2024-03-14T09:01:15.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.461+0000 7f15fef77700 1 -- 172.21.15.154:0/2520877912 wait complete. 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.462+0000 7f15fef77700 1 Processor -- start 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.462+0000 7f15fef77700 1 -- start start 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8135df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f81550b0 0x7f15f8151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8134a80 con 0x7f15f814e470 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8135020 con 0x7f15f81550b0 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fef77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81ec010 con 0x7f15f80c3250 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fd514700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fd514700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44942/0 (socket says 172.21.15.154:44942) 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.463+0000 7f15fd514700 1 -- 172.21.15.154:0/1239146999 learned_addr learned my addr 172.21.15.154:0/1239146999 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15f7fff700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f81550b0 0x7f15f8151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fcd13700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 -- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 msgr2=0x7f15f8135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 -- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f81550b0 msgr2=0x7f15f8151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f81550b0 0x7f15f8151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 -- 172.21.15.154:0/1239146999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009550 con 0x7f15f814e470 2024-03-14T09:01:15.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fd514700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c7a0 tx=0x7f15f000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003e30 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0005070 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f8134d40 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0005420 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.464+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f8151d70 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.465+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f8136970 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.466+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7f15f0005580 con 0x7f15f814e470 2024-03-14T09:01:15.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.466+0000 7f15f57fa700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 0x7f15dc06cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.466+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15f0087ca0 con 0x7f15f814e470 2024-03-14T09:01:15.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.471+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f15f8136970 con 0x7f15f814e470 2024-03-14T09:01:15.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.474+0000 7f15fcd13700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 0x7f15dc06cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.475+0000 7f15fcd13700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 0x7f15dc06cc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15f8070d70 tx=0x7f15ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.723+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15f813ade0 con 0x7f15f814e470 2024-03-14T09:01:15.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.723+0000 7f15f57fa700 1 -- 172.21.15.154:0/1239146999 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7f15f813ade0 con 0x7f15f814e470 2024-03-14T09:01:15.723 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:15.723 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-14T09:01:15.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 msgr2=0x7f15dc06cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 0x7f15dc06cc60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15f8070d70 tx=0x7f15ec008030 comp rx=0 tx=0).stop 2024-03-14T09:01:15.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 msgr2=0x7f15f814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c7a0 tx=0x7f15f000cab0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 shutdown_connections 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f15dc06a7a0 0x7f15dc06cc60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15f814e470 0x7f15f814e900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f80c3250 0x7f15f8135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 --2- 172.21.15.154:0/1239146999 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f81550b0 0x7f15f8151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 >> 172.21.15.154:0/1239146999 conn(0x7f15f80c3880 msgr2=0x7f15f8138a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 shutdown_connections 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.725+0000 7f15fef77700 1 -- 172.21.15.154:0/1239146999 wait complete. 2024-03-14T09:01:15.726 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 3 2024-03-14T09:01:15.739 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.862+0000 7f3949992700 1 Processor -- start 2024-03-14T09:01:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.862+0000 7f3949992700 1 -- start start 2024-03-14T09:01:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.862+0000 7f3949992700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f394414e980 0x7f39441552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3949992700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39440c4a80 0x7f39440c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3949992700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394414ed60 con 0x7f3944158370 2024-03-14T09:01:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3949992700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440c6b10 con 0x7f39440c4a80 2024-03-14T09:01:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3949992700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3944155820 con 0x7f394414e980 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39440c4a80 0x7f39440c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39440c4a80 0x7f39440c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44956/0 (socket says 172.21.15.154:44956) 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 -- 172.21.15.154:0/1109396245 learned_addr learned my addr 172.21.15.154:0/1109396245 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3942ffd700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f394414e980 0x7f39441552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 -- 172.21.15.154:0/1109396245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f394414e980 msgr2=0x7f39441552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f394414e980 0x7f39441552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 -- 172.21.15.154:0/1109396245 >> v1:172.21.15.191:6789/0 conn(0x7f3944158370 legacy=0x7f3944158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 -- 172.21.15.154:0/1109396245 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440c6cf0 con 0x7f39440c4a80 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3941ffb700 1 -- 172.21.15.154:0/1109396245 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188152870 0 0) 0x7f394414ed60 con 0x7f3944158370 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f39427fc700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39440c4a80 0x7f39440c4f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39380098f0 tx=0x7f3938009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a1bcec986dfbc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3941ffb700 1 -- 172.21.15.154:0/1109396245 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011830 con 0x7f39440c4a80 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3941ffb700 1 -- 172.21.15.154:0/1109396245 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938011990 con 0x7f39440c4a80 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.863+0000 7f3941ffb700 1 -- 172.21.15.154:0/1109396245 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011d20 con 0x7f39440c4a80 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 -- 172.21.15.154:0/1109396245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39440c4a80 msgr2=0x7f39440c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39440c4a80 0x7f39440c4f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f39380098f0 tx=0x7f3938009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:15.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 -- 172.21.15.154:0/1109396245 shutdown_connections 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f394414e980 0x7f39441552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 --2- 172.21.15.154:0/1109396245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39440c4a80 0x7f39440c4f30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 -- 172.21.15.154:0/1109396245 >> 172.21.15.154:0/1109396245 conn(0x7f39440abcf0 msgr2=0x7f39440ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 -- 172.21.15.154:0/1109396245 shutdown_connections 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.864+0000 7f3949992700 1 -- 172.21.15.154:0/1109396245 wait complete. 2024-03-14T09:01:15.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.865+0000 7f3949992700 1 Processor -- start 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.865+0000 7f3949992700 1 -- start start 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39440c4a80 0x7f39441edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3944158370 0x7f394414b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394414bb80 con 0x7f394414e980 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441ee6e0 con 0x7f3944158370 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3949992700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39441eb990 con 0x7f39440c4a80 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44968/0 (socket says 172.21.15.154:44968) 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 -- 172.21.15.154:0/788466692 learned_addr learned my addr 172.21.15.154:0/788466692 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f3942ffd700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39440c4a80 0x7f39441edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39437fe700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3944158370 0x7f394414b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 -- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39440c4a80 msgr2=0x7f39441edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39440c4a80 0x7f39441edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 -- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3944158370 msgr2=0x7f394414b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3944158370 0x7f394414b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 -- 172.21.15.154:0/788466692 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938009550 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.866+0000 7f39427fc700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3938009c40 tx=0x7f3938004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.867+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938004450 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.867+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393801e070 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.867+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39441ebc70 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.867+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380038e0 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.867+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39440be040 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.868+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39440697d0 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.868+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7f3938003a40 con 0x7f394414e980 2024-03-14T09:01:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.868+0000 7f392bfff700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 0x7f393006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.869+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f39380896b0 con 0x7f394414e980 2024-03-14T09:01:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.869+0000 7f3942ffd700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 0x7f393006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.869+0000 7f3942ffd700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 0x7f393006cc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3934009b40 tx=0x7f3934009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:15.873+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3938054f10 con 0x7f394414e980 2024-03-14T09:01:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.078+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39440bd5b0 con 0x7f394414e980 2024-03-14T09:01:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.079+0000 7f392bfff700 1 -- 172.21.15.154:0/788466692 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12430 (secure 0 0 0) 0x7f3938058530 con 0x7f394414e980 2024-03-14T09:01:16.079 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:16.079 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:15.685770+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 msgr2=0x7f393006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 0x7f393006cc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3934009b40 tx=0x7f3934009210 comp rx=0 tx=0).stop 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 msgr2=0x7f394414b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3938009c40 tx=0x7f3938004750 comp rx=0 tx=0).stop 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 shutdown_connections 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3944158370 0x7f394414b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39440c4a80 0x7f39441edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f393006a7d0 0x7f393006cc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 --2- 172.21.15.154:0/788466692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f394414e980 0x7f394414b100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 >> 172.21.15.154:0/788466692 conn(0x7f39440abcf0 msgr2=0x7f394406c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 shutdown_connections 2024-03-14T09:01:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.081+0000 7f3949992700 1 -- 172.21.15.154:0/788466692 wait complete. 2024-03-14T09:01:16.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:16.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:16.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.219+0000 7f9352e76700 1 Processor -- start 2024-03-14T09:01:16.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.219+0000 7f9352e76700 1 -- start start 2024-03-14T09:01:16.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 0x7f934c158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 0x7f934c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f934c0bf9c0 0x7f934c0bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c154f40 con 0x7f934c0bf9c0 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c158d80 con 0x7f934c1553d0 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9352e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c154930 con 0x7f934c0c1d50 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f934c0bf9c0 0x7f934c0bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f934c0bf9c0 0x7f934c0bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44982/0 (socket says 172.21.15.154:44982) 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 -- 172.21.15.154:0/4253448660 learned_addr learned my addr 172.21.15.154:0/4253448660 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f934bfff700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 0x7f934c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9350c12700 1 --2- 172.21.15.154:0/4253448660 >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 0x7f934c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 -- 172.21.15.154:0/4253448660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 msgr2=0x7f934c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 0x7f934c0c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 -- 172.21.15.154:0/4253448660 >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 msgr2=0x7f934c158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 --2- 172.21.15.154:0/4253448660 >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 0x7f934c158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 -- 172.21.15.154:0/4253448660 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0c57e0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f934bfff700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 0x7f934c0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9350c12700 1 --2- 172.21.15.154:0/4253448660 >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 0x7f934c158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.220+0000 7f9351413700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c0bfe20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f934c06c6c0 tx=0x7f934400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dba84a429fcf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f934b7fe700 1 -- 172.21.15.154:0/4253448660 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934400e070 con 0x7f934c0bf9c0 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f934b7fe700 1 -- 172.21.15.154:0/4253448660 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934400b9c0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f934b7fe700 1 -- 172.21.15.154:0/4253448660 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934400bd70 con 0x7f934c0bf9c0 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 -- 172.21.15.154:0/4253448660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 msgr2=0x7f934c0bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c0bfe20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f934c06c6c0 tx=0x7f934400b330 comp rx=0 tx=0).stop 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 -- 172.21.15.154:0/4253448660 shutdown_connections 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c0bfe20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 --2- 172.21.15.154:0/4253448660 >> v2:172.21.15.191:3300/0 conn(0x7f934c1553d0 0x7f934c158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 --2- 172.21.15.154:0/4253448660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c0c1d50 0x7f934c0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 -- 172.21.15.154:0/4253448660 >> 172.21.15.154:0/4253448660 conn(0x7f934c06a770 msgr2=0x7f934c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 -- 172.21.15.154:0/4253448660 shutdown_connections 2024-03-14T09:01:16.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.221+0000 7f9352e76700 1 -- 172.21.15.154:0/4253448660 wait complete. 2024-03-14T09:01:16.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.222+0000 7f9352e76700 1 Processor -- start 2024-03-14T09:01:16.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.222+0000 7f9352e76700 1 -- start start 2024-03-14T09:01:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 0x7f934c09a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 0x7f934c0a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1eb4e0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0a9380 con 0x7f934c0c1d50 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9352e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0a9660 con 0x7f934c1553d0 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9350c12700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9350c12700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44986/0 (socket says 172.21.15.154:44986) 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9350c12700 1 -- 172.21.15.154:0/2436298216 learned_addr learned my addr 172.21.15.154:0/2436298216 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f934bfff700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 0x7f934c09a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.223+0000 7f9351413700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 0x7f934c0a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 -- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 msgr2=0x7f934c0a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 0x7f934c0a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 -- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 msgr2=0x7f934c09a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 0x7f934c09a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 -- 172.21.15.154:0/2436298216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009b30 con 0x7f934c0bf9c0 2024-03-14T09:01:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f934bfff700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 0x7f934c09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9351413700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 0x7f934c0a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9350c12700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f934000e810 tx=0x7f934000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340013610 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340013770 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c157070 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340013b20 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.224+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c157630 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.225+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c0697d0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.225+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 88763+0+0 (secure 0 0 0) 0x7f9340013c80 con 0x7f934c0bf9c0 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.226+0000 7f93497fa700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 0x7f933006cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.226+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93400884d0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.226+0000 7f934bfff700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 0x7f933006cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.227+0000 7f934bfff700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 0x7f933006cc60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9338009820 tx=0x7f9338008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.230+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9340053db0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.412+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9340054180 con 0x7f934c0bf9c0 2024-03-14T09:01:16.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.426+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f934c066550 con 0x7f934c0bf9c0 2024-03-14T09:01:16.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.427+0000 7f93497fa700 1 -- 172.21.15.154:0/2436298216 <== mon.0 v2:172.21.15.154:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93400880a0 con 0x7f934c0bf9c0 2024-03-14T09:01:16.426 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 msgr2=0x7f933006cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 0x7f933006cc60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9338009820 tx=0x7f9338008040 comp rx=0 tx=0).stop 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 msgr2=0x7f934c1eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f934000e810 tx=0x7f934000ebd0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 shutdown_connections 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f934c1553d0 0x7f934c0a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f933006a7a0 0x7f933006cc60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f934c0bf9c0 0x7f934c1eafa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 --2- 172.21.15.154:0/2436298216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f934c0c1d50 0x7f934c09a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 >> 172.21.15.154:0/2436298216 conn(0x7f934c06a770 msgr2=0x7f934c152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 shutdown_connections 2024-03-14T09:01:16.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.429+0000 7f9352e76700 1 -- 172.21.15.154:0/2436298216 wait complete. 2024-03-14T09:01:16.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:01:16.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.562+0000 7f7268908700 1 Processor -- start 2024-03-14T09:01:16.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.562+0000 7f7268908700 1 -- start start 2024-03-14T09:01:16.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7268908700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f72640c4380 0x7f726414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7268908700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72640c5d60 0x7f72640c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7268908700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640acd10 con 0x7f72640702f0 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7268908700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7264150110 con 0x7f72640c5d60 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7268908700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72640c66b0 con 0x7f72640c4380 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7262d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72640c5d60 0x7f72640c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7262d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72640c5d60 0x7f72640c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44992/0 (socket says 172.21.15.154:44992) 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7262d9d700 1 -- 172.21.15.154:0/941684189 learned_addr learned my addr 172.21.15.154:0/941684189 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.563+0000 7f7261d9b700 1 --2- 172.21.15.154:0/941684189 >> v2:172.21.15.191:3300/0 conn(0x7f72640c4380 0x7f726414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7262d9d700 1 -- 172.21.15.154:0/941684189 >> v2:172.21.15.191:3300/0 conn(0x7f72640c4380 msgr2=0x7f726414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7262d9d700 1 --2- 172.21.15.154:0/941684189 >> v2:172.21.15.191:3300/0 conn(0x7f72640c4380 0x7f726414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7262d9d700 1 -- 172.21.15.154:0/941684189 >> v1:172.21.15.191:6789/0 conn(0x7f72640702f0 legacy=0x7f72640c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7262d9d700 1 -- 172.21.15.154:0/941684189 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640ac9c0 con 0x7f72640c5d60 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7262d9d700 1 --2- 172.21.15.154:0/941684189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c5d60 0x7f72640c6170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72640c5980 tx=0x7f725c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2dcdcddf905451 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f726159a700 1 -- 172.21.15.154:0/941684189 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c011830 con 0x7f72640c5d60 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f726159a700 1 -- 172.21.15.154:0/941684189 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c011990 con 0x7f72640c5d60 2024-03-14T09:01:16.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f726159a700 1 -- 172.21.15.154:0/941684189 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c011d20 con 0x7f72640c5d60 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7268908700 1 -- 172.21.15.154:0/941684189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c5d60 msgr2=0x7f72640c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.564+0000 7f7268908700 1 --2- 172.21.15.154:0/941684189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c5d60 0x7f72640c6170 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72640c5980 tx=0x7f725c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 -- 172.21.15.154:0/941684189 shutdown_connections 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 --2- 172.21.15.154:0/941684189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c5d60 0x7f72640c6170 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 --2- 172.21.15.154:0/941684189 >> v2:172.21.15.191:3300/0 conn(0x7f72640c4380 0x7f726414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 -- 172.21.15.154:0/941684189 >> 172.21.15.154:0/941684189 conn(0x7f72640a9bf0 msgr2=0x7f72640a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 -- 172.21.15.154:0/941684189 shutdown_connections 2024-03-14T09:01:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.565+0000 7f7268908700 1 -- 172.21.15.154:0/941684189 wait complete. 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.566+0000 7f7268908700 1 Processor -- start 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.566+0000 7f7268908700 1 -- start start 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72640702f0 0x7f7264156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72640c5d60 0x7f72641eb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726406fea0 con 0x7f72640c4380 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641ec1a0 con 0x7f72640702f0 2024-03-14T09:01:16.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72641ec480 con 0x7f72640c5d60 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44994/0 (socket says 172.21.15.154:44994) 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 -- 172.21.15.154:0/2651853836 learned_addr learned my addr 172.21.15.154:0/2651853836 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7262d9d700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72640c5d60 0x7f72641eb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f726259c700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72640702f0 0x7f7264156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 -- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72640c5d60 msgr2=0x7f72641eb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72640c5d60 0x7f72641eb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 -- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72640702f0 msgr2=0x7f7264156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72640702f0 0x7f7264156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 -- 172.21.15.154:0/2651853836 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009550 con 0x7f72640c4380 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7261d9b700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e9b0 tx=0x7f724c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c009710 con 0x7f72640c4380 2024-03-14T09:01:16.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00ee50 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726406fac0 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00ca50 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.567+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72640aa0f0 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.568+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726406ffe0 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.569+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f724c00cbb0 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.569+0000 7f725affd700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 0x7f724406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.569+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f724c0889d0 con 0x7f72640c4380 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.569+0000 7f726259c700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 0x7f724406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.570+0000 7f726259c700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 0x7f724406cd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72540012e0 tx=0x7f7254009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.574+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f726406ffe0 con 0x7f72640c4380 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.773+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f726406ffe0 con 0x7f72640c4380 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.773+0000 7f725affd700 1 -- 172.21.15.154:0/2651853836 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f726406ffe0 con 0x7f72640c4380 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:01:16.773 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:01:16.774 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:16.774 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:01:16.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 msgr2=0x7f724406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 0x7f724406cd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72540012e0 tx=0x7f7254009040 comp rx=0 tx=0).stop 2024-03-14T09:01:16.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 msgr2=0x7f7264153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e9b0 tx=0x7f724c00ecc0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 shutdown_connections 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72640c5d60 0x7f72641eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72640702f0 0x7f7264156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f724406a860 0x7f724406cd20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 --2- 172.21.15.154:0/2651853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72640c4380 0x7f7264153940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 >> 172.21.15.154:0/2651853836 conn(0x7f72640a9bf0 msgr2=0x7f7264147fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 shutdown_connections 2024-03-14T09:01:16.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.775+0000 7f7268908700 1 -- 172.21.15.154:0/2651853836 wait complete. 2024-03-14T09:01:16.787 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T09:01:16.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T09:01:16.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 Processor -- start 2024-03-14T09:01:16.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 -- start start 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdc0806d050 0x7fdc08158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdc080c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc08158c90 con 0x7fdc081554e0 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0806b110 con 0x7fdc0806d050 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.913+0000 7fdc0e3ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806ab50 con 0x7fdc08155060 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdc081554e0 0x7fdc080c18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58958/0 (socket says 172.21.15.154:58958) 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 -- 172.21.15.154:0/3256319010 learned_addr learned my addr 172.21.15.154:0/3256319010 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc077fe700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdc080c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc07fff700 1 --2- 172.21.15.154:0/3256319010 >> v2:172.21.15.191:3300/0 conn(0x7fdc0806d050 0x7fdc08158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640955839 0 0) 0x7fdc08158c90 con 0x7fdc081554e0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf00035b0 con 0x7fdc081554e0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4035321656 0 0) 0x7fdbf00035b0 con 0x7fdc081554e0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 msgr2=0x7fdc080c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdc080c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 >> v2:172.21.15.191:3300/0 conn(0x7fdc0806d050 msgr2=0x7fdc08158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 --2- 172.21.15.154:0/3256319010 >> v2:172.21.15.191:3300/0 conn(0x7fdc0806d050 0x7fdc08158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0806ae30 con 0x7fdc081554e0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fdc000019f0 con 0x7fdc081554e0 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 >> v1:172.21.15.154:6789/0 conn(0x7fdc081554e0 legacy=0x7fdc080c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0003a00 0x7fdbf0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdbf0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbf0004d90 0x7fdbf0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806b110 con 0x7fdbf0004d90 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806ab50 con 0x7fdbf0003a00 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf000ba10 con 0x7fdc08155060 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbf0004d90 0x7fdbf0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc07fff700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0003a00 0x7fdbf0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc077fe700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdbf0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 -- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 msgr2=0x7fdbf0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc08155060 0x7fdbf0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 -- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0003a00 msgr2=0x7fdbf0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0003a00 0x7fdbf0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.914+0000 7fdc0c98c700 1 -- 172.21.15.154:0/3256319010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00035b0 con 0x7fdbf0004d90 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbf0004d90 0x7fdbf0005210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc080710d0 tx=0x7fdbf0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624fb55134d48980 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000a340 con 0x7fdbf0004d90 2024-03-14T09:01:16.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00012600 con 0x7fdbf0004d90 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc06ffd700 1 -- 172.21.15.154:0/3256319010 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc000129b0 con 0x7fdbf0004d90 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/3256319010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbf0004d90 msgr2=0x7fdbf0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/3256319010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbf0004d90 0x7fdbf0005210 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc080710d0 tx=0x7fdbf0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0c98c700 1 -- 172.21.15.154:0/3256319010 reap_dead start 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/3256319010 shutdown_connections 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/3256319010 >> 172.21.15.154:0/3256319010 conn(0x7fdc080c39f0 msgr2=0x7fdc080c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/3256319010 shutdown_connections 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.915+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/3256319010 wait complete. 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.916+0000 7fdc0e3ef700 1 Processor -- start 2024-03-14T09:01:16.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 -- start start 2024-03-14T09:01:16.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 0x7fdc0814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 0x7fdc08144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc081541f0 con 0x7fdc081459d0 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc08144e70 con 0x7fdbf0004d90 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0e3ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc08152be0 con 0x7fdc081475e0 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0c98c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0c98c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45004/0 (socket says 172.21.15.154:45004) 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.917+0000 7fdc0c98c700 1 -- 172.21.15.154:0/926906285 learned_addr learned my addr 172.21.15.154:0/926906285 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc077fe700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 0x7fdc08144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc07fff700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 0x7fdc0814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 -- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 msgr2=0x7fdc08144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 0x7fdc08144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 -- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 msgr2=0x7fdc0814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 0x7fdc0814cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 -- 172.21.15.154:0/926906285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08150490 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc07fff700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 0x7fdc0814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc077fe700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 0x7fdc08144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0c98c700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc00001cb0 tx=0x7fdc00001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000a340 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc000019c0 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00026640 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc08152900 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.918+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0814c600 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.919+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc080697d0 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.919+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdc00019030 con 0x7fdc081459d0 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.920+0000 7fdc04ff9700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 0x7fdbf406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.920+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc0008a500 con 0x7fdc081459d0 2024-03-14T09:01:16.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.920+0000 7fdc07fff700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 0x7fdbf406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:16.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.920+0000 7fdc07fff700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 0x7fdbf406ccf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc08145650 tx=0x7fdbf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:16.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:16.924+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdc00055be0 con 0x7fdc081459d0 2024-03-14T09:01:17.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.117+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdc080aca90 con 0x7fdc081459d0 2024-03-14T09:01:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.703+0000 7fdc04ff9700 1 -- 172.21.15.154:0/926906285 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc00059200 con 0x7fdc081459d0 2024-03-14T09:01:17.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.707+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 msgr2=0x7fdbf406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.707+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 0x7fdbf406ccf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdc08145650 tx=0x7fdbf8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:17.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.707+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 msgr2=0x7fdc08149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.707+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc00001cb0 tx=0x7fdc00001360 comp rx=0 tx=0).stop 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 shutdown_connections 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbf406a830 0x7fdbf406ccf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdc081459d0 0x7fdc08149700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf0004d90 0x7fdc0814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 --2- 172.21.15.154:0/926906285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdc081475e0 0x7fdc08144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 >> 172.21.15.154:0/926906285 conn(0x7fdc080c39f0 msgr2=0x7fdc08071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.708+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 shutdown_connections 2024-03-14T09:01:17.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.709+0000 7fdc0e3ef700 1 -- 172.21.15.154:0/926906285 wait complete. 2024-03-14T09:01:17.709 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2024-03-14T09:01:17.721 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:01:17.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.849+0000 7ff4ab9d9700 1 Processor -- start 2024-03-14T09:01:17.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.850+0000 7ff4ab9d9700 1 -- start start 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.850+0000 7ff4ab9d9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff4a413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.850+0000 7ff4ab9d9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a413ade0 con 0x7ff4a40a8540 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.850+0000 7ff4ab9d9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a406fb30 con 0x7ff4a40c3250 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.850+0000 7ff4ab9d9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a406ec30 con 0x7ff4a40c1e90 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9775700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff4a40a8540 0x7ff4a4154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58968/0 (socket says 172.21.15.154:58968) 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9775700 1 -- 172.21.15.154:0/2720477230 learned_addr learned my addr 172.21.15.154:0/2720477230 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff4a413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400176804 0 0) 0x7ff49c001110 con 0x7ff4a40a8540 2024-03-14T09:01:17.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0035b0 con 0x7ff4a40a8540 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870642924 0 0) 0x7ff4a406fb30 con 0x7ff4a40c3250 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c001110 con 0x7ff4a40c3250 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801340133 0 0) 0x7ff48c0035b0 con 0x7ff4a40a8540 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 msgr2=0x7ff4a413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff4a413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 >> v1:172.21.15.191:6789/0 conn(0x7ff4a40c3250 legacy=0x7ff4a40c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a406ef10 con 0x7ff4a40a8540 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff4a413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7ff49c001800 con 0x7ff4a40a8540 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 >> v1:172.21.15.154:6789/0 conn(0x7ff4a40a8540 legacy=0x7ff4a4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff48c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c004150 0x7ff48c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004d90 0x7ff48c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a406ec30 con 0x7ff48c004150 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c00de60 con 0x7ff48c004d90 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c00e020 con 0x7ff4a40c1e90 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a8f74700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c004150 0x7ff48c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9775700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff48c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.851+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004d90 0x7ff48c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 -- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 msgr2=0x7ff48c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a40c1e90 0x7ff48c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 -- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004d90 msgr2=0x7ff48c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004d90 0x7ff48c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 -- 172.21.15.154:0/2720477230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009a70 con 0x7ff48c004150 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a8f74700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c004150 0x7ff48c004850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff498001980 tx=0x7ff48c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab4e55269a5370c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498001470 con 0x7ff48c004150 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4980052d0 con 0x7ff48c004150 2024-03-14T09:01:17.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a3fff700 1 -- 172.21.15.154:0/2720477230 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498003680 con 0x7ff48c004150 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/2720477230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c004150 msgr2=0x7ff48c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/2720477230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff48c004150 0x7ff48c004850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff498001980 tx=0x7ff48c005880 comp rx=0 tx=0).stop 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4a9f76700 1 -- 172.21.15.154:0/2720477230 reap_dead start 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/2720477230 shutdown_connections 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/2720477230 >> 172.21.15.154:0/2720477230 conn(0x7ff4a40c3880 msgr2=0x7ff4a40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/2720477230 shutdown_connections 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.852+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/2720477230 wait complete. 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 Processor -- start 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 -- start start 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 0x7ff4a414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a4154d40 0x7ff4a41e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40abf50 con 0x7ff4a40a9f90 2024-03-14T09:01:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41e5fc0 con 0x7ff48c004150 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4ab9d9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40abb90 con 0x7ff4a4154d40 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4a9f76700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4a9f76700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45016/0 (socket says 172.21.15.154:45016) 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4a9f76700 1 -- 172.21.15.154:0/3932338165 learned_addr learned my addr 172.21.15.154:0/3932338165 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4a8f74700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a4154d40 0x7ff4a41e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.854+0000 7ff4a9775700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 0x7ff4a414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 -- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a4154d40 msgr2=0x7ff4a41e5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a4154d40 0x7ff4a41e5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 -- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 msgr2=0x7ff4a414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 0x7ff4a414f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 -- 172.21.15.154:0/3932338165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a41e6260 con 0x7ff4a40a9f90 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9775700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 0x7ff4a414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a9f76700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4070df0 tx=0x7ff49400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400e040 con 0x7ff4a40a9f90 2024-03-14T09:01:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400f040 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a41e6540 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494003a20 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.855+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40aa410 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.856+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a40697d0 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.857+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff494003b80 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.857+0000 7ff4a1ffb700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 0x7ff49006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.857+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff494089170 con 0x7ff4a40a9f90 2024-03-14T09:01:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.857+0000 7ff4a9775700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 0x7ff49006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.858+0000 7ff4a9775700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 0x7ff49006ccf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff49c007d40 tx=0x7ff49c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:17.862+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff494054800 con 0x7ff4a40a9f90 2024-03-14T09:01:18.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.059+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff4a406d420 con 0x7ff4a40a9f90 2024-03-14T09:01:18.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.704+0000 7ff4a1ffb700 1 -- 172.21.15.154:0/3932338165 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff494057e20 con 0x7ff4a40a9f90 2024-03-14T09:01:18.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.707+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 msgr2=0x7ff49006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:18.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.707+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 0x7ff49006ccf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff49c007d40 tx=0x7ff49c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:18.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.707+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 msgr2=0x7ff4a40ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:18.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.707+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4070df0 tx=0x7ff49400f490 comp rx=0 tx=0).stop 2024-03-14T09:01:18.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 shutdown_connections 2024-03-14T09:01:18.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff49006a830 0x7ff49006ccf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4a40a9f90 0x7ff4a40ac370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff48c004150 0x7ff4a414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 --2- 172.21.15.154:0/3932338165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4a4154d40 0x7ff4a41e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.708+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 >> 172.21.15.154:0/3932338165 conn(0x7ff4a40c3880 msgr2=0x7ff4a40c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:18.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.709+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 shutdown_connections 2024-03-14T09:01:18.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.709+0000 7ff4ab9d9700 1 -- 172.21.15.154:0/3932338165 wait complete. 2024-03-14T09:01:18.714 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2024-03-14T09:01:18.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T09:01:18.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.855+0000 7f500fa2e700 1 Processor -- start 2024-03-14T09:01:18.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.855+0000 7f500fa2e700 1 -- start start 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f500806c6e0 0x7f500806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5008070a10 0x7f50080c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806d600 0x7f50080c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080c5f10 con 0x7f500806c6e0 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080c7d50 con 0x7f5008070a10 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500fa2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50080c8030 con 0x7f500806d600 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500d7ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f500806c6e0 0x7f500806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500d7ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f500806c6e0 0x7f500806d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45030/0 (socket says 172.21.15.154:45030) 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500d7ca700 1 -- 172.21.15.154:0/247739339 learned_addr learned my addr 172.21.15.154:0/247739339 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500dfcb700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806d600 0x7f50080c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.856+0000 7f500cfc9700 1 --2- 172.21.15.154:0/247739339 >> v2:172.21.15.191:3300/0 conn(0x7f5008070a10 0x7f50080c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 -- 172.21.15.154:0/247739339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806d600 msgr2=0x7f50080c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806d600 0x7f50080c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 -- 172.21.15.154:0/247739339 >> v2:172.21.15.191:3300/0 conn(0x7f5008070a10 msgr2=0x7f50080c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 --2- 172.21.15.154:0/247739339 >> v2:172.21.15.191:3300/0 conn(0x7f5008070a10 0x7f50080c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 -- 172.21.15.154:0/247739339 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080c8310 con 0x7f500806c6e0 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500d7ca700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500806c6e0 0x7f500806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50080c4620 tx=0x7f4ffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54ae92fbcfdbf07 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f4ffbfff700 1 -- 172.21.15.154:0/247739339 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00b870 con 0x7f500806c6e0 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f4ffbfff700 1 -- 172.21.15.154:0/247739339 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc00b9d0 con 0x7f500806c6e0 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f4ffbfff700 1 -- 172.21.15.154:0/247739339 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00bd60 con 0x7f500806c6e0 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 -- 172.21.15.154:0/247739339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500806c6e0 msgr2=0x7f500806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500806c6e0 0x7f500806d0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50080c4620 tx=0x7f4ffc009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 -- 172.21.15.154:0/247739339 shutdown_connections 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806d600 0x7f50080c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 --2- 172.21.15.154:0/247739339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f500806c6e0 0x7f500806d0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 --2- 172.21.15.154:0/247739339 >> v2:172.21.15.191:3300/0 conn(0x7f5008070a10 0x7f50080c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 -- 172.21.15.154:0/247739339 >> 172.21.15.154:0/247739339 conn(0x7f50080a9bf0 msgr2=0x7f50080a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:18.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 -- 172.21.15.154:0/247739339 shutdown_connections 2024-03-14T09:01:18.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.857+0000 7f500fa2e700 1 -- 172.21.15.154:0/247739339 wait complete. 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 Processor -- start 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 -- start start 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 0x7f50081ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 0x7f50081dca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500809bc20 con 0x7f5008070a10 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080ac4e0 con 0x7f500806d600 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500fa2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50080abf80 con 0x7f500806c6e0 2024-03-14T09:01:18.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.859+0000 7f500dfcb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45044/0 (socket says 172.21.15.154:45044) 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 -- 172.21.15.154:0/2716634808 learned_addr learned my addr 172.21.15.154:0/2716634808 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500d7ca700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 0x7f50081ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500cfc9700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 0x7f50081dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 -- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 msgr2=0x7f50081ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 0x7f50081ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 -- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 msgr2=0x7f50081dca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 0x7f50081dca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 -- 172.21.15.154:0/2716634808 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009550 con 0x7f5008070a10 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500cfc9700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 0x7f50081dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500d7ca700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 0x7f50081ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:18.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500dfcb700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f500000cd70 tx=0x7f500000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000003a20 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000003b80 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50081ee120 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000004020 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.860+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50081dcf80 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.861+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50080ac690 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.862+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5000004180 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.862+0000 7f4ff9ffb700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 0x7f4fec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.862+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5000012070 con 0x7f5008070a10 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.862+0000 7f500d7ca700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 0x7f4fec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:18.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.863+0000 7f500d7ca700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 0x7f4fec06ccf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc000c00 tx=0x7f4ffc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:18.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:18.867+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f50080ac690 con 0x7f5008070a10 2024-03-14T09:01:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.105+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f50080c3190 con 0x7f5008070a10 2024-03-14T09:01:19.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.110+0000 7f4ff9ffb700 1 -- 172.21.15.154:0/2716634808 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5000054930 con 0x7f5008070a10 2024-03-14T09:01:19.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.117+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 msgr2=0x7f4fec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.117+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 0x7f4fec06ccf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc000c00 tx=0x7f4ffc008030 comp rx=0 tx=0).stop 2024-03-14T09:01:19.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.117+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 msgr2=0x7f50080a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.117+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f500000cd70 tx=0x7f500000a570 comp rx=0 tx=0).stop 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 shutdown_connections 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fec06a830 0x7f4fec06ccf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5008070a10 0x7f50080a6dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f500806c6e0 0x7f50081ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 --2- 172.21.15.154:0/2716634808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f500806d600 0x7f50081dca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.121+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 >> 172.21.15.154:0/2716634808 conn(0x7f50080a9bf0 msgr2=0x7f50080a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.124+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 shutdown_connections 2024-03-14T09:01:19.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.125+0000 7f500fa2e700 1 -- 172.21.15.154:0/2716634808 wait complete. 2024-03-14T09:01:19.126 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 2 and data pool 3 2024-03-14T09:01:19.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:19.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.263+0000 7f84e110a700 1 Processor -- start 2024-03-14T09:01:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.263+0000 7f84e110a700 1 -- start start 2024-03-14T09:01:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84e110a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f84dc15e280 0x7f84dc160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84e110a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc0ac8e0 con 0x7f84dc14eaf0 2024-03-14T09:01:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84e110a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84dc06de30 con 0x7f84dc155df0 2024-03-14T09:01:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84e110a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84dc06b310 con 0x7f84dc15e280 2024-03-14T09:01:19.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84dad9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f84dc14eaf0 0x7f84dc06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58972/0 (socket says 172.21.15.154:58972) 2024-03-14T09:01:19.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84dad9d700 1 -- 172.21.15.154:0/2312942379 learned_addr learned my addr 172.21.15.154:0/2312942379 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:19.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84db59e700 1 --2- 172.21.15.154:0/2312942379 >> v2:172.21.15.191:3300/0 conn(0x7f84dc15e280 0x7f84dc160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.264+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373877726 0 0) 0x7f84dc0ac8e0 con 0x7f84dc14eaf0 2024-03-14T09:01:19.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c40035b0 con 0x7f84dc14eaf0 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 942111644 0 0) 0x7f84c40035b0 con 0x7f84dc14eaf0 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 >> v2:172.21.15.191:3300/0 conn(0x7f84dc15e280 msgr2=0x7f84dc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 --2- 172.21.15.154:0/2312942379 >> v2:172.21.15.191:3300/0 conn(0x7f84dc15e280 0x7f84dc160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 >> v1:172.21.15.191:6789/0 conn(0x7f84dc155df0 legacy=0x7f84dc156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc156960 con 0x7f84dc14eaf0 2024-03-14T09:01:19.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f84cc0019f0 con 0x7f84dc14eaf0 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 >> v1:172.21.15.154:6789/0 conn(0x7f84dc14eaf0 legacy=0x7f84dc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4003a00 0x7f84c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c4004850 0x7f84c4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 0x7f84c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc06b310 con 0x7f84c4005330 2024-03-14T09:01:19.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400c200 con 0x7f84c4004850 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c400c3c0 con 0x7f84c4003a00 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 0x7f84c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84dad9d700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4003a00 0x7f84c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84da59c700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c4004850 0x7f84c4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 -- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4003a00 msgr2=0x7f84c4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4003a00 0x7f84c4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 -- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c4004850 msgr2=0x7f84c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c4004850 0x7f84c4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 -- 172.21.15.154:0/2312942379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c40035b0 con 0x7f84c4005330 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 -- 172.21.15.154:0/2312942379 reap_dead start 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.265+0000 7f84db59e700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 0x7f84c400ba10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009b30 tx=0x7f84c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32404fcaef6ca886 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4004020 con 0x7f84c4005330 2024-03-14T09:01:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d4004180 con 0x7f84c4005330 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84d9d9b700 1 -- 172.21.15.154:0/2312942379 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4004510 con 0x7f84c4005330 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 -- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 msgr2=0x7f84c400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 0x7f84c400ba10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009b30 tx=0x7f84c4005880 comp rx=0 tx=0).stop 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 -- 172.21.15.154:0/2312942379 shutdown_connections 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 --2- 172.21.15.154:0/2312942379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c4005330 0x7f84c400ba10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 -- 172.21.15.154:0/2312942379 >> 172.21.15.154:0/2312942379 conn(0x7f84dc0c3880 msgr2=0x7f84dc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 -- 172.21.15.154:0/2312942379 shutdown_connections 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.266+0000 7f84e110a700 1 -- 172.21.15.154:0/2312942379 wait complete. 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 Processor -- start 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 -- start start 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4005330 0x7f84dc14a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84dc1489d0 0x7f84dc143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc143e90 con 0x7f84dc1521b0 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc151d00 con 0x7f84dc1489d0 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84e110a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1513d0 con 0x7f84c4005330 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84da59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84da59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45066/0 (socket says 172.21.15.154:45066) 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84da59c700 1 -- 172.21.15.154:0/2558046253 learned_addr learned my addr 172.21.15.154:0/2558046253 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84db59e700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84dc1489d0 0x7f84dc143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.268+0000 7f84dad9d700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4005330 0x7f84dc14a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 -- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4005330 msgr2=0x7f84dc14a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4005330 0x7f84dc14a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 -- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84dc1489d0 msgr2=0x7f84dc143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84dc1489d0 0x7f84dc143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 -- 172.21.15.154:0/2558046253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc1516b0 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84da59c700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84d000f820 tx=0x7f84d000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d00052e0 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0005440 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc151990 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d00038a0 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.269+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc151060 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.270+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc0697d0 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.270+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f84d0003a00 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.271+0000 7f84c37fe700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 0x7f84c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.271+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84d00888c0 con 0x7f84dc1521b0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.271+0000 7f84dad9d700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 0x7f84c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.271+0000 7f84dad9d700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 0x7f84c806ccf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84cc007180 tx=0x7f84cc006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.275+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f84d0053d70 con 0x7f84dc1521b0 2024-03-14T09:01:19.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.471+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84dc06d870 con 0x7f84dc1521b0 2024-03-14T09:01:19.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.472+0000 7f84c37fe700 1 -- 172.21.15.154:0/2558046253 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16098 (secure 0 0 0) 0x7f84d0003cb0 con 0x7f84dc1521b0 2024-03-14T09:01:19.473 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:19.473 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":19,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:19.106625+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:19.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 msgr2=0x7f84c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 0x7f84c806ccf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84cc007180 tx=0x7f84cc006040 comp rx=0 tx=0).stop 2024-03-14T09:01:19.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 msgr2=0x7f84dc143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84d000f820 tx=0x7f84d000fb30 comp rx=0 tx=0).stop 2024-03-14T09:01:19.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 shutdown_connections 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84dc1489d0 0x7f84dc143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84c4005330 0x7f84dc14a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f84c806a830 0x7f84c806ccf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 --2- 172.21.15.154:0/2558046253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84dc1521b0 0x7f84dc143410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 >> 172.21.15.154:0/2558046253 conn(0x7f84dc0c3880 msgr2=0x7f84dc154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.474+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 shutdown_connections 2024-03-14T09:01:19.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.475+0000 7f84e110a700 1 -- 172.21.15.154:0/2558046253 wait complete. 2024-03-14T09:01:19.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:19.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.609+0000 7f6b17b4c700 1 Processor -- start 2024-03-14T09:01:19.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.609+0000 7f6b17b4c700 1 -- start start 2024-03-14T09:01:19.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b17b4c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100c4af0 0x7f6b100c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b17b4c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6b1015d370 0x7f6b1015f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b17b4c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100c55e0 con 0x7f6b1006b300 2024-03-14T09:01:19.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b17b4c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100c5ac0 con 0x7f6b100c4af0 2024-03-14T09:01:19.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b17b4c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100c8290 con 0x7f6b1015d370 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100c4af0 0x7f6b100c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b100c4af0 0x7f6b100c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48508/0 (socket says 172.21.15.154:48508) 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 -- 172.21.15.154:0/1685753520 learned_addr learned my addr 172.21.15.154:0/1685753520 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b160e9700 1 --2- 172.21.15.154:0/1685753520 >> v2:172.21.15.191:3300/0 conn(0x7f6b1015d370 0x7f6b1015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 -- 172.21.15.154:0/1685753520 >> v2:172.21.15.191:3300/0 conn(0x7f6b1015d370 msgr2=0x7f6b1015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 --2- 172.21.15.154:0/1685753520 >> v2:172.21.15.191:3300/0 conn(0x7f6b1015d370 0x7f6b1015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 -- 172.21.15.154:0/1685753520 >> v1:172.21.15.191:6789/0 conn(0x7f6b1006b300 legacy=0x7f6b1006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 -- 172.21.15.154:0/1685753520 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10154bf0 con 0x7f6b100c4af0 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.610+0000 7f6b158e8700 1 --2- 172.21.15.154:0/1685753520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b100c4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b100c6ba0 tx=0x7f6b0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1aa9883a79aeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b148e6700 1 -- 172.21.15.154:0/1685753520 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004230 con 0x7f6b100c4af0 2024-03-14T09:01:19.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b148e6700 1 -- 172.21.15.154:0/1685753520 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c004390 con 0x7f6b100c4af0 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b148e6700 1 -- 172.21.15.154:0/1685753520 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0036a0 con 0x7f6b100c4af0 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 -- 172.21.15.154:0/1685753520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 msgr2=0x7f6b100c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/1685753520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b100c4ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b100c6ba0 tx=0x7f6b0c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 -- 172.21.15.154:0/1685753520 shutdown_connections 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/1685753520 >> v2:172.21.15.191:3300/0 conn(0x7f6b1015d370 0x7f6b1015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/1685753520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b100c4ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 -- 172.21.15.154:0/1685753520 >> 172.21.15.154:0/1685753520 conn(0x7f6b100abc60 msgr2=0x7f6b100ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 -- 172.21.15.154:0/1685753520 shutdown_connections 2024-03-14T09:01:19.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.611+0000 7f6b17b4c700 1 -- 172.21.15.154:0/1685753520 wait complete. 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.612+0000 7f6b17b4c700 1 Processor -- start 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.612+0000 7f6b17b4c700 1 -- start start 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1006b300 0x7f6b101514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1015d370 0x7f6b101eb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10158820 con 0x7f6b100c4af0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101eedc0 con 0x7f6b1015d370 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b17b4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100c0dc0 con 0x7f6b1006b300 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48520/0 (socket says 172.21.15.154:48520) 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 -- 172.21.15.154:0/712479537 learned_addr learned my addr 172.21.15.154:0/712479537 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b160e9700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1015d370 0x7f6b101eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b158e8700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1006b300 0x7f6b101514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 -- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1006b300 msgr2=0x7f6b101514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1006b300 0x7f6b101514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 -- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1015d370 msgr2=0x7f6b101eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1015d370 0x7f6b101eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.613+0000 7f6b150e7700 1 -- 172.21.15.154:0/712479537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009550 con 0x7f6b100c4af0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b150e7700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c500 tx=0x7f6b0000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00011790 con 0x7f6b100c4af0 2024-03-14T09:01:19.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000118f0 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b101eab70 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00011ca0 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.614+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b100c0ae0 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.615+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10067bd0 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.615+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6b0001c460 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.615+0000 7f6b067fc700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 0x7f6af406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.616+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b00088970 con 0x7f6b100c4af0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.616+0000 7f6b158e8700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 0x7f6af406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.616+0000 7f6b158e8700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 0x7f6af406ccf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009130 tx=0x7f6b0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.620+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6b00053e20 con 0x7f6b100c4af0 2024-03-14T09:01:19.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.820+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b101ea940 con 0x7f6b100c4af0 2024-03-14T09:01:19.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.821+0000 7f6b067fc700 1 -- 172.21.15.154:0/712479537 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16098 (secure 0 0 0) 0x7f6b101ea940 con 0x7f6b100c4af0 2024-03-14T09:01:19.820 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":19,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:19.106625+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 msgr2=0x7f6af406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 0x7f6af406ccf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009130 tx=0x7f6b0c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 msgr2=0x7f6b101eb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c500 tx=0x7f6b0000c810 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 shutdown_connections 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1015d370 0x7f6b101eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1006b300 0x7f6b101514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6af406a830 0x7f6af406ccf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 --2- 172.21.15.154:0/712479537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b100c4af0 0x7f6b101eb000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 >> 172.21.15.154:0/712479537 conn(0x7f6b100abc60 msgr2=0x7f6b100c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 shutdown_connections 2024-03-14T09:01:19.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.823+0000 7f6b17b4c700 1 -- 172.21.15.154:0/712479537 wait complete. 2024-03-14T09:01:19.835 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T09:01:19.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.959+0000 7f8f04003700 1 Processor -- start 2024-03-14T09:01:19.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.959+0000 7f8f04003700 1 -- start start 2024-03-14T09:01:19.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f04003700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f04003700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f04003700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc0ac8e0 con 0x7f8efc156330 2024-03-14T09:01:19.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f04003700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc06b310 con 0x7f8efc14eaf0 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f04003700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc06de30 con 0x7f8efc06acb0 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f01d9f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:45876/0 (socket says 172.21.15.154:45876) 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f01d9f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38190/0 (socket says 172.21.15.154:38190) 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f025a0700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8efc156330 0x7f8efc157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38238/0 (socket says 172.21.15.154:38238) 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 -- 172.21.15.154:0/2095437333 learned_addr learned my addr 172.21.15.154:0/2095437333 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 -- 172.21.15.154:0/2095437333 >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 msgr2=0x7f8efc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 --2- 172.21.15.154:0/2095437333 >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 -- 172.21.15.154:0/2095437333 >> v1:172.21.15.191:6789/0 conn(0x7f8efc156330 legacy=0x7f8efc157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 -- 172.21.15.154:0/2095437333 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0acb30 con 0x7f8efc06acb0 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f01d9f700 1 --2- 172.21.15.154:0/2095437333 >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.960+0000 7f8f0159e700 1 --2- 172.21.15.154:0/2095437333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0098f0 tx=0x7f8eec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130adb470f590cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f00d9d700 1 -- 172.21.15.154:0/2095437333 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec011830 con 0x7f8efc06acb0 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f00d9d700 1 -- 172.21.15.154:0/2095437333 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec011990 con 0x7f8efc06acb0 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f00d9d700 1 -- 172.21.15.154:0/2095437333 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec011d20 con 0x7f8efc06acb0 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 -- 172.21.15.154:0/2095437333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 msgr2=0x7f8efc155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 --2- 172.21.15.154:0/2095437333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0098f0 tx=0x7f8eec009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 -- 172.21.15.154:0/2095437333 shutdown_connections 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 --2- 172.21.15.154:0/2095437333 >> v2:172.21.15.191:3300/0 conn(0x7f8efc14eaf0 0x7f8efc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 --2- 172.21.15.154:0/2095437333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc155df0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 -- 172.21.15.154:0/2095437333 >> 172.21.15.154:0/2095437333 conn(0x7f8efc0c3880 msgr2=0x7f8efc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 -- 172.21.15.154:0/2095437333 shutdown_connections 2024-03-14T09:01:19.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.961+0000 7f8f04003700 1 -- 172.21.15.154:0/2095437333 wait complete. 2024-03-14T09:01:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.962+0000 7f8f04003700 1 Processor -- start 2024-03-14T09:01:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.962+0000 7f8f04003700 1 -- start start 2024-03-14T09:01:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc156330 0x7f8efc141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc154650 con 0x7f8efc14eaf0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc142750 con 0x7f8efc156330 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f04003700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1429b0 con 0x7f8efc06acb0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48534/0 (socket says 172.21.15.154:48534) 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 -- 172.21.15.154:0/1304240268 learned_addr learned my addr 172.21.15.154:0/1304240268 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f01d9f700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f025a0700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc156330 0x7f8efc141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 -- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 msgr2=0x7f8efc142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 -- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc156330 msgr2=0x7f8efc141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc156330 0x7f8efc141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.963+0000 7f8f0159e700 1 -- 172.21.15.154:0/1304240268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80094a0 con 0x7f8efc14eaf0 2024-03-14T09:01:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8f0159e700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009c40 tx=0x7f8eec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec004510 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec01e070 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec009550 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0038e0 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.964+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc143440 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.965+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc1432a0 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.965+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8eec003a40 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.965+0000 7f8ef27fc700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 0x7f8ee006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.966+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8eec089b70 con 0x7f8efc14eaf0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.966+0000 7f8f01d9f700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 0x7f8ee006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:19.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.966+0000 7f8f01d9f700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 0x7f8ee006ccf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009cd0 tx=0x7f8ef8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:19.970+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8efc1432a0 con 0x7f8efc14eaf0 2024-03-14T09:01:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:20.211+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8efc1432a0 con 0x7f8efc14eaf0 2024-03-14T09:01:21.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.128+0000 7f8ef27fc700 1 -- 172.21.15.154:0/1304240268 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8efc1432a0 con 0x7f8efc14eaf0 2024-03-14T09:01:21.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 msgr2=0x7f8ee006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 0x7f8ee006ccf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009cd0 tx=0x7f8ef8009210 comp rx=0 tx=0).stop 2024-03-14T09:01:21.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 msgr2=0x7f8efc0c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009c40 tx=0x7f8eec004750 comp rx=0 tx=0).stop 2024-03-14T09:01:21.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 shutdown_connections 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc156330 0x7f8efc141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc06acb0 0x7f8efc142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee006a830 0x7f8ee006ccf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 --2- 172.21.15.154:0/1304240268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc14eaf0 0x7f8efc0c6330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 >> 172.21.15.154:0/1304240268 conn(0x7f8efc0c3880 msgr2=0x7f8efc155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 shutdown_connections 2024-03-14T09:01:21.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.131+0000 7f8f04003700 1 -- 172.21.15.154:0/1304240268 wait complete. 2024-03-14T09:01:21.145 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:01:21.145 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T09:01:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.276+0000 7fd463262700 1 Processor -- start 2024-03-14T09:01:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 -- start start 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd45c14e980 0x7fd45c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd45c158a80 0x7fd45c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c14ed60 con 0x7fd45c0c4a80 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c0c6b10 con 0x7fd45c14e980 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd463262700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c155820 con 0x7fd45c158a80 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd460ffe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd45c14e980 0x7fd45c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd45bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd45c0c4a80 0x7fd45c158370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34450/0 (socket says 172.21.15.154:34450) 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd45bfff700 1 -- 172.21.15.154:0/3389832474 learned_addr learned my addr 172.21.15.154:0/3389832474 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.277+0000 7fd460ffe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd45c14e980 0x7fd45c1552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48536/0 (socket says 172.21.15.154:48536) 2024-03-14T09:01:21.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd4617ff700 1 --2- 172.21.15.154:0/3389832474 >> v2:172.21.15.191:3300/0 conn(0x7fd45c158a80 0x7fd45c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd460ffe700 1 -- 172.21.15.154:0/3389832474 >> v2:172.21.15.191:3300/0 conn(0x7fd45c158a80 msgr2=0x7fd45c0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd460ffe700 1 --2- 172.21.15.154:0/3389832474 >> v2:172.21.15.191:3300/0 conn(0x7fd45c158a80 0x7fd45c0c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd460ffe700 1 -- 172.21.15.154:0/3389832474 >> v1:172.21.15.154:6789/0 conn(0x7fd45c0c4a80 legacy=0x7fd45c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd460ffe700 1 -- 172.21.15.154:0/3389832474 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c0c6cf0 con 0x7fd45c14e980 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd45b7fe700 1 -- 172.21.15.154:0/3389832474 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857293398 0 0) 0x7fd45c14ed60 con 0x7fd45c0c4a80 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd460ffe700 1 --2- 172.21.15.154:0/3389832474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c14e980 0x7fd45c1552e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0c9fc0 tx=0x7fd44c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df0cfbfc1ec6266 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd45b7fe700 1 -- 172.21.15.154:0/3389832474 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c011830 con 0x7fd45c14e980 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd45b7fe700 1 -- 172.21.15.154:0/3389832474 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c011990 con 0x7fd45c14e980 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.278+0000 7fd45b7fe700 1 -- 172.21.15.154:0/3389832474 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c011d20 con 0x7fd45c14e980 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 -- 172.21.15.154:0/3389832474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c14e980 msgr2=0x7fd45c1552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 --2- 172.21.15.154:0/3389832474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c14e980 0x7fd45c1552e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0c9fc0 tx=0x7fd44c009b20 comp rx=0 tx=0).stop 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 -- 172.21.15.154:0/3389832474 shutdown_connections 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 --2- 172.21.15.154:0/3389832474 >> v2:172.21.15.191:3300/0 conn(0x7fd45c158a80 0x7fd45c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 --2- 172.21.15.154:0/3389832474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c14e980 0x7fd45c1552e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 -- 172.21.15.154:0/3389832474 >> 172.21.15.154:0/3389832474 conn(0x7fd45c0abcf0 msgr2=0x7fd45c0ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:21.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 -- 172.21.15.154:0/3389832474 shutdown_connections 2024-03-14T09:01:21.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.279+0000 7fd463262700 1 -- 172.21.15.154:0/3389832474 wait complete. 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.280+0000 7fd463262700 1 Processor -- start 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.280+0000 7fd463262700 1 -- start start 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd45c0c4a80 0x7fd45c1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd45c14e980 0x7fd45c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c1ee400 con 0x7fd45c158a80 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c154c60 con 0x7fd45c14e980 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c152b90 con 0x7fd45c0c4a80 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48538/0 (socket says 172.21.15.154:48538) 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 -- 172.21.15.154:0/2186177364 learned_addr learned my addr 172.21.15.154:0/2186177364 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:21.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd45bfff700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd45c14e980 0x7fd45c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd460ffe700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd45c0c4a80 0x7fd45c1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 -- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd45c0c4a80 msgr2=0x7fd45c1edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd45c0c4a80 0x7fd45c1edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 -- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd45c14e980 msgr2=0x7fd45c14aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd45c14e980 0x7fd45c14aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 -- 172.21.15.154:0/2186177364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0094a0 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4617ff700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd45400cd70 tx=0x7fd45400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454003a20 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd454003b80 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454004020 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c14b950 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.281+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c0af4f0 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.282+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c0697d0 con 0x7fd45c158a80 2024-03-14T09:01:21.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.283+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd454004180 con 0x7fd45c158a80 2024-03-14T09:01:21.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.283+0000 7fd4597fa700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 0x7fd44806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:21.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.283+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd454012070 con 0x7fd45c158a80 2024-03-14T09:01:21.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.283+0000 7fd460ffe700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 0x7fd44806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:21.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.284+0000 7fd460ffe700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 0x7fd44806ccf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0090b0 tx=0x7fd44c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:21.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.288+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd4540549d0 con 0x7fd45c158a80 2024-03-14T09:01:21.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:21.526+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd45c06cc80 con 0x7fd45c158a80 2024-03-14T09:01:22.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.306+0000 7fd4597fa700 1 -- 172.21.15.154:0/2186177364 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd454057ff0 con 0x7fd45c158a80 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 msgr2=0x7fd44806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 0x7fd44806ccf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0090b0 tx=0x7fd44c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 msgr2=0x7fd45c14b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd45400cd70 tx=0x7fd45400a570 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 shutdown_connections 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd44806a830 0x7fd44806ccf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd45c158a80 0x7fd45c14b410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd45c0c4a80 0x7fd45c1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 --2- 172.21.15.154:0/2186177364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd45c14e980 0x7fd45c14aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 >> 172.21.15.154:0/2186177364 conn(0x7fd45c0abcf0 msgr2=0x7fd45c06b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 shutdown_connections 2024-03-14T09:01:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.309+0000 7fd463262700 1 -- 172.21.15.154:0/2186177364 wait complete. 2024-03-14T09:01:22.324 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T09:01:22.325 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.452+0000 7fac3e54c700 1 Processor -- start 2024-03-14T09:01:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3e54c700 1 -- start start 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3e54c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac38151fd0 0x7fac3815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3e54c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac38155a40 con 0x7fac3814edf0 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3e54c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac38072a90 con 0x7fac380c26c0 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3e54c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac380c2f10 con 0x7fac38151fd0 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac3cae9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac38151fd0 0x7fac3815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac377fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fac3814edf0 0x7fac381518c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34456/0 (socket says 172.21.15.154:34456) 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.453+0000 7fac377fe700 1 -- 172.21.15.154:0/2280819800 learned_addr learned my addr 172.21.15.154:0/2280819800 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac3cae9700 1 -- 172.21.15.154:0/2280819800 >> v1:172.21.15.191:6789/0 conn(0x7fac380c26c0 legacy=0x7fac380c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac3cae9700 1 -- 172.21.15.154:0/2280819800 >> v1:172.21.15.154:6789/0 conn(0x7fac3814edf0 legacy=0x7fac381518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac3cae9700 1 -- 172.21.15.154:0/2280819800 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac381556d0 con 0x7fac38151fd0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac36ffd700 1 -- 172.21.15.154:0/2280819800 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038112127 0 0) 0x7fac38155a40 con 0x7fac3814edf0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac3cae9700 1 --2- 172.21.15.154:0/2280819800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac38151fd0 0x7fac3815ce60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac30009d90 tx=0x7fac3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46bd054f2c9118a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac36ffd700 1 -- 172.21.15.154:0/2280819800 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000e070 con 0x7fac38151fd0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac36ffd700 1 -- 172.21.15.154:0/2280819800 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000bbd0 con 0x7fac38151fd0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.454+0000 7fac36ffd700 1 -- 172.21.15.154:0/2280819800 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30012430 con 0x7fac38151fd0 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 -- 172.21.15.154:0/2280819800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac38151fd0 msgr2=0x7fac3815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 --2- 172.21.15.154:0/2280819800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac38151fd0 0x7fac3815ce60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac30009d90 tx=0x7fac3000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 -- 172.21.15.154:0/2280819800 shutdown_connections 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 --2- 172.21.15.154:0/2280819800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac38151fd0 0x7fac3815ce60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 -- 172.21.15.154:0/2280819800 >> 172.21.15.154:0/2280819800 conn(0x7fac3806a770 msgr2=0x7fac3806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 -- 172.21.15.154:0/2280819800 shutdown_connections 2024-03-14T09:01:22.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.455+0000 7fac3e54c700 1 -- 172.21.15.154:0/2280819800 wait complete. 2024-03-14T09:01:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.456+0000 7fac3e54c700 1 Processor -- start 2024-03-14T09:01:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.456+0000 7fac3e54c700 1 -- start start 2024-03-14T09:01:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac380c26c0 0x7fac381ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac38151fd0 0x7fac38150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3814beb0 con 0x7fac3814edf0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac38153540 con 0x7fac380c26c0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac38153820 con 0x7fac38151fd0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48552/0 (socket says 172.21.15.154:48552) 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 -- 172.21.15.154:0/3347012164 learned_addr learned my addr 172.21.15.154:0/3347012164 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac37fff700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac380c26c0 0x7fac381ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3cae9700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac38151fd0 0x7fac38150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 -- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac38151fd0 msgr2=0x7fac38150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac38151fd0 0x7fac38150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 -- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac380c26c0 msgr2=0x7fac381ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac380c26c0 0x7fac381ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 -- 172.21.15.154:0/3347012164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30009a70 con 0x7fac3814edf0 2024-03-14T09:01:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac377fe700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c520 tx=0x7fac2c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c004c80 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c004e00 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac381ed9e0 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c007310 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.457+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac381edfa0 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.458+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac381ee1c0 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.459+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fac2c007470 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.459+0000 7fac34ff9700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 0x7fac2406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.459+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac2c0888f0 con 0x7fac3814edf0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.459+0000 7fac37fff700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 0x7fac2406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.460+0000 7fac37fff700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 0x7fac2406ccf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac3814c7d0 tx=0x7fac28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.463+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fac2c053da0 con 0x7fac3814edf0 2024-03-14T09:01:22.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.701+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac380697d0 con 0x7fac3814edf0 2024-03-14T09:01:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.701+0000 7fac34ff9700 1 -- 172.21.15.154:0/3347012164 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fac2c0573c0 con 0x7fac3814edf0 2024-03-14T09:01:22.701 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:22.701 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:22.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 msgr2=0x7fac2406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 0x7fac2406ccf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac3814c7d0 tx=0x7fac28009040 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 msgr2=0x7fac3814b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c520 tx=0x7fac2c00c830 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 shutdown_connections 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac38151fd0 0x7fac38150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac380c26c0 0x7fac381ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac2406a830 0x7fac2406ccf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 --2- 172.21.15.154:0/3347012164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac3814edf0 0x7fac3814b970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 >> 172.21.15.154:0/3347012164 conn(0x7fac3806a770 msgr2=0x7fac3814dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 shutdown_connections 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.703+0000 7fac3e54c700 1 -- 172.21.15.154:0/3347012164 wait complete. 2024-03-14T09:01:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:22.715 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:22.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 Processor -- start 2024-03-14T09:01:22.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 -- start start 2024-03-14T09:01:22.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed20158840 0x7fed200c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fed200bf9c0 0x7fed200bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed20154f40 con 0x7fed201553d0 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed20154930 con 0x7fed20158840 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.838+0000 7fed28443700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed200c57e0 con 0x7fed200bf9c0 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed20158840 0x7fed200c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed20158840 0x7fed200c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48560/0 (socket says 172.21.15.154:48560) 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 -- 172.21.15.154:0/2615921839 learned_addr learned my addr 172.21.15.154:0/2615921839 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed269e0700 1 --2- 172.21.15.154:0/2615921839 >> v2:172.21.15.191:3300/0 conn(0x7fed200bf9c0 0x7fed200bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 -- 172.21.15.154:0/2615921839 >> v2:172.21.15.191:3300/0 conn(0x7fed200bf9c0 msgr2=0x7fed200bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 --2- 172.21.15.154:0/2615921839 >> v2:172.21.15.191:3300/0 conn(0x7fed200bf9c0 0x7fed200bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 -- 172.21.15.154:0/2615921839 >> v1:172.21.15.191:6789/0 conn(0x7fed201553d0 legacy=0x7fed200c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 -- 172.21.15.154:0/2615921839 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200c5ac0 con 0x7fed20158840 2024-03-14T09:01:22.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed259de700 1 --2- 172.21.15.154:0/2615921839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed20158840 0x7fed200c0f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed20067500 tx=0x7fed14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188de95fb771f097 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed251dd700 1 -- 172.21.15.154:0/2615921839 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14004230 con 0x7fed20158840 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed251dd700 1 -- 172.21.15.154:0/2615921839 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14004390 con 0x7fed20158840 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.839+0000 7fed251dd700 1 -- 172.21.15.154:0/2615921839 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140036a0 con 0x7fed20158840 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 -- 172.21.15.154:0/2615921839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed20158840 msgr2=0x7fed200c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 --2- 172.21.15.154:0/2615921839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed20158840 0x7fed200c0f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed20067500 tx=0x7fed14009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 -- 172.21.15.154:0/2615921839 shutdown_connections 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 --2- 172.21.15.154:0/2615921839 >> v2:172.21.15.191:3300/0 conn(0x7fed200bf9c0 0x7fed200bfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 --2- 172.21.15.154:0/2615921839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed20158840 0x7fed200c0f90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 -- 172.21.15.154:0/2615921839 >> 172.21.15.154:0/2615921839 conn(0x7fed2006a770 msgr2=0x7fed2006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 -- 172.21.15.154:0/2615921839 shutdown_connections 2024-03-14T09:01:22.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.840+0000 7fed28443700 1 -- 172.21.15.154:0/2615921839 wait complete. 2024-03-14T09:01:22.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.841+0000 7fed28443700 1 Processor -- start 2024-03-14T09:01:22.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.841+0000 7fed28443700 1 -- start start 2024-03-14T09:01:22.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed200bf9c0 0x7fed201edab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20158840 0x7fed200718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2014b0f0 con 0x7fed201553d0 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200a93d0 con 0x7fed200bf9c0 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed201dde90 con 0x7fed20158840 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48564/0 (socket says 172.21.15.154:48564) 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 -- 172.21.15.154:0/1038111201 learned_addr learned my addr 172.21.15.154:0/1038111201 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 -- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20158840 msgr2=0x7fed200718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20158840 0x7fed200718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed261df700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed200bf9c0 0x7fed201edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 -- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed200bf9c0 msgr2=0x7fed201edab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed200bf9c0 0x7fed201edab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 -- 172.21.15.154:0/1038111201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14009550 con 0x7fed201553d0 2024-03-14T09:01:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed259de700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed14009c10 tx=0x7fed14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14003d80 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1401f070 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed201de0e0 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.842+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14011850 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.843+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed200ab5f0 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.843+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200ab810 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.844+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fed14021030 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.844+0000 7fed12ffd700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 0x7fed0406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.844+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed1408a250 con 0x7fed201553d0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.844+0000 7fed261df700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 0x7fed0406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.845+0000 7fed261df700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 0x7fed0406cd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed1c001670 tx=0x7fed1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:22.849+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fed14055700 con 0x7fed201553d0 2024-03-14T09:01:23.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.043+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed200697d0 con 0x7fed201553d0 2024-03-14T09:01:23.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.044+0000 7fed12ffd700 1 -- 172.21.15.154:0/1038111201 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fed14058d20 con 0x7fed201553d0 2024-03-14T09:01:23.044 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:23.044 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 msgr2=0x7fed0406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 0x7fed0406cd20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed1c001670 tx=0x7fed1c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 msgr2=0x7fed2014abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed14009c10 tx=0x7fed14004750 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 shutdown_connections 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20158840 0x7fed200718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed200bf9c0 0x7fed201edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed0406a860 0x7fed0406cd20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 --2- 172.21.15.154:0/1038111201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed201553d0 0x7fed2014abb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 >> 172.21.15.154:0/1038111201 conn(0x7fed2006a770 msgr2=0x7fed201539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 shutdown_connections 2024-03-14T09:01:23.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.047+0000 7fed28443700 1 -- 172.21.15.154:0/1038111201 wait complete. 2024-03-14T09:01:23.059 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:01:23.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:01:23.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.181+0000 7f6214fdf700 1 Processor -- start 2024-03-14T09:01:23.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.181+0000 7f6214fdf700 1 -- start start 2024-03-14T09:01:23.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f6214fdf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f62100c6490 0x7f62100c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f6214fdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f621006cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f6214fdf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100c3760 con 0x7f621006d440 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f6214fdf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100c6210 con 0x7f62100c6490 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f6214fdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c3230 con 0x7f62100c7dc0 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620e59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f62100c6490 0x7f62100c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620dd9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f621006cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620e59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f62100c6490 0x7f62100c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38242/0 (socket says 172.21.15.154:38242) 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620e59c700 1 -- 172.21.15.154:0/2481284432 learned_addr learned my addr 172.21.15.154:0/2481284432 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620ed9d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f621006d440 0x7f62100c1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38258/0 (socket says 172.21.15.154:38258) 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.182+0000 7f620dd9b700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f621006cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:45908/0 (socket says 172.21.15.154:45908) 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620e59c700 1 -- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 msgr2=0x7f621006cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620e59c700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f621006cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620e59c700 1 -- 172.21.15.154:0/2481284432 >> v1:172.21.15.191:6789/0 conn(0x7f621006d440 legacy=0x7f62100c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620e59c700 1 -- 172.21.15.154:0/2481284432 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100c3480 con 0x7f62100c6490 2024-03-14T09:01:23.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620d59a700 1 -- 172.21.15.154:0/2481284432 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516081621 0 0) 0x7f62100c3760 con 0x7f621006d440 2024-03-14T09:01:23.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620e59c700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62100c6490 0x7f62100c7880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62040098f0 tx=0x7f6204009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf8730482acec2b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620d59a700 1 -- 172.21.15.154:0/2481284432 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011830 con 0x7f62100c6490 2024-03-14T09:01:23.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620d59a700 1 -- 172.21.15.154:0/2481284432 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6204011990 con 0x7f62100c6490 2024-03-14T09:01:23.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f620d59a700 1 -- 172.21.15.154:0/2481284432 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204011d20 con 0x7f62100c6490 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f6214fdf700 1 -- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62100c6490 msgr2=0x7f62100c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.183+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62100c6490 0x7f62100c7880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62040098f0 tx=0x7f6204009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 -- 172.21.15.154:0/2481284432 shutdown_connections 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62100c6490 0x7f62100c7880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2481284432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f621006cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 -- 172.21.15.154:0/2481284432 >> 172.21.15.154:0/2481284432 conn(0x7f62100a9bf0 msgr2=0x7f62100a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 -- 172.21.15.154:0/2481284432 shutdown_connections 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.184+0000 7f6214fdf700 1 -- 172.21.15.154:0/2481284432 wait complete. 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.185+0000 7f6214fdf700 1 Processor -- start 2024-03-14T09:01:23.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.185+0000 7f6214fdf700 1 -- start start 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f62100a8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62101dad30 0x7f6210099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621009ac20 con 0x7f621006d440 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62101dd640 con 0x7f62101dad30 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f6214fdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62101eabf0 con 0x7f62100c7dc0 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f620e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f620e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48576/0 (socket says 172.21.15.154:48576) 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f620e59c700 1 -- 172.21.15.154:0/2158068553 learned_addr learned my addr 172.21.15.154:0/2158068553 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f620dd9b700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f62100a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.186+0000 7f620ed9d700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62101dad30 0x7f6210099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 -- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 msgr2=0x7f62100a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f62100a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 -- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62101dad30 msgr2=0x7f6210099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62101dad30 0x7f6210099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 -- 172.21.15.154:0/2158068553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204009550 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f620e59c700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62100740a0 tx=0x7f6204004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204004340 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620401f070 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62101eae90 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62040036a0 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.187+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62101ddb70 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.188+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62101de5d0 con 0x7f621006d440 2024-03-14T09:01:23.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.188+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6204021030 con 0x7f621006d440 2024-03-14T09:01:23.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.189+0000 7f61feffd700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 0x7f61f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.189+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6204089fd0 con 0x7f621006d440 2024-03-14T09:01:23.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.189+0000 7f620dd9b700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 0x7f61f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.189+0000 7f620dd9b700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 0x7f61f006ccf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62100acb30 tx=0x7f6200009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.193+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6204055480 con 0x7f621006d440 2024-03-14T09:01:23.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.384+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f621006c150 con 0x7f61f006a830 2024-03-14T09:01:23.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.392+0000 7f61feffd700 1 -- 172.21.15.154:0/2158068553 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246770 (secure 0 0 0) 0x7f6200011830 con 0x7f61f006a830 2024-03-14T09:01:23.395 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:23.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 msgr2=0x7f61f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 0x7f61f006ccf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62100acb30 tx=0x7f6200009210 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 msgr2=0x7f62101de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62100740a0 tx=0x7f6204004750 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 shutdown_connections 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62101dad30 0x7f6210099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f61f006a830 0x7f61f006ccf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f621006d440 0x7f62101de090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 --2- 172.21.15.154:0/2158068553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62100c7dc0 0x7f62100a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 >> 172.21.15.154:0/2158068553 conn(0x7f62100a9bf0 msgr2=0x7f62100a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 shutdown_connections 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.395+0000 7f6214fdf700 1 -- 172.21.15.154:0/2158068553 wait complete. 2024-03-14T09:01:23.398 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:01:23.407 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-14T09:01:21.517523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218556,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756164,"statfs":{"total":773094113280,"available":772870311936,"internally_reserved":0,"allocated":2433024,"data_stored":1546584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.816616"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-14T09:01:16.203887+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:16.203887+0000","last_peered":"2024-03-14T09:01:16.203887+0000","last_clean":"2024-03-14T09:01:16.203887+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:16.203887+0000","last_undegraded":"2024-03-14T09:01:16.203887+0000","last_fullsized":"2024-03-14T09:01:16.203887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:17.693398+0000","last_change":"2024-03-14T09:01:17.693398+0000","last_active":"2024-03-14T09:01:17.693398+0000","last_peered":"2024-03-14T09:01:17.693398+0000","last_clean":"2024-03-14T09:01:17.693398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:17.693398+0000","last_undegraded":"2024-03-14T09:01:17.693398+0000","last_fullsized":"2024-03-14T09:01:17.693398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:01:18.699844+0000","last_change":"2024-03-14T09:01:18.699844+0000","last_active":"2024-03-14T09:01:18.699844+0000","last_peered":"2024-03-14T09:01:18.699844+0000","last_clean":"2024-03-14T09:01:18.699844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:01:18.699844+0000","last_undegraded":"2024-03-14T09:01:18.699844+0000","last_fullsized":"2024-03-14T09:01:18.699844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":636099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:01:23.414 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:01:23.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.570+0000 7f6ee7e57700 1 Processor -- start 2024-03-14T09:01:23.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.570+0000 7f6ee7e57700 1 -- start start 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee7e57700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ee00c8350 0x7f6ee01547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee7e57700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee0154d30 0x7f6ee01551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee7e57700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee0157e80 con 0x7f6ee00c7f70 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee7e57700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee0158ac0 con 0x7f6ee00c8350 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee7e57700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee0158120 con 0x7f6ee0154d30 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ee00c8350 0x7f6ee01547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ee00c8350 0x7f6ee01547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48584/0 (socket says 172.21.15.154:48584) 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 -- 172.21.15.154:0/2725554286 learned_addr learned my addr 172.21.15.154:0/2725554286 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee63f4700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee0154d30 0x7f6ee01551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 -- 172.21.15.154:0/2725554286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee0154d30 msgr2=0x7f6ee01551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:23.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee0154d30 0x7f6ee01551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 -- 172.21.15.154:0/2725554286 >> v1:172.21.15.191:6789/0 conn(0x7f6ee00c7f70 legacy=0x7f6ee00c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 -- 172.21.15.154:0/2725554286 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0158400 con 0x7f6ee00c8350 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee53f2700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee00c8350 0x7f6ee01547f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00098f0 tx=0x7f6ed0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a433d5747d3906cc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee4bf1700 1 -- 172.21.15.154:0/2725554286 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0004230 con 0x7f6ee00c8350 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee4bf1700 1 -- 172.21.15.154:0/2725554286 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0004390 con 0x7f6ee00c8350 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.571+0000 7f6ee4bf1700 1 -- 172.21.15.154:0/2725554286 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00036a0 con 0x7f6ee00c8350 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 -- 172.21.15.154:0/2725554286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee00c8350 msgr2=0x7f6ee01547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee00c8350 0x7f6ee01547f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00098f0 tx=0x7f6ed0009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 -- 172.21.15.154:0/2725554286 shutdown_connections 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee0154d30 0x7f6ee01551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/2725554286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee00c8350 0x7f6ee01547f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 -- 172.21.15.154:0/2725554286 >> 172.21.15.154:0/2725554286 conn(0x7f6ee0072080 msgr2=0x7f6ee0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 -- 172.21.15.154:0/2725554286 shutdown_connections 2024-03-14T09:01:23.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.572+0000 7f6ee7e57700 1 -- 172.21.15.154:0/2725554286 wait complete. 2024-03-14T09:01:23.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.573+0000 7f6ee7e57700 1 Processor -- start 2024-03-14T09:01:23.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.573+0000 7f6ee7e57700 1 -- start start 2024-03-14T09:01:23.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee00c7f70 0x7f6ee01e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee00c8350 0x7f6ee01e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00af2d0 con 0x7f6ee0154d30 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01e7c20 con 0x7f6ee00c8350 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee7e57700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00ae1a0 con 0x7f6ee00c7f70 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48598/0 (socket says 172.21.15.154:48598) 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 -- 172.21.15.154:0/1979707404 learned_addr learned my addr 172.21.15.154:0/1979707404 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee5bf3700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee00c7f70 0x7f6ee01e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee53f2700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee00c8350 0x7f6ee01e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 -- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee00c7f70 msgr2=0x7f6ee01e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee00c7f70 0x7f6ee01e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 -- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee00c8350 msgr2=0x7f6ee01e76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee00c8350 0x7f6ee01e76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 -- 172.21.15.154:0/1979707404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed0009550 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ee63f4700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8007e90 tx=0x7f6ed800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.574+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed800ed30 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.575+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed800ee90 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.575+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee00ad140 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.575+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8014660 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.575+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee00af800 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.575+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee00697d0 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.576+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6ed80147c0 con 0x7f6ee0154d30 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.576+0000 7f6ed67fc700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 0x7f6ec406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.577+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ed80892d0 con 0x7f6ee0154d30 2024-03-14T09:01:23.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.577+0000 7f6ee5bf3700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 0x7f6ec406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.577+0000 7f6ee5bf3700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 0x7f6ec406ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6edc007fd0 tx=0x7f6edc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.581+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6ed8054830 con 0x7f6ee0154d30 2024-03-14T09:01:23.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.768+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ee00bd8b0 con 0x7f6ec406a830 2024-03-14T09:01:23.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.776+0000 7f6ed67fc700 1 -- 172.21.15.154:0/1979707404 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264702 (secure 0 0 0) 0x7f6edc003a90 con 0x7f6ec406a830 2024-03-14T09:01:23.779 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 msgr2=0x7f6ec406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 0x7f6ec406ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6edc007fd0 tx=0x7f6edc007ec0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 msgr2=0x7f6ee01ed760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8007e90 tx=0x7f6ed800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 shutdown_connections 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6ec406a830 0x7f6ec406ccf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ee0154d30 0x7f6ee01ed760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee00c7f70 0x7f6ee01e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 --2- 172.21.15.154:0/1979707404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee00c8350 0x7f6ee01e76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 >> 172.21.15.154:0/1979707404 conn(0x7f6ee0072080 msgr2=0x7f6ee00c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 shutdown_connections 2024-03-14T09:01:23.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.779+0000 7f6ee7e57700 1 -- 172.21.15.154:0/1979707404 wait complete. 2024-03-14T09:01:23.782 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:01:23.790 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-03-14T09:01:23.518037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.817130"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745804+0000","last_change":"2024-03-14T09:01:19.140304+0000","last_active":"2024-03-14T09:01:20.745804+0000","last_peered":"2024-03-14T09:01:20.745804+0000","last_clean":"2024-03-14T09:01:20.745804+0000","last_became_active":"2024-03-14T09:01:19.140051+0000","last_became_peered":"2024-03-14T09:01:19.140051+0000","last_unstale":"2024-03-14T09:01:20.745804+0000","last_undegraded":"2024-03-14T09:01:20.745804+0000","last_fullsized":"2024-03-14T09:01:20.745804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744549+0000","last_change":"2024-03-14T09:01:19.119652+0000","last_active":"2024-03-14T09:01:20.744549+0000","last_peered":"2024-03-14T09:01:20.744549+0000","last_clean":"2024-03-14T09:01:20.744549+0000","last_became_active":"2024-03-14T09:01:19.119408+0000","last_became_peered":"2024-03-14T09:01:19.119408+0000","last_unstale":"2024-03-14T09:01:20.744549+0000","last_undegraded":"2024-03-14T09:01:20.744549+0000","last_fullsized":"2024-03-14T09:01:20.744549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119387+0000","last_change":"2024-03-14T09:01:19.122167+0000","last_active":"2024-03-14T09:01:20.119387+0000","last_peered":"2024-03-14T09:01:20.119387+0000","last_clean":"2024-03-14T09:01:20.119387+0000","last_became_active":"2024-03-14T09:01:19.121201+0000","last_became_peered":"2024-03-14T09:01:19.121201+0000","last_unstale":"2024-03-14T09:01:20.119387+0000","last_undegraded":"2024-03-14T09:01:20.119387+0000","last_fullsized":"2024-03-14T09:01:20.119387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746011+0000","last_change":"2024-03-14T09:01:19.129747+0000","last_active":"2024-03-14T09:01:20.746011+0000","last_peered":"2024-03-14T09:01:20.746011+0000","last_clean":"2024-03-14T09:01:20.746011+0000","last_became_active":"2024-03-14T09:01:19.129254+0000","last_became_peered":"2024-03-14T09:01:19.129254+0000","last_unstale":"2024-03-14T09:01:20.746011+0000","last_undegraded":"2024-03-14T09:01:20.746011+0000","last_fullsized":"2024-03-14T09:01:20.746011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745535+0000","last_change":"2024-03-14T09:01:19.138324+0000","last_active":"2024-03-14T09:01:20.745535+0000","last_peered":"2024-03-14T09:01:20.745535+0000","last_clean":"2024-03-14T09:01:20.745535+0000","last_became_active":"2024-03-14T09:01:19.138018+0000","last_became_peered":"2024-03-14T09:01:19.138018+0000","last_unstale":"2024-03-14T09:01:20.745535+0000","last_undegraded":"2024-03-14T09:01:20.745535+0000","last_fullsized":"2024-03-14T09:01:20.745535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:51.127253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744951+0000","last_change":"2024-03-14T09:01:19.129902+0000","last_active":"2024-03-14T09:01:20.744951+0000","last_peered":"2024-03-14T09:01:20.744951+0000","last_clean":"2024-03-14T09:01:20.744951+0000","last_became_active":"2024-03-14T09:01:19.128311+0000","last_became_peered":"2024-03-14T09:01:19.128311+0000","last_unstale":"2024-03-14T09:01:20.744951+0000","last_undegraded":"2024-03-14T09:01:20.744951+0000","last_fullsized":"2024-03-14T09:01:20.744951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745141+0000","last_change":"2024-03-14T09:01:19.136657+0000","last_active":"2024-03-14T09:01:20.745141+0000","last_peered":"2024-03-14T09:01:20.745141+0000","last_clean":"2024-03-14T09:01:20.745141+0000","last_became_active":"2024-03-14T09:01:19.136253+0000","last_became_peered":"2024-03-14T09:01:19.136253+0000","last_unstale":"2024-03-14T09:01:20.745141+0000","last_undegraded":"2024-03-14T09:01:20.745141+0000","last_fullsized":"2024-03-14T09:01:20.745141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122779+0000","last_change":"2024-03-14T09:01:19.123809+0000","last_active":"2024-03-14T09:01:20.122779+0000","last_peered":"2024-03-14T09:01:20.122779+0000","last_clean":"2024-03-14T09:01:20.122779+0000","last_became_active":"2024-03-14T09:01:19.122471+0000","last_became_peered":"2024-03-14T09:01:19.122471+0000","last_unstale":"2024-03-14T09:01:20.122779+0000","last_undegraded":"2024-03-14T09:01:20.122779+0000","last_fullsized":"2024-03-14T09:01:20.122779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745747+0000","last_change":"2024-03-14T09:01:19.142452+0000","last_active":"2024-03-14T09:01:20.745747+0000","last_peered":"2024-03-14T09:01:20.745747+0000","last_clean":"2024-03-14T09:01:20.745747+0000","last_became_active":"2024-03-14T09:01:19.142164+0000","last_became_peered":"2024-03-14T09:01:19.142164+0000","last_unstale":"2024-03-14T09:01:20.745747+0000","last_undegraded":"2024-03-14T09:01:20.745747+0000","last_fullsized":"2024-03-14T09:01:20.745747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745182+0000","last_change":"2024-03-14T09:01:19.131516+0000","last_active":"2024-03-14T09:01:20.745182+0000","last_peered":"2024-03-14T09:01:20.745182+0000","last_clean":"2024-03-14T09:01:20.745182+0000","last_became_active":"2024-03-14T09:01:19.131190+0000","last_became_peered":"2024-03-14T09:01:19.131190+0000","last_unstale":"2024-03-14T09:01:20.745182+0000","last_undegraded":"2024-03-14T09:01:20.745182+0000","last_fullsized":"2024-03-14T09:01:20.745182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745021+0000","last_change":"2024-03-14T09:01:19.137722+0000","last_active":"2024-03-14T09:01:20.745021+0000","last_peered":"2024-03-14T09:01:20.745021+0000","last_clean":"2024-03-14T09:01:20.745021+0000","last_became_active":"2024-03-14T09:01:19.137447+0000","last_became_peered":"2024-03-14T09:01:19.137447+0000","last_unstale":"2024-03-14T09:01:20.745021+0000","last_undegraded":"2024-03-14T09:01:20.745021+0000","last_fullsized":"2024-03-14T09:01:20.745021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123173+0000","last_change":"2024-03-14T09:01:19.127732+0000","last_active":"2024-03-14T09:01:20.123173+0000","last_peered":"2024-03-14T09:01:20.123173+0000","last_clean":"2024-03-14T09:01:20.123173+0000","last_became_active":"2024-03-14T09:01:19.126961+0000","last_became_peered":"2024-03-14T09:01:19.126961+0000","last_unstale":"2024-03-14T09:01:20.123173+0000","last_undegraded":"2024-03-14T09:01:20.123173+0000","last_fullsized":"2024-03-14T09:01:20.123173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122936+0000","last_change":"2024-03-14T09:01:19.131805+0000","last_active":"2024-03-14T09:01:20.122936+0000","last_peered":"2024-03-14T09:01:20.122936+0000","last_clean":"2024-03-14T09:01:20.122936+0000","last_became_active":"2024-03-14T09:01:19.131545+0000","last_became_peered":"2024-03-14T09:01:19.131545+0000","last_unstale":"2024-03-14T09:01:20.122936+0000","last_undegraded":"2024-03-14T09:01:20.122936+0000","last_fullsized":"2024-03-14T09:01:20.122936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119342+0000","last_change":"2024-03-14T09:01:19.121888+0000","last_active":"2024-03-14T09:01:20.119342+0000","last_peered":"2024-03-14T09:01:20.119342+0000","last_clean":"2024-03-14T09:01:20.119342+0000","last_became_active":"2024-03-14T09:01:19.121604+0000","last_became_peered":"2024-03-14T09:01:19.121604+0000","last_unstale":"2024-03-14T09:01:20.119342+0000","last_undegraded":"2024-03-14T09:01:20.119342+0000","last_fullsized":"2024-03-14T09:01:20.119342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743969+0000","last_change":"2024-03-14T09:01:19.125119+0000","last_active":"2024-03-14T09:01:20.743969+0000","last_peered":"2024-03-14T09:01:20.743969+0000","last_clean":"2024-03-14T09:01:20.743969+0000","last_became_active":"2024-03-14T09:01:19.124829+0000","last_became_peered":"2024-03-14T09:01:19.124829+0000","last_unstale":"2024-03-14T09:01:20.743969+0000","last_undegraded":"2024-03-14T09:01:20.743969+0000","last_fullsized":"2024-03-14T09:01:20.743969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744669+0000","last_change":"2024-03-14T09:01:19.128411+0000","last_active":"2024-03-14T09:01:20.744669+0000","last_peered":"2024-03-14T09:01:20.744669+0000","last_clean":"2024-03-14T09:01:20.744669+0000","last_became_active":"2024-03-14T09:01:19.127935+0000","last_became_peered":"2024-03-14T09:01:19.127935+0000","last_unstale":"2024-03-14T09:01:20.744669+0000","last_undegraded":"2024-03-14T09:01:20.744669+0000","last_fullsized":"2024-03-14T09:01:20.744669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745529+0000","last_change":"2024-03-14T09:01:19.139316+0000","last_active":"2024-03-14T09:01:20.745529+0000","last_peered":"2024-03-14T09:01:20.745529+0000","last_clean":"2024-03-14T09:01:20.745529+0000","last_became_active":"2024-03-14T09:01:19.139045+0000","last_became_peered":"2024-03-14T09:01:19.139045+0000","last_unstale":"2024-03-14T09:01:20.745529+0000","last_undegraded":"2024-03-14T09:01:20.745529+0000","last_fullsized":"2024-03-14T09:01:20.745529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744233+0000","last_change":"2024-03-14T09:01:19.120202+0000","last_active":"2024-03-14T09:01:20.744233+0000","last_peered":"2024-03-14T09:01:20.744233+0000","last_clean":"2024-03-14T09:01:20.744233+0000","last_became_active":"2024-03-14T09:01:19.119935+0000","last_became_peered":"2024-03-14T09:01:19.119935+0000","last_unstale":"2024-03-14T09:01:20.744233+0000","last_undegraded":"2024-03-14T09:01:20.744233+0000","last_fullsized":"2024-03-14T09:01:20.744233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746144+0000","last_change":"2024-03-14T09:01:19.139436+0000","last_active":"2024-03-14T09:01:20.746144+0000","last_peered":"2024-03-14T09:01:20.746144+0000","last_clean":"2024-03-14T09:01:20.746144+0000","last_became_active":"2024-03-14T09:01:19.139165+0000","last_became_peered":"2024-03-14T09:01:19.139165+0000","last_unstale":"2024-03-14T09:01:20.746144+0000","last_undegraded":"2024-03-14T09:01:20.746144+0000","last_fullsized":"2024-03-14T09:01:20.746144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:44:30.654286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119464+0000","last_change":"2024-03-14T09:01:18.720548+0000","last_active":"2024-03-14T09:01:20.119464+0000","last_peered":"2024-03-14T09:01:20.119464+0000","last_clean":"2024-03-14T09:01:20.119464+0000","last_became_active":"2024-03-14T09:01:18.720258+0000","last_became_peered":"2024-03-14T09:01:18.720258+0000","last_unstale":"2024-03-14T09:01:20.119464+0000","last_undegraded":"2024-03-14T09:01:20.119464+0000","last_fullsized":"2024-03-14T09:01:20.119464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123199+0000","last_change":"2024-03-14T09:01:19.131625+0000","last_active":"2024-03-14T09:01:20.123199+0000","last_peered":"2024-03-14T09:01:20.123199+0000","last_clean":"2024-03-14T09:01:20.123199+0000","last_became_active":"2024-03-14T09:01:19.131308+0000","last_became_peered":"2024-03-14T09:01:19.131308+0000","last_unstale":"2024-03-14T09:01:20.123199+0000","last_undegraded":"2024-03-14T09:01:20.123199+0000","last_fullsized":"2024-03-14T09:01:20.123199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123216+0000","last_change":"2024-03-14T09:01:18.722396+0000","last_active":"2024-03-14T09:01:20.123216+0000","last_peered":"2024-03-14T09:01:20.123216+0000","last_clean":"2024-03-14T09:01:20.123216+0000","last_became_active":"2024-03-14T09:01:18.721852+0000","last_became_peered":"2024-03-14T09:01:18.721852+0000","last_unstale":"2024-03-14T09:01:20.123216+0000","last_undegraded":"2024-03-14T09:01:20.123216+0000","last_fullsized":"2024-03-14T09:01:20.123216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119416+0000","last_change":"2024-03-14T09:01:19.127918+0000","last_active":"2024-03-14T09:01:20.119416+0000","last_peered":"2024-03-14T09:01:20.119416+0000","last_clean":"2024-03-14T09:01:20.119416+0000","last_became_active":"2024-03-14T09:01:19.127663+0000","last_became_peered":"2024-03-14T09:01:19.127663+0000","last_unstale":"2024-03-14T09:01:20.119416+0000","last_undegraded":"2024-03-14T09:01:20.119416+0000","last_fullsized":"2024-03-14T09:01:20.119416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746324+0000","last_change":"2024-03-14T09:01:18.727557+0000","last_active":"2024-03-14T09:01:20.746324+0000","last_peered":"2024-03-14T09:01:20.746324+0000","last_clean":"2024-03-14T09:01:20.746324+0000","last_became_active":"2024-03-14T09:01:18.727245+0000","last_became_peered":"2024-03-14T09:01:18.727245+0000","last_unstale":"2024-03-14T09:01:20.746324+0000","last_undegraded":"2024-03-14T09:01:20.746324+0000","last_fullsized":"2024-03-14T09:01:20.746324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745761+0000","last_change":"2024-03-14T09:01:19.138918+0000","last_active":"2024-03-14T09:01:20.745761+0000","last_peered":"2024-03-14T09:01:20.745761+0000","last_clean":"2024-03-14T09:01:20.745761+0000","last_became_active":"2024-03-14T09:01:19.137549+0000","last_became_peered":"2024-03-14T09:01:19.137549+0000","last_unstale":"2024-03-14T09:01:20.745761+0000","last_undegraded":"2024-03-14T09:01:20.745761+0000","last_fullsized":"2024-03-14T09:01:20.745761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745273+0000","last_change":"2024-03-14T09:01:18.724542+0000","last_active":"2024-03-14T09:01:20.745273+0000","last_peered":"2024-03-14T09:01:20.745273+0000","last_clean":"2024-03-14T09:01:20.745273+0000","last_became_active":"2024-03-14T09:01:18.724257+0000","last_became_peered":"2024-03-14T09:01:18.724257+0000","last_unstale":"2024-03-14T09:01:20.745273+0000","last_undegraded":"2024-03-14T09:01:20.745273+0000","last_fullsized":"2024-03-14T09:01:20.745273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745073+0000","last_change":"2024-03-14T09:01:19.125917+0000","last_active":"2024-03-14T09:01:20.745073+0000","last_peered":"2024-03-14T09:01:20.745073+0000","last_clean":"2024-03-14T09:01:20.745073+0000","last_became_active":"2024-03-14T09:01:19.125654+0000","last_became_peered":"2024-03-14T09:01:19.125654+0000","last_unstale":"2024-03-14T09:01:20.745073+0000","last_undegraded":"2024-03-14T09:01:20.745073+0000","last_fullsized":"2024-03-14T09:01:20.745073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123032+0000","last_change":"2024-03-14T09:01:18.725364+0000","last_active":"2024-03-14T09:01:20.123032+0000","last_peered":"2024-03-14T09:01:20.123032+0000","last_clean":"2024-03-14T09:01:20.123032+0000","last_became_active":"2024-03-14T09:01:18.725141+0000","last_became_peered":"2024-03-14T09:01:18.725141+0000","last_unstale":"2024-03-14T09:01:20.123032+0000","last_undegraded":"2024-03-14T09:01:20.123032+0000","last_fullsized":"2024-03-14T09:01:20.123032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122963+0000","last_change":"2024-03-14T09:01:19.128226+0000","last_active":"2024-03-14T09:01:20.122963+0000","last_peered":"2024-03-14T09:01:20.122963+0000","last_clean":"2024-03-14T09:01:20.122963+0000","last_became_active":"2024-03-14T09:01:19.127601+0000","last_became_peered":"2024-03-14T09:01:19.127601+0000","last_unstale":"2024-03-14T09:01:20.122963+0000","last_undegraded":"2024-03-14T09:01:20.122963+0000","last_fullsized":"2024-03-14T09:01:20.122963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:05.796691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745089+0000","last_change":"2024-03-14T09:01:18.722204+0000","last_active":"2024-03-14T09:01:20.745089+0000","last_peered":"2024-03-14T09:01:20.745089+0000","last_clean":"2024-03-14T09:01:20.745089+0000","last_became_active":"2024-03-14T09:01:18.721901+0000","last_became_peered":"2024-03-14T09:01:18.721901+0000","last_unstale":"2024-03-14T09:01:20.745089+0000","last_undegraded":"2024-03-14T09:01:20.745089+0000","last_fullsized":"2024-03-14T09:01:20.745089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119757+0000","last_change":"2024-03-14T09:01:19.127938+0000","last_active":"2024-03-14T09:01:20.119757+0000","last_peered":"2024-03-14T09:01:20.119757+0000","last_clean":"2024-03-14T09:01:20.119757+0000","last_became_active":"2024-03-14T09:01:19.127718+0000","last_became_peered":"2024-03-14T09:01:19.127718+0000","last_unstale":"2024-03-14T09:01:20.119757+0000","last_undegraded":"2024-03-14T09:01:20.119757+0000","last_fullsized":"2024-03-14T09:01:20.119757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122920+0000","last_change":"2024-03-14T09:01:18.726440+0000","last_active":"2024-03-14T09:01:20.122920+0000","last_peered":"2024-03-14T09:01:20.122920+0000","last_clean":"2024-03-14T09:01:20.122920+0000","last_became_active":"2024-03-14T09:01:18.726143+0000","last_became_peered":"2024-03-14T09:01:18.726143+0000","last_unstale":"2024-03-14T09:01:20.122920+0000","last_undegraded":"2024-03-14T09:01:20.122920+0000","last_fullsized":"2024-03-14T09:01:20.122920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122876+0000","last_change":"2024-03-14T09:01:19.117249+0000","last_active":"2024-03-14T09:01:20.122876+0000","last_peered":"2024-03-14T09:01:20.122876+0000","last_clean":"2024-03-14T09:01:20.122876+0000","last_became_active":"2024-03-14T09:01:19.116999+0000","last_became_peered":"2024-03-14T09:01:19.116999+0000","last_unstale":"2024-03-14T09:01:20.122876+0000","last_undegraded":"2024-03-14T09:01:20.122876+0000","last_fullsized":"2024-03-14T09:01:20.122876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744666+0000","last_change":"2024-03-14T09:01:18.721268+0000","last_active":"2024-03-14T09:01:20.744666+0000","last_peered":"2024-03-14T09:01:20.744666+0000","last_clean":"2024-03-14T09:01:20.744666+0000","last_became_active":"2024-03-14T09:01:18.720984+0000","last_became_peered":"2024-03-14T09:01:18.720984+0000","last_unstale":"2024-03-14T09:01:20.744666+0000","last_undegraded":"2024-03-14T09:01:20.744666+0000","last_fullsized":"2024-03-14T09:01:20.744666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744322+0000","last_change":"2024-03-14T09:01:19.120543+0000","last_active":"2024-03-14T09:01:20.744322+0000","last_peered":"2024-03-14T09:01:20.744322+0000","last_clean":"2024-03-14T09:01:20.744322+0000","last_became_active":"2024-03-14T09:01:19.119215+0000","last_became_peered":"2024-03-14T09:01:19.119215+0000","last_unstale":"2024-03-14T09:01:20.744322+0000","last_undegraded":"2024-03-14T09:01:20.744322+0000","last_fullsized":"2024-03-14T09:01:20.744322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745938+0000","last_change":"2024-03-14T09:01:18.724696+0000","last_active":"2024-03-14T09:01:20.745938+0000","last_peered":"2024-03-14T09:01:20.745938+0000","last_clean":"2024-03-14T09:01:20.745938+0000","last_became_active":"2024-03-14T09:01:18.724393+0000","last_became_peered":"2024-03-14T09:01:18.724393+0000","last_unstale":"2024-03-14T09:01:20.745938+0000","last_undegraded":"2024-03-14T09:01:20.745938+0000","last_fullsized":"2024-03-14T09:01:20.745938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745050+0000","last_change":"2024-03-14T09:01:19.129590+0000","last_active":"2024-03-14T09:01:20.745050+0000","last_peered":"2024-03-14T09:01:20.745050+0000","last_clean":"2024-03-14T09:01:20.745050+0000","last_became_active":"2024-03-14T09:01:19.129324+0000","last_became_peered":"2024-03-14T09:01:19.129324+0000","last_unstale":"2024-03-14T09:01:20.745050+0000","last_undegraded":"2024-03-14T09:01:20.745050+0000","last_fullsized":"2024-03-14T09:01:20.745050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744921+0000","last_change":"2024-03-14T09:01:18.720316+0000","last_active":"2024-03-14T09:01:20.744921+0000","last_peered":"2024-03-14T09:01:20.744921+0000","last_clean":"2024-03-14T09:01:20.744921+0000","last_became_active":"2024-03-14T09:01:18.720030+0000","last_became_peered":"2024-03-14T09:01:18.720030+0000","last_unstale":"2024-03-14T09:01:20.744921+0000","last_undegraded":"2024-03-14T09:01:20.744921+0000","last_fullsized":"2024-03-14T09:01:20.744921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119562+0000","last_change":"2024-03-14T09:01:19.126442+0000","last_active":"2024-03-14T09:01:20.119562+0000","last_peered":"2024-03-14T09:01:20.119562+0000","last_clean":"2024-03-14T09:01:20.119562+0000","last_became_active":"2024-03-14T09:01:19.126192+0000","last_became_peered":"2024-03-14T09:01:19.126192+0000","last_unstale":"2024-03-14T09:01:20.119562+0000","last_undegraded":"2024-03-14T09:01:20.119562+0000","last_fullsized":"2024-03-14T09:01:20.119562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:57:19.311712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744536+0000","last_change":"2024-03-14T09:01:18.714712+0000","last_active":"2024-03-14T09:01:20.744536+0000","last_peered":"2024-03-14T09:01:20.744536+0000","last_clean":"2024-03-14T09:01:20.744536+0000","last_became_active":"2024-03-14T09:01:18.714304+0000","last_became_peered":"2024-03-14T09:01:18.714304+0000","last_unstale":"2024-03-14T09:01:20.744536+0000","last_undegraded":"2024-03-14T09:01:20.744536+0000","last_fullsized":"2024-03-14T09:01:20.744536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123108+0000","last_change":"2024-03-14T09:01:19.129609+0000","last_active":"2024-03-14T09:01:20.123108+0000","last_peered":"2024-03-14T09:01:20.123108+0000","last_clean":"2024-03-14T09:01:20.123108+0000","last_became_active":"2024-03-14T09:01:19.129345+0000","last_became_peered":"2024-03-14T09:01:19.129345+0000","last_unstale":"2024-03-14T09:01:20.123108+0000","last_undegraded":"2024-03-14T09:01:20.123108+0000","last_fullsized":"2024-03-14T09:01:20.123108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:57.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746528+0000","last_change":"2024-03-14T09:01:18.721291+0000","last_active":"2024-03-14T09:01:20.746528+0000","last_peered":"2024-03-14T09:01:20.746528+0000","last_clean":"2024-03-14T09:01:20.746528+0000","last_became_active":"2024-03-14T09:01:18.719243+0000","last_became_peered":"2024-03-14T09:01:18.719243+0000","last_unstale":"2024-03-14T09:01:20.746528+0000","last_undegraded":"2024-03-14T09:01:20.746528+0000","last_fullsized":"2024-03-14T09:01:20.746528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746209+0000","last_change":"2024-03-14T09:01:19.139742+0000","last_active":"2024-03-14T09:01:20.746209+0000","last_peered":"2024-03-14T09:01:20.746209+0000","last_clean":"2024-03-14T09:01:20.746209+0000","last_became_active":"2024-03-14T09:01:19.139476+0000","last_became_peered":"2024-03-14T09:01:19.139476+0000","last_unstale":"2024-03-14T09:01:20.746209+0000","last_undegraded":"2024-03-14T09:01:20.746209+0000","last_fullsized":"2024-03-14T09:01:20.746209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119687+0000","last_change":"2024-03-14T09:01:18.717256+0000","last_active":"2024-03-14T09:01:20.119687+0000","last_peered":"2024-03-14T09:01:20.119687+0000","last_clean":"2024-03-14T09:01:20.119687+0000","last_became_active":"2024-03-14T09:01:18.716989+0000","last_became_peered":"2024-03-14T09:01:18.716989+0000","last_unstale":"2024-03-14T09:01:20.119687+0000","last_undegraded":"2024-03-14T09:01:20.119687+0000","last_fullsized":"2024-03-14T09:01:20.119687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119309+0000","last_change":"2024-03-14T09:01:19.129023+0000","last_active":"2024-03-14T09:01:20.119309+0000","last_peered":"2024-03-14T09:01:20.119309+0000","last_clean":"2024-03-14T09:01:20.119309+0000","last_became_active":"2024-03-14T09:01:19.128683+0000","last_became_peered":"2024-03-14T09:01:19.128683+0000","last_unstale":"2024-03-14T09:01:20.119309+0000","last_undegraded":"2024-03-14T09:01:20.119309+0000","last_fullsized":"2024-03-14T09:01:20.119309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745348+0000","last_change":"2024-03-14T09:01:18.722021+0000","last_active":"2024-03-14T09:01:20.745348+0000","last_peered":"2024-03-14T09:01:20.745348+0000","last_clean":"2024-03-14T09:01:20.745348+0000","last_became_active":"2024-03-14T09:01:18.721794+0000","last_became_peered":"2024-03-14T09:01:18.721794+0000","last_unstale":"2024-03-14T09:01:20.745348+0000","last_undegraded":"2024-03-14T09:01:20.745348+0000","last_fullsized":"2024-03-14T09:01:20.745348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745561+0000","last_change":"2024-03-14T09:01:19.141735+0000","last_active":"2024-03-14T09:01:20.745561+0000","last_peered":"2024-03-14T09:01:20.745561+0000","last_clean":"2024-03-14T09:01:20.745561+0000","last_became_active":"2024-03-14T09:01:19.141460+0000","last_became_peered":"2024-03-14T09:01:19.141460+0000","last_unstale":"2024-03-14T09:01:20.745561+0000","last_undegraded":"2024-03-14T09:01:20.745561+0000","last_fullsized":"2024-03-14T09:01:20.745561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744488+0000","last_change":"2024-03-14T09:01:18.717974+0000","last_active":"2024-03-14T09:01:20.744488+0000","last_peered":"2024-03-14T09:01:20.744488+0000","last_clean":"2024-03-14T09:01:20.744488+0000","last_became_active":"2024-03-14T09:01:18.717677+0000","last_became_peered":"2024-03-14T09:01:18.717677+0000","last_unstale":"2024-03-14T09:01:20.744488+0000","last_undegraded":"2024-03-14T09:01:20.744488+0000","last_fullsized":"2024-03-14T09:01:20.744488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.127971+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.127683+0000","last_became_peered":"2024-03-14T09:01:19.127683+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743925+0000","last_change":"2024-03-14T09:01:18.727064+0000","last_active":"2024-03-14T09:01:20.743925+0000","last_peered":"2024-03-14T09:01:20.743925+0000","last_clean":"2024-03-14T09:01:20.743925+0000","last_became_active":"2024-03-14T09:01:18.726766+0000","last_became_peered":"2024-03-14T09:01:18.726766+0000","last_unstale":"2024-03-14T09:01:20.743925+0000","last_undegraded":"2024-03-14T09:01:20.743925+0000","last_fullsized":"2024-03-14T09:01:20.743925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745322+0000","last_change":"2024-03-14T09:01:19.136107+0000","last_active":"2024-03-14T09:01:20.745322+0000","last_peered":"2024-03-14T09:01:20.745322+0000","last_clean":"2024-03-14T09:01:20.745322+0000","last_became_active":"2024-03-14T09:01:19.135855+0000","last_became_peered":"2024-03-14T09:01:19.135855+0000","last_unstale":"2024-03-14T09:01:20.745322+0000","last_undegraded":"2024-03-14T09:01:20.745322+0000","last_fullsized":"2024-03-14T09:01:20.745322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123060+0000","last_change":"2024-03-14T09:01:18.717651+0000","last_active":"2024-03-14T09:01:20.123060+0000","last_peered":"2024-03-14T09:01:20.123060+0000","last_clean":"2024-03-14T09:01:20.123060+0000","last_became_active":"2024-03-14T09:01:18.717323+0000","last_became_peered":"2024-03-14T09:01:18.717323+0000","last_unstale":"2024-03-14T09:01:20.123060+0000","last_undegraded":"2024-03-14T09:01:20.123060+0000","last_fullsized":"2024-03-14T09:01:20.123060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:18:35.107355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745513+0000","last_change":"2024-03-14T09:01:19.141404+0000","last_active":"2024-03-14T09:01:20.745513+0000","last_peered":"2024-03-14T09:01:20.745513+0000","last_clean":"2024-03-14T09:01:20.745513+0000","last_became_active":"2024-03-14T09:01:19.141138+0000","last_became_peered":"2024-03-14T09:01:19.141138+0000","last_unstale":"2024-03-14T09:01:20.745513+0000","last_undegraded":"2024-03-14T09:01:20.745513+0000","last_fullsized":"2024-03-14T09:01:20.745513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744508+0000","last_change":"2024-03-14T09:01:18.714145+0000","last_active":"2024-03-14T09:01:20.744508+0000","last_peered":"2024-03-14T09:01:20.744508+0000","last_clean":"2024-03-14T09:01:20.744508+0000","last_became_active":"2024-03-14T09:01:18.713880+0000","last_became_peered":"2024-03-14T09:01:18.713880+0000","last_unstale":"2024-03-14T09:01:20.744508+0000","last_undegraded":"2024-03-14T09:01:20.744508+0000","last_fullsized":"2024-03-14T09:01:20.744508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745392+0000","last_change":"2024-03-14T09:01:19.120923+0000","last_active":"2024-03-14T09:01:20.745392+0000","last_peered":"2024-03-14T09:01:20.745392+0000","last_clean":"2024-03-14T09:01:20.745392+0000","last_became_active":"2024-03-14T09:01:19.120028+0000","last_became_peered":"2024-03-14T09:01:19.120028+0000","last_unstale":"2024-03-14T09:01:20.745392+0000","last_undegraded":"2024-03-14T09:01:20.745392+0000","last_fullsized":"2024-03-14T09:01:20.745392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745276+0000","last_change":"2024-03-14T09:01:18.715127+0000","last_active":"2024-03-14T09:01:20.745276+0000","last_peered":"2024-03-14T09:01:20.745276+0000","last_clean":"2024-03-14T09:01:20.745276+0000","last_became_active":"2024-03-14T09:01:18.714778+0000","last_became_peered":"2024-03-14T09:01:18.714778+0000","last_unstale":"2024-03-14T09:01:20.745276+0000","last_undegraded":"2024-03-14T09:01:20.745276+0000","last_fullsized":"2024-03-14T09:01:20.745276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122941+0000","last_change":"2024-03-14T09:01:19.130468+0000","last_active":"2024-03-14T09:01:20.122941+0000","last_peered":"2024-03-14T09:01:20.122941+0000","last_clean":"2024-03-14T09:01:20.122941+0000","last_became_active":"2024-03-14T09:01:19.130155+0000","last_became_peered":"2024-03-14T09:01:19.130155+0000","last_unstale":"2024-03-14T09:01:20.122941+0000","last_undegraded":"2024-03-14T09:01:20.122941+0000","last_fullsized":"2024-03-14T09:01:20.122941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:37.408944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744060+0000","last_change":"2024-03-14T09:01:19.118942+0000","last_active":"2024-03-14T09:01:20.744060+0000","last_peered":"2024-03-14T09:01:20.744060+0000","last_clean":"2024-03-14T09:01:20.744060+0000","last_became_active":"2024-03-14T09:01:19.118681+0000","last_became_peered":"2024-03-14T09:01:19.118681+0000","last_unstale":"2024-03-14T09:01:20.744060+0000","last_undegraded":"2024-03-14T09:01:20.744060+0000","last_fullsized":"2024-03-14T09:01:20.744060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743928+0000","last_change":"2024-03-14T09:01:18.722916+0000","last_active":"2024-03-14T09:01:20.743928+0000","last_peered":"2024-03-14T09:01:20.743928+0000","last_clean":"2024-03-14T09:01:20.743928+0000","last_became_active":"2024-03-14T09:01:18.722610+0000","last_became_peered":"2024-03-14T09:01:18.722610+0000","last_unstale":"2024-03-14T09:01:20.743928+0000","last_undegraded":"2024-03-14T09:01:20.743928+0000","last_fullsized":"2024-03-14T09:01:20.743928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123315+0000","last_change":"2024-03-14T09:01:19.124237+0000","last_active":"2024-03-14T09:01:20.123315+0000","last_peered":"2024-03-14T09:01:20.123315+0000","last_clean":"2024-03-14T09:01:20.123315+0000","last_became_active":"2024-03-14T09:01:19.123958+0000","last_became_peered":"2024-03-14T09:01:19.123958+0000","last_unstale":"2024-03-14T09:01:20.123315+0000","last_undegraded":"2024-03-14T09:01:20.123315+0000","last_fullsized":"2024-03-14T09:01:20.123315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744772+0000","last_change":"2024-03-14T09:01:18.716867+0000","last_active":"2024-03-14T09:01:20.744772+0000","last_peered":"2024-03-14T09:01:20.744772+0000","last_clean":"2024-03-14T09:01:20.744772+0000","last_became_active":"2024-03-14T09:01:18.716352+0000","last_became_peered":"2024-03-14T09:01:18.716352+0000","last_unstale":"2024-03-14T09:01:20.744772+0000","last_undegraded":"2024-03-14T09:01:20.744772+0000","last_fullsized":"2024-03-14T09:01:20.744772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-14T09:01:19.129296+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:19.129296+0000","last_peered":"2024-03-14T09:01:19.129296+0000","last_clean":"2024-03-14T09:01:19.129296+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:19.129296+0000","last_undegraded":"2024-03-14T09:01:19.129296+0000","last_fullsized":"2024-03-14T09:01:19.129296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119237+0000","last_change":"2024-03-14T09:01:19.125813+0000","last_active":"2024-03-14T09:01:20.119237+0000","last_peered":"2024-03-14T09:01:20.119237+0000","last_clean":"2024-03-14T09:01:20.119237+0000","last_became_active":"2024-03-14T09:01:19.125536+0000","last_became_peered":"2024-03-14T09:01:19.125536+0000","last_unstale":"2024-03-14T09:01:20.119237+0000","last_undegraded":"2024-03-14T09:01:20.119237+0000","last_fullsized":"2024-03-14T09:01:20.119237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745288+0000","last_change":"2024-03-14T09:01:18.728259+0000","last_active":"2024-03-14T09:01:20.745288+0000","last_peered":"2024-03-14T09:01:20.745288+0000","last_clean":"2024-03-14T09:01:20.745288+0000","last_became_active":"2024-03-14T09:01:18.728000+0000","last_became_peered":"2024-03-14T09:01:18.728000+0000","last_unstale":"2024-03-14T09:01:20.745288+0000","last_undegraded":"2024-03-14T09:01:20.745288+0000","last_fullsized":"2024-03-14T09:01:20.745288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746270+0000","last_change":"2024-03-14T09:01:18.722659+0000","last_active":"2024-03-14T09:01:20.746270+0000","last_peered":"2024-03-14T09:01:20.746270+0000","last_clean":"2024-03-14T09:01:20.746270+0000","last_became_active":"2024-03-14T09:01:18.722309+0000","last_became_peered":"2024-03-14T09:01:18.722309+0000","last_unstale":"2024-03-14T09:01:20.746270+0000","last_undegraded":"2024-03-14T09:01:20.746270+0000","last_fullsized":"2024-03-14T09:01:20.746270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744400+0000","last_change":"2024-03-14T09:01:19.119183+0000","last_active":"2024-03-14T09:01:20.744400+0000","last_peered":"2024-03-14T09:01:20.744400+0000","last_clean":"2024-03-14T09:01:20.744400+0000","last_became_active":"2024-03-14T09:01:19.118862+0000","last_became_peered":"2024-03-14T09:01:19.118862+0000","last_unstale":"2024-03-14T09:01:20.744400+0000","last_undegraded":"2024-03-14T09:01:20.744400+0000","last_fullsized":"2024-03-14T09:01:20.744400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744455+0000","last_change":"2024-03-14T09:01:18.719534+0000","last_active":"2024-03-14T09:01:20.744455+0000","last_peered":"2024-03-14T09:01:20.744455+0000","last_clean":"2024-03-14T09:01:20.744455+0000","last_became_active":"2024-03-14T09:01:18.719220+0000","last_became_peered":"2024-03-14T09:01:18.719220+0000","last_unstale":"2024-03-14T09:01:20.744455+0000","last_undegraded":"2024-03-14T09:01:20.744455+0000","last_fullsized":"2024-03-14T09:01:20.744455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123213+0000","last_change":"2024-03-14T09:01:19.135946+0000","last_active":"2024-03-14T09:01:20.123213+0000","last_peered":"2024-03-14T09:01:20.123213+0000","last_clean":"2024-03-14T09:01:20.123213+0000","last_became_active":"2024-03-14T09:01:19.135763+0000","last_became_peered":"2024-03-14T09:01:19.135763+0000","last_unstale":"2024-03-14T09:01:20.123213+0000","last_undegraded":"2024-03-14T09:01:20.123213+0000","last_fullsized":"2024-03-14T09:01:20.123213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745836+0000","last_change":"2024-03-14T09:01:18.723947+0000","last_active":"2024-03-14T09:01:20.745836+0000","last_peered":"2024-03-14T09:01:20.745836+0000","last_clean":"2024-03-14T09:01:20.745836+0000","last_became_active":"2024-03-14T09:01:18.723622+0000","last_became_peered":"2024-03-14T09:01:18.723622+0000","last_unstale":"2024-03-14T09:01:20.745836+0000","last_undegraded":"2024-03-14T09:01:20.745836+0000","last_fullsized":"2024-03-14T09:01:20.745836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119439+0000","last_change":"2024-03-14T09:01:19.121831+0000","last_active":"2024-03-14T09:01:20.119439+0000","last_peered":"2024-03-14T09:01:20.119439+0000","last_clean":"2024-03-14T09:01:20.119439+0000","last_became_active":"2024-03-14T09:01:19.121672+0000","last_became_peered":"2024-03-14T09:01:19.121672+0000","last_unstale":"2024-03-14T09:01:20.119439+0000","last_undegraded":"2024-03-14T09:01:20.119439+0000","last_fullsized":"2024-03-14T09:01:20.119439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744636+0000","last_change":"2024-03-14T09:01:18.725255+0000","last_active":"2024-03-14T09:01:20.744636+0000","last_peered":"2024-03-14T09:01:20.744636+0000","last_clean":"2024-03-14T09:01:20.744636+0000","last_became_active":"2024-03-14T09:01:18.724977+0000","last_became_peered":"2024-03-14T09:01:18.724977+0000","last_unstale":"2024-03-14T09:01:20.744636+0000","last_undegraded":"2024-03-14T09:01:20.744636+0000","last_fullsized":"2024-03-14T09:01:20.744636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:54.443401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122789+0000","last_change":"2024-03-14T09:01:19.127280+0000","last_active":"2024-03-14T09:01:20.122789+0000","last_peered":"2024-03-14T09:01:20.122789+0000","last_clean":"2024-03-14T09:01:20.122789+0000","last_became_active":"2024-03-14T09:01:19.126973+0000","last_became_peered":"2024-03-14T09:01:19.126973+0000","last_unstale":"2024-03-14T09:01:20.122789+0000","last_undegraded":"2024-03-14T09:01:20.122789+0000","last_fullsized":"2024-03-14T09:01:20.122789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744026+0000","last_change":"2024-03-14T09:01:18.719084+0000","last_active":"2024-03-14T09:01:20.744026+0000","last_peered":"2024-03-14T09:01:20.744026+0000","last_clean":"2024-03-14T09:01:20.744026+0000","last_became_active":"2024-03-14T09:01:18.718785+0000","last_became_peered":"2024-03-14T09:01:18.718785+0000","last_unstale":"2024-03-14T09:01:20.744026+0000","last_undegraded":"2024-03-14T09:01:20.744026+0000","last_fullsized":"2024-03-14T09:01:20.744026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122827+0000","last_change":"2024-03-14T09:01:19.123558+0000","last_active":"2024-03-14T09:01:20.122827+0000","last_peered":"2024-03-14T09:01:20.122827+0000","last_clean":"2024-03-14T09:01:20.122827+0000","last_became_active":"2024-03-14T09:01:19.122663+0000","last_became_peered":"2024-03-14T09:01:19.122663+0000","last_unstale":"2024-03-14T09:01:20.122827+0000","last_undegraded":"2024-03-14T09:01:20.122827+0000","last_fullsized":"2024-03-14T09:01:20.122827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744437+0000","last_change":"2024-03-14T09:01:18.717632+0000","last_active":"2024-03-14T09:01:20.744437+0000","last_peered":"2024-03-14T09:01:20.744437+0000","last_clean":"2024-03-14T09:01:20.744437+0000","last_became_active":"2024-03-14T09:01:18.717322+0000","last_became_peered":"2024-03-14T09:01:18.717322+0000","last_unstale":"2024-03-14T09:01:20.744437+0000","last_undegraded":"2024-03-14T09:01:20.744437+0000","last_fullsized":"2024-03-14T09:01:20.744437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123124+0000","last_change":"2024-03-14T09:01:19.124078+0000","last_active":"2024-03-14T09:01:20.123124+0000","last_peered":"2024-03-14T09:01:20.123124+0000","last_clean":"2024-03-14T09:01:20.123124+0000","last_became_active":"2024-03-14T09:01:19.123793+0000","last_became_peered":"2024-03-14T09:01:19.123793+0000","last_unstale":"2024-03-14T09:01:20.123124+0000","last_undegraded":"2024-03-14T09:01:20.123124+0000","last_fullsized":"2024-03-14T09:01:20.123124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745643+0000","last_change":"2024-03-14T09:01:18.725304+0000","last_active":"2024-03-14T09:01:20.745643+0000","last_peered":"2024-03-14T09:01:20.745643+0000","last_clean":"2024-03-14T09:01:20.745643+0000","last_became_active":"2024-03-14T09:01:18.725064+0000","last_became_peered":"2024-03-14T09:01:18.725064+0000","last_unstale":"2024-03-14T09:01:20.745643+0000","last_undegraded":"2024-03-14T09:01:20.745643+0000","last_fullsized":"2024-03-14T09:01:20.745643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:53:28.235917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.129552+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.129287+0000","last_became_peered":"2024-03-14T09:01:19.129287+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744170+0000","last_change":"2024-03-14T09:01:18.721721+0000","last_active":"2024-03-14T09:01:20.744170+0000","last_peered":"2024-03-14T09:01:20.744170+0000","last_clean":"2024-03-14T09:01:20.744170+0000","last_became_active":"2024-03-14T09:01:18.721450+0000","last_became_peered":"2024-03-14T09:01:18.721450+0000","last_unstale":"2024-03-14T09:01:20.744170+0000","last_undegraded":"2024-03-14T09:01:20.744170+0000","last_fullsized":"2024-03-14T09:01:20.744170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746277+0000","last_change":"2024-03-14T09:01:19.142624+0000","last_active":"2024-03-14T09:01:20.746277+0000","last_peered":"2024-03-14T09:01:20.746277+0000","last_clean":"2024-03-14T09:01:20.746277+0000","last_became_active":"2024-03-14T09:01:19.142294+0000","last_became_peered":"2024-03-14T09:01:19.142294+0000","last_unstale":"2024-03-14T09:01:20.746277+0000","last_undegraded":"2024-03-14T09:01:20.746277+0000","last_fullsized":"2024-03-14T09:01:20.746277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744505+0000","last_change":"2024-03-14T09:01:18.714548+0000","last_active":"2024-03-14T09:01:20.744505+0000","last_peered":"2024-03-14T09:01:20.744505+0000","last_clean":"2024-03-14T09:01:20.744505+0000","last_became_active":"2024-03-14T09:01:18.713927+0000","last_became_peered":"2024-03-14T09:01:18.713927+0000","last_unstale":"2024-03-14T09:01:20.744505+0000","last_undegraded":"2024-03-14T09:01:20.744505+0000","last_fullsized":"2024-03-14T09:01:20.744505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745857+0000","last_change":"2024-03-14T09:01:19.139093+0000","last_active":"2024-03-14T09:01:20.745857+0000","last_peered":"2024-03-14T09:01:20.745857+0000","last_clean":"2024-03-14T09:01:20.745857+0000","last_became_active":"2024-03-14T09:01:19.138560+0000","last_became_peered":"2024-03-14T09:01:19.138560+0000","last_unstale":"2024-03-14T09:01:20.745857+0000","last_undegraded":"2024-03-14T09:01:20.745857+0000","last_fullsized":"2024-03-14T09:01:20.745857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744326+0000","last_change":"2024-03-14T09:01:18.716783+0000","last_active":"2024-03-14T09:01:20.744326+0000","last_peered":"2024-03-14T09:01:20.744326+0000","last_clean":"2024-03-14T09:01:20.744326+0000","last_became_active":"2024-03-14T09:01:18.716538+0000","last_became_peered":"2024-03-14T09:01:18.716538+0000","last_unstale":"2024-03-14T09:01:20.744326+0000","last_undegraded":"2024-03-14T09:01:20.744326+0000","last_fullsized":"2024-03-14T09:01:20.744326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745724+0000","last_change":"2024-03-14T09:01:19.141337+0000","last_active":"2024-03-14T09:01:20.745724+0000","last_peered":"2024-03-14T09:01:20.745724+0000","last_clean":"2024-03-14T09:01:20.745724+0000","last_became_active":"2024-03-14T09:01:19.141093+0000","last_became_peered":"2024-03-14T09:01:19.141093+0000","last_unstale":"2024-03-14T09:01:20.745724+0000","last_undegraded":"2024-03-14T09:01:20.745724+0000","last_fullsized":"2024-03-14T09:01:20.745724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744540+0000","last_change":"2024-03-14T09:01:19.128035+0000","last_active":"2024-03-14T09:01:20.744540+0000","last_peered":"2024-03-14T09:01:20.744540+0000","last_clean":"2024-03-14T09:01:20.744540+0000","last_became_active":"2024-03-14T09:01:19.127787+0000","last_became_peered":"2024-03-14T09:01:19.127787+0000","last_unstale":"2024-03-14T09:01:20.744540+0000","last_undegraded":"2024-03-14T09:01:20.744540+0000","last_fullsized":"2024-03-14T09:01:20.744540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743991+0000","last_change":"2024-03-14T09:01:19.121494+0000","last_active":"2024-03-14T09:01:20.743991+0000","last_peered":"2024-03-14T09:01:20.743991+0000","last_clean":"2024-03-14T09:01:20.743991+0000","last_became_active":"2024-03-14T09:01:19.120995+0000","last_became_peered":"2024-03-14T09:01:19.120995+0000","last_unstale":"2024-03-14T09:01:20.743991+0000","last_undegraded":"2024-03-14T09:01:20.743991+0000","last_fullsized":"2024-03-14T09:01:20.743991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745172+0000","last_change":"2024-03-14T09:01:19.136830+0000","last_active":"2024-03-14T09:01:20.745172+0000","last_peered":"2024-03-14T09:01:20.745172+0000","last_clean":"2024-03-14T09:01:20.745172+0000","last_became_active":"2024-03-14T09:01:19.136568+0000","last_became_peered":"2024-03-14T09:01:19.136568+0000","last_unstale":"2024-03-14T09:01:20.745172+0000","last_undegraded":"2024-03-14T09:01:20.745172+0000","last_fullsized":"2024-03-14T09:01:20.745172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119641+0000","last_change":"2024-03-14T09:01:19.118186+0000","last_active":"2024-03-14T09:01:20.119641+0000","last_peered":"2024-03-14T09:01:20.119641+0000","last_clean":"2024-03-14T09:01:20.119641+0000","last_became_active":"2024-03-14T09:01:19.117955+0000","last_became_peered":"2024-03-14T09:01:19.117955+0000","last_unstale":"2024-03-14T09:01:20.119641+0000","last_undegraded":"2024-03-14T09:01:20.119641+0000","last_fullsized":"2024-03-14T09:01:20.119641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123167+0000","last_change":"2024-03-14T09:01:19.126685+0000","last_active":"2024-03-14T09:01:20.123167+0000","last_peered":"2024-03-14T09:01:20.123167+0000","last_clean":"2024-03-14T09:01:20.123167+0000","last_became_active":"2024-03-14T09:01:19.126196+0000","last_became_peered":"2024-03-14T09:01:19.126196+0000","last_unstale":"2024-03-14T09:01:20.123167+0000","last_undegraded":"2024-03-14T09:01:20.123167+0000","last_fullsized":"2024-03-14T09:01:20.123167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744458+0000","last_change":"2024-03-14T09:01:19.119001+0000","last_active":"2024-03-14T09:01:20.744458+0000","last_peered":"2024-03-14T09:01:20.744458+0000","last_clean":"2024-03-14T09:01:20.744458+0000","last_became_active":"2024-03-14T09:01:19.118755+0000","last_became_peered":"2024-03-14T09:01:19.118755+0000","last_unstale":"2024-03-14T09:01:20.744458+0000","last_undegraded":"2024-03-14T09:01:20.744458+0000","last_fullsized":"2024-03-14T09:01:20.744458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745084+0000","last_change":"2024-03-14T09:01:19.136652+0000","last_active":"2024-03-14T09:01:20.745084+0000","last_peered":"2024-03-14T09:01:20.745084+0000","last_clean":"2024-03-14T09:01:20.745084+0000","last_became_active":"2024-03-14T09:01:19.136252+0000","last_became_peered":"2024-03-14T09:01:19.136252+0000","last_unstale":"2024-03-14T09:01:20.745084+0000","last_undegraded":"2024-03-14T09:01:20.745084+0000","last_fullsized":"2024-03-14T09:01:20.745084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123129+0000","last_change":"2024-03-14T09:01:19.130297+0000","last_active":"2024-03-14T09:01:20.123129+0000","last_peered":"2024-03-14T09:01:20.123129+0000","last_clean":"2024-03-14T09:01:20.123129+0000","last_became_active":"2024-03-14T09:01:19.130024+0000","last_became_peered":"2024-03-14T09:01:19.130024+0000","last_unstale":"2024-03-14T09:01:20.123129+0000","last_undegraded":"2024-03-14T09:01:20.123129+0000","last_fullsized":"2024-03-14T09:01:20.123129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744787+0000","last_change":"2024-03-14T09:01:19.137747+0000","last_active":"2024-03-14T09:01:20.744787+0000","last_peered":"2024-03-14T09:01:20.744787+0000","last_clean":"2024-03-14T09:01:20.744787+0000","last_became_active":"2024-03-14T09:01:19.137471+0000","last_became_peered":"2024-03-14T09:01:19.137471+0000","last_unstale":"2024-03-14T09:01:20.744787+0000","last_undegraded":"2024-03-14T09:01:20.744787+0000","last_fullsized":"2024-03-14T09:01:20.744787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122916+0000","last_change":"2024-03-14T09:01:19.128038+0000","last_active":"2024-03-14T09:01:20.122916+0000","last_peered":"2024-03-14T09:01:20.122916+0000","last_clean":"2024-03-14T09:01:20.122916+0000","last_became_active":"2024-03-14T09:01:19.127847+0000","last_became_peered":"2024-03-14T09:01:19.127847+0000","last_unstale":"2024-03-14T09:01:20.122916+0000","last_undegraded":"2024-03-14T09:01:20.122916+0000","last_fullsized":"2024-03-14T09:01:20.122916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745074+0000","last_change":"2024-03-14T09:01:19.128566+0000","last_active":"2024-03-14T09:01:20.745074+0000","last_peered":"2024-03-14T09:01:20.745074+0000","last_clean":"2024-03-14T09:01:20.745074+0000","last_became_active":"2024-03-14T09:01:19.127893+0000","last_became_peered":"2024-03-14T09:01:19.127893+0000","last_unstale":"2024-03-14T09:01:20.745074+0000","last_undegraded":"2024-03-14T09:01:20.745074+0000","last_fullsized":"2024-03-14T09:01:20.745074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119666+0000","last_change":"2024-03-14T09:01:19.134739+0000","last_active":"2024-03-14T09:01:20.119666+0000","last_peered":"2024-03-14T09:01:20.119666+0000","last_clean":"2024-03-14T09:01:20.119666+0000","last_became_active":"2024-03-14T09:01:19.134442+0000","last_became_peered":"2024-03-14T09:01:19.134442+0000","last_unstale":"2024-03-14T09:01:20.119666+0000","last_undegraded":"2024-03-14T09:01:20.119666+0000","last_fullsized":"2024-03-14T09:01:20.119666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746583+0000","last_change":"2024-03-14T09:01:19.132199+0000","last_active":"2024-03-14T09:01:20.746583+0000","last_peered":"2024-03-14T09:01:20.746583+0000","last_clean":"2024-03-14T09:01:20.746583+0000","last_became_active":"2024-03-14T09:01:19.131887+0000","last_became_peered":"2024-03-14T09:01:19.131887+0000","last_unstale":"2024-03-14T09:01:20.746583+0000","last_undegraded":"2024-03-14T09:01:20.746583+0000","last_fullsized":"2024-03-14T09:01:20.746583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:01:23.794 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:01:23.795 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-14T09:01:23.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-14T09:01:23.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-14T09:01:23.795 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-14T09:01:23.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 Processor -- start 2024-03-14T09:01:23.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 -- start start 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 0x7f9518156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f951815d270 0x7f951815f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95180ac8e0 con 0x7f951814eaf0 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951806b310 con 0x7f951815d270 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.952+0000 7f951e1cc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951806de30 con 0x7f9518155df0 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f951815d270 0x7f951815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f951815d270 0x7f951815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48604/0 (socket says 172.21.15.154:48604) 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 -- 172.21.15.154:0/3904922693 learned_addr learned my addr 172.21.15.154:0/3904922693 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9516ffd700 1 --2- 172.21.15.154:0/3904922693 >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 0x7f9518156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 -- 172.21.15.154:0/3904922693 >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 msgr2=0x7f9518156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 --2- 172.21.15.154:0/3904922693 >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 0x7f9518156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 -- 172.21.15.154:0/3904922693 >> v1:172.21.15.191:6789/0 conn(0x7f951814eaf0 legacy=0x7f951806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:23.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 -- 172.21.15.154:0/3904922693 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180acb30 con 0x7f951815d270 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9516ffd700 1 --2- 172.21.15.154:0/3904922693 >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 0x7f9518156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f9517fff700 1 --2- 172.21.15.154:0/3904922693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f951815d270 0x7f951815f660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f951814f1c0 tx=0x7f951000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee53529d8789cbb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f95167fc700 1 -- 172.21.15.154:0/3904922693 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951000e070 con 0x7f951815d270 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f95167fc700 1 -- 172.21.15.154:0/3904922693 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510003a20 con 0x7f951815d270 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.953+0000 7f95167fc700 1 -- 172.21.15.154:0/3904922693 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510003dd0 con 0x7f951815d270 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 -- 172.21.15.154:0/3904922693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f951815d270 msgr2=0x7f951815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3904922693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f951815d270 0x7f951815f660 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f951814f1c0 tx=0x7f951000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 -- 172.21.15.154:0/3904922693 shutdown_connections 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3904922693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f951815d270 0x7f951815f660 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3904922693 >> v2:172.21.15.191:3300/0 conn(0x7f9518155df0 0x7f9518156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 -- 172.21.15.154:0/3904922693 >> 172.21.15.154:0/3904922693 conn(0x7f95180c3880 msgr2=0x7f95180c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 -- 172.21.15.154:0/3904922693 shutdown_connections 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.954+0000 7f951e1cc700 1 -- 172.21.15.154:0/3904922693 wait complete. 2024-03-14T09:01:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.955+0000 7f951e1cc700 1 Processor -- start 2024-03-14T09:01:23.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.955+0000 7f951e1cc700 1 -- start start 2024-03-14T09:01:23.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f951814eaf0 0x7f951814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f951815d270 0x7f95180c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518150e20 con 0x7f9518155df0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9518152260 con 0x7f951815d270 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f951e1cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9518151f10 con 0x7f951814eaf0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48618/0 (socket says 172.21.15.154:48618) 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 -- 172.21.15.154:0/3368415157 learned_addr learned my addr 172.21.15.154:0/3368415157 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9517fff700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f951815d270 0x7f95180c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f95177fe700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f951814eaf0 0x7f951814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 -- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f951814eaf0 msgr2=0x7f951814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f951814eaf0 0x7f951814f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 -- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f951815d270 msgr2=0x7f95180c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f951815d270 0x7f95180c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 -- 172.21.15.154:0/3368415157 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510009b30 con 0x7f9518155df0 2024-03-14T09:01:23.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f9516ffd700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c750 tx=0x7f950c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.956+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c011800 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.957+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c011960 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.957+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c011cd0 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.957+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951814f540 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.957+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95181eb3f0 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.957+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95181523f0 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.958+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f950c021350 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.958+0000 7f94fffff700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 0x7f950406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.958+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f950c088d00 con 0x7f9518155df0 2024-03-14T09:01:23.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.958+0000 7f95177fe700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 0x7f950406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:23.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.959+0000 7f95177fe700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 0x7f950406ccf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9508003e60 tx=0x7f9508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:23.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:23.963+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f950c054120 con 0x7f9518155df0 2024-03-14T09:01:24.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.214+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f95180697d0 con 0x7f9518155df0 2024-03-14T09:01:24.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.217+0000 7f94fffff700 1 -- 172.21.15.154:0/3368415157 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170397 (secure 0 0 0) 0x7f950c01e090 con 0x7f9518155df0 2024-03-14T09:01:24.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 msgr2=0x7f950406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 0x7f950406ccf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9508003e60 tx=0x7f9508009040 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 msgr2=0x7f95181502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f950c00c750 tx=0x7f950c00cb10 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 shutdown_connections 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f951815d270 0x7f95180c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f951814eaf0 0x7f951814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f950406a830 0x7f950406ccf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 --2- 172.21.15.154:0/3368415157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9518155df0 0x7f95181502c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 >> 172.21.15.154:0/3368415157 conn(0x7f95180c3880 msgr2=0x7f95181571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:24.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 shutdown_connections 2024-03-14T09:01:24.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.220+0000 7f951e1cc700 1 -- 172.21.15.154:0/3368415157 wait complete. 2024-03-14T09:01:24.233 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":46876},{"type":"v1","addr":"172.21.15.191:6833","nonce":46876}]},"active_addr":"172.21.15.191:6833/46876","active_change":"2024-03-14T09:01:07.506417+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":2731571504}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":2527411268}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":483211291}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1959179879}]}]} 2024-03-14T09:01:24.237 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-14T09:01:24.237 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-14T09:01:24.237 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:24.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.361+0000 7ff8a694a700 1 Processor -- start 2024-03-14T09:01:24.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.361+0000 7ff8a694a700 1 -- start start 2024-03-14T09:01:24.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8a00702f0 0x7ff8a00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8a014fbd0 0x7ff8a00c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a00c78c0 0x7ff8a00c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a00ac7b0 con 0x7ff8a014fbd0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a00acd10 con 0x7ff8a00702f0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a694a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0150050 con 0x7ff8a00c78c0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8a014fbd0 0x7ff8a00c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8a014fbd0 0x7ff8a00c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48632/0 (socket says 172.21.15.154:48632) 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 -- 172.21.15.154:0/2697617300 learned_addr learned my addr 172.21.15.154:0/2697617300 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89ffff700 1 --2- 172.21.15.154:0/2697617300 >> v2:172.21.15.191:3300/0 conn(0x7ff8a00702f0 0x7ff8a00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff8a4ee7700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a00c78c0 0x7ff8a00c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 -- 172.21.15.154:0/2697617300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a00c78c0 msgr2=0x7ff8a00c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a00c78c0 0x7ff8a00c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 -- 172.21.15.154:0/2697617300 >> v2:172.21.15.191:3300/0 conn(0x7ff8a00702f0 msgr2=0x7ff8a00c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/2697617300 >> v2:172.21.15.191:3300/0 conn(0x7ff8a00702f0 0x7ff8a00c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 -- 172.21.15.154:0/2697617300 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00ac9c0 con 0x7ff8a014fbd0 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a014fbd0 0x7ff8a00c7380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c8c00 tx=0x7ff894009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e257a13e48c36f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89effd700 1 -- 172.21.15.154:0/2697617300 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400b870 con 0x7ff8a014fbd0 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89effd700 1 -- 172.21.15.154:0/2697617300 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400b9d0 con 0x7ff8a014fbd0 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.362+0000 7ff89effd700 1 -- 172.21.15.154:0/2697617300 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400bd60 con 0x7ff8a014fbd0 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 -- 172.21.15.154:0/2697617300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a014fbd0 msgr2=0x7ff8a00c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a014fbd0 0x7ff8a00c7380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c8c00 tx=0x7ff894009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 -- 172.21.15.154:0/2697617300 shutdown_connections 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a00c78c0 0x7ff8a00c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 --2- 172.21.15.154:0/2697617300 >> v2:172.21.15.191:3300/0 conn(0x7ff8a00702f0 0x7ff8a00c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 --2- 172.21.15.154:0/2697617300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a014fbd0 0x7ff8a00c7380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 -- 172.21.15.154:0/2697617300 >> 172.21.15.154:0/2697617300 conn(0x7ff8a00a9bf0 msgr2=0x7ff8a00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 -- 172.21.15.154:0/2697617300 shutdown_connections 2024-03-14T09:01:24.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.363+0000 7ff8a694a700 1 -- 172.21.15.154:0/2697617300 wait complete. 2024-03-14T09:01:24.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.364+0000 7ff8a694a700 1 Processor -- start 2024-03-14T09:01:24.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.364+0000 7ff8a694a700 1 -- start start 2024-03-14T09:01:24.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a00702f0 0x7ff8a01eba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a014fbd0 0x7ff8a0147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0153250 con 0x7ff8a00c78c0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a0150460 con 0x7ff8a00702f0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a694a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01e8950 con 0x7ff8a014fbd0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48638/0 (socket says 172.21.15.154:48638) 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 -- 172.21.15.154:0/1222653466 learned_addr learned my addr 172.21.15.154:0/1222653466 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89ffff700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a00702f0 0x7ff8a01eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff8a4ee7700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a014fbd0 0x7ff8a0147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 -- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a014fbd0 msgr2=0x7ff8a0147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a014fbd0 0x7ff8a0147700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 -- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a00702f0 msgr2=0x7ff8a01eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a00702f0 0x7ff8a01eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 -- 172.21.15.154:0/1222653466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009550 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.365+0000 7ff89f7fe700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8a006b710 tx=0x7ff894004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.366+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894004480 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.366+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940045e0 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.366+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a0150690 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.366+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940038e0 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.366+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a01eaf40 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.367+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a00697d0 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.367+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff894021070 con 0x7ff8a00c78c0 2024-03-14T09:01:24.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.367+0000 7ff89cff9700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 0x7ff88c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.368+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff89408a380 con 0x7ff8a00c78c0 2024-03-14T09:01:24.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.368+0000 7ff89ffff700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 0x7ff88c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.368+0000 7ff89ffff700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 0x7ff88c06ccf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8a01e57d0 tx=0x7ff890008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.372+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff894055950 con 0x7ff8a00c78c0 2024-03-14T09:01:24.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.567+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8a00c7160 con 0x7ff8a00c78c0 2024-03-14T09:01:24.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.569+0000 7ff89cff9700 1 -- 172.21.15.154:0/1222653466 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7ff894058f70 con 0x7ff8a00c78c0 2024-03-14T09:01:24.569 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:24.569 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 msgr2=0x7ff88c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 0x7ff88c06ccf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8a01e57d0 tx=0x7ff890008040 comp rx=0 tx=0).stop 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 msgr2=0x7ff8a01e8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8a006b710 tx=0x7ff894004750 comp rx=0 tx=0).stop 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 shutdown_connections 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8a014fbd0 0x7ff8a0147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8a00702f0 0x7ff8a01eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff88c06a830 0x7ff88c06ccf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 --2- 172.21.15.154:0/1222653466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8a00c78c0 0x7ff8a01e8410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 >> 172.21.15.154:0/1222653466 conn(0x7ff8a00a9bf0 msgr2=0x7ff8a00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:24.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 shutdown_connections 2024-03-14T09:01:24.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.571+0000 7ff8a694a700 1 -- 172.21.15.154:0/1222653466 wait complete. 2024-03-14T09:01:24.583 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-14T09:01:24.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:24.682 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-14T09:01:24.682+0000 7f1cb2e30700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:01:24.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.706+0000 7f2f0e870700 1 Processor -- start 2024-03-14T09:01:24.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.706+0000 7f2f0e870700 1 -- start start 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f080702f0 0x7f2f080c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0814fbd0 0x7f2f080c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f080c78c0 0x7f2f080c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080ac7b0 con 0x7f2f080c78c0 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080acd10 con 0x7f2f080702f0 2024-03-14T09:01:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0e870700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f08150050 con 0x7f2f0814fbd0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f080c78c0 0x7f2f080c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f080c78c0 0x7f2f080c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48644/0 (socket says 172.21.15.154:48644) 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/111896414 learned_addr learned my addr 172.21.15.154:0/111896414 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f077fe700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0814fbd0 0x7f2f080c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/111896414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0814fbd0 msgr2=0x7f2f080c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f07fff700 1 --2- 172.21.15.154:0/111896414 >> v2:172.21.15.191:3300/0 conn(0x7f2f080702f0 0x7f2f080c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0814fbd0 0x7f2f080c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/111896414 >> v2:172.21.15.191:3300/0 conn(0x7f2f080702f0 msgr2=0x7f2f080c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/111896414 >> v2:172.21.15.191:3300/0 conn(0x7f2f080702f0 0x7f2f080c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/111896414 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080ac9c0 con 0x7f2f080c78c0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f080c78c0 0x7f2f080c5d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f080c8c00 tx=0x7f2f0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ef3467fd129c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f06ffd700 1 -- 172.21.15.154:0/111896414 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000e070 con 0x7f2f080c78c0 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.707+0000 7f2f06ffd700 1 -- 172.21.15.154:0/111896414 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000b9c0 con 0x7f2f080c78c0 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f06ffd700 1 -- 172.21.15.154:0/111896414 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000bd70 con 0x7f2f080c78c0 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 -- 172.21.15.154:0/111896414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f080c78c0 msgr2=0x7f2f080c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f080c78c0 0x7f2f080c5d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f080c8c00 tx=0x7f2f0000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 -- 172.21.15.154:0/111896414 shutdown_connections 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f080c78c0 0x7f2f080c5d60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 --2- 172.21.15.154:0/111896414 >> v2:172.21.15.191:3300/0 conn(0x7f2f080702f0 0x7f2f080c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 --2- 172.21.15.154:0/111896414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0814fbd0 0x7f2f080c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 -- 172.21.15.154:0/111896414 >> 172.21.15.154:0/111896414 conn(0x7f2f080a9bf0 msgr2=0x7f2f080a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 -- 172.21.15.154:0/111896414 shutdown_connections 2024-03-14T09:01:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.708+0000 7f2f0e870700 1 -- 172.21.15.154:0/111896414 wait complete. 2024-03-14T09:01:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.709+0000 7f2f0e870700 1 Processor -- start 2024-03-14T09:01:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.709+0000 7f2f0e870700 1 -- start start 2024-03-14T09:01:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f080702f0 0x7f2f0806cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 0x7f2f081ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c1d50 con 0x7f2f0814fbd0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c18c0 con 0x7f2f080c78c0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0e870700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081ee330 con 0x7f2f080702f0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0ce0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0ce0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48656/0 (socket says 172.21.15.154:48656) 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/3047938359 learned_addr learned my addr 172.21.15.154:0/3047938359 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f07fff700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f080702f0 0x7f2f0806cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.710+0000 7f2f077fe700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 0x7f2f081ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f080702f0 msgr2=0x7f2f0806cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f080702f0 0x7f2f0806cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 msgr2=0x7f2f081ea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 0x7f2f081ea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 -- 172.21.15.154:0/3047938359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00009b30 con 0x7f2f0814fbd0 2024-03-14T09:01:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f077fe700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 0x7f2f081ea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0ce0d700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f0806b710 tx=0x7f2f00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00004250 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f000043d0 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f081ee610 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00004780 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.711+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f080c1550 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.712+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f080c1710 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.712+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2f0001b430 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.713+0000 7f2f04ff9700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 0x7f2ef406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.713+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f0008a290 con 0x7f2f0814fbd0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.713+0000 7f2f07fff700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 0x7f2ef406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.713+0000 7f2f07fff700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 0x7f2ef406ccf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800a950 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:24.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.717+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2f000556a0 con 0x7f2f0814fbd0 2024-03-14T09:01:24.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.910+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f0814f430 con 0x7f2f0814fbd0 2024-03-14T09:01:24.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.911+0000 7f2f04ff9700 1 -- 172.21.15.154:0/3047938359 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f2f00058cc0 con 0x7f2f0814fbd0 2024-03-14T09:01:24.911 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:24.911 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:24.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 msgr2=0x7f2ef406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 0x7f2ef406ccf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800a950 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:24.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 msgr2=0x7f2f081ea7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f0806b710 tx=0x7f2f00004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 shutdown_connections 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef406a830 0x7f2ef406ccf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0814fbd0 0x7f2f081ea7e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f080702f0 0x7f2f0806cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 --2- 172.21.15.154:0/3047938359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f080c78c0 0x7f2f081ea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 >> 172.21.15.154:0/3047938359 conn(0x7f2f080a9bf0 msgr2=0x7f2f080c2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.913+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 shutdown_connections 2024-03-14T09:01:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:24.914+0000 7f2f0e870700 1 -- 172.21.15.154:0/3047938359 wait complete. 2024-03-14T09:01:24.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-14T09:01:24.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-14T09:01:24.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-14T09:01:24.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-14T09:01:24.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-14T09:01:24.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-14T09:01:24.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-14T09:01:24.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-14T09:01:25.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.139+0000 7f3d5cc60700 1 Processor -- start 2024-03-14T09:01:25.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.139+0000 7f3d5cc60700 1 -- start start 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.140+0000 7f3d5cc60700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d58158c10 0x7f3d58152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.140+0000 7f3d5cc60700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 0x7f3d5806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.140+0000 7f3d5cc60700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580c1780 con 0x7f3d5806d420 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.140+0000 7f3d5cc60700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5806f660 con 0x7f3d58158c10 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.140+0000 7f3d5cc60700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d58153180 con 0x7f3d5806a770 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d58158c10 0x7f3d58152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d58158c10 0x7f3d58152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48660/0 (socket says 172.21.15.154:48660) 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 -- 172.21.15.154:0/637255070 learned_addr learned my addr 172.21.15.154:0/637255070 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d55d9b700 1 --2- 172.21.15.154:0/637255070 >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 0x7f3d5806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 -- 172.21.15.154:0/637255070 >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 msgr2=0x7f3d5806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 --2- 172.21.15.154:0/637255070 >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 0x7f3d5806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 -- 172.21.15.154:0/637255070 >> v1:172.21.15.191:6789/0 conn(0x7f3d5806d420 legacy=0x7f3d5806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d5659c700 1 -- 172.21.15.154:0/637255070 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580c25e0 con 0x7f3d58158c10 2024-03-14T09:01:25.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f3d55d9b700 1 --2- 172.21.15.154:0/637255070 >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 0x7f3d5806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:25.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.141+0000 7f20df642700 1 Processor -- start 2024-03-14T09:01:25.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.142+0000 7f3d5659c700 1 --2- 172.21.15.154:0/637255070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d58152c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d5806b820 tx=0x7f3d4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5f6041ffd44cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.143+0000 7f3d5559a700 1 -- 172.21.15.154:0/637255070 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c004230 con 0x7f3d58158c10 2024-03-14T09:01:25.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.143+0000 7f3d5559a700 1 -- 172.21.15.154:0/637255070 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c004390 con 0x7f3d58158c10 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.143+0000 7f3d5559a700 1 -- 172.21.15.154:0/637255070 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0036a0 con 0x7f3d58158c10 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.142+0000 7f20df642700 1 -- start start 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20df642700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f20d8155fd0 0x7f20d8156480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20df642700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d8159c20 con 0x7f20d80aad60 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20df642700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80aa010 con 0x7f20d8159840 2024-03-14T09:01:25.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20df642700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80aa280 con 0x7f20d8155fd0 2024-03-14T09:01:25.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f3d5cc60700 1 -- 172.21.15.154:0/637255070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 msgr2=0x7f3d58152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/637255070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d58152c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d5806b820 tx=0x7f3d4c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20dd3de700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f20d8159840 0x7f20d8070750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38300/0 (socket says 172.21.15.154:38300) 2024-03-14T09:01:25.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20dd3de700 1 -- 172.21.15.154:0/837779803 learned_addr learned my addr 172.21.15.154:0/837779803 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f3c7bd52700 1 Processor -- start 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f20dcbdd700 1 --2- 172.21.15.154:0/837779803 >> v2:172.21.15.191:3300/0 conn(0x7f20d8155fd0 0x7f20d8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.143+0000 7fa74b78f700 1 Processor -- start 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.143+0000 7fa74b78f700 1 -- start start 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7fa74b78f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 0x7fa74415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7fa74b78f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7fa74b78f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa744071030 con 0x7fa7440701e0 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7fa74b78f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74406e0d0 con 0x7fa744161d70 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7fa74b78f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa744070ba0 con 0x7fa74406c1d0 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.145+0000 7fa749d2c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.145+0000 7fa749d2c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48672/0 (socket says 172.21.15.154:48672) 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.145+0000 7fa749d2c700 1 -- 172.21.15.154:0/1569656049 learned_addr learned my addr 172.21.15.154:0/1569656049 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.144+0000 7f3c7bd52700 1 -- start start 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c7bd52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 0x7f3c741644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c7bd52700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c74072200 con 0x7f3c74072410 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c7bd52700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c74158cb0 con 0x7f3c74159750 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c7bd52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740c0f80 con 0x7f3c74156080 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c792ed700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f3c74159750 0x7f3c74071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38306/0 (socket says 172.21.15.154:38306) 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c79aee700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3c74072410 0x7f3c740c0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34494/0 (socket says 172.21.15.154:34494) 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c792ed700 1 -- 172.21.15.154:0/1999138038 learned_addr learned my addr 172.21.15.154:0/1999138038 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c7a2ef700 1 --2- 172.21.15.154:0/1999138038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 0x7f3c741644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127410049 0 0) 0x7f3c74072200 con 0x7f3c74072410 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c600035b0 con 0x7f3c74072410 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983630285 0 0) 0x7f3c74158cb0 con 0x7f3c74159750 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74072200 con 0x7f3c74159750 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/1999138038 >> v1:172.21.15.191:6789/0 conn(0x7f3c74159750 legacy=0x7f3c74071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042982959 0 0) 0x7f3c600035b0 con 0x7f3c74072410 2024-03-14T09:01:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/1999138038 >> v1:172.21.15.154:6789/0 conn(0x7f3c74072410 legacy=0x7f3c740c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/1999138038 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74071f20 con 0x7f3c74156080 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c7a2ef700 1 --2- 172.21.15.154:0/1999138038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 0x7f3c741644a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009870 tx=0x7f3c64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=768fadbf4b72f9b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f196952b700 1 Processor -- start 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f196952b700 1 -- start start 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f196952b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1964159840 0x7f1964070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f196952b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1964159c20 con 0x7f19640ab470 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f196952b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19640aa010 con 0x7f1964155fd0 2024-03-14T09:01:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f196952b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640aa280 con 0x7f1964159840 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f20dcbdd700 1 -- 172.21.15.154:0/837779803 >> v1:172.21.15.191:6789/0 conn(0x7f20d8159840 legacy=0x7f20d8070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f20dcbdd700 1 -- 172.21.15.154:0/837779803 >> v1:172.21.15.154:6789/0 conn(0x7f20d80aad60 legacy=0x7f20d80ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f20dcbdd700 1 -- 172.21.15.154:0/837779803 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80aa560 con 0x7f20d8155fd0 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.146+0000 7f20dcbdd700 1 --2- 172.21.15.154:0/837779803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d8155fd0 0x7f20d8156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009de0 tx=0x7f20d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483d3a2436748b1a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f19627fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1964155fd0 0x7f19640aad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38316/0 (socket says 172.21.15.154:38316) 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f19627fc700 1 -- 172.21.15.154:0/1238946041 learned_addr learned my addr 172.21.15.154:0/1238946041 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70001670 con 0x7f3c74156080 2024-03-14T09:01:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c64004030 con 0x7f3c74156080 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7f3c78aec700 1 -- 172.21.15.154:0/1999138038 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640043c0 con 0x7f3c74156080 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 -- 172.21.15.154:0/1999138038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 msgr2=0x7f3c741644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 --2- 172.21.15.154:0/1999138038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 0x7f3c741644a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009870 tx=0x7f3c64009b80 comp rx=0 tx=0).stop 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 -- 172.21.15.154:0/1999138038 shutdown_connections 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 --2- 172.21.15.154:0/1999138038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74156080 0x7f3c741644a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 -- 172.21.15.154:0/1999138038 >> 172.21.15.154:0/1999138038 conn(0x7f3c740c5e70 msgr2=0x7f3c740c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f1962ffd700 1 --2- 172.21.15.154:0/1238946041 >> v2:172.21.15.154:3300/0 conn(0x7f1964159840 0x7f1964070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7f3c7bd52700 1 -- 172.21.15.154:0/1999138038 shutdown_connections 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f20cffff700 1 -- 172.21.15.154:0/837779803 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d000e070 con 0x7f20d8155fd0 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1961ffb700 1 -- 172.21.15.154:0/1238946041 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164931102 0 0) 0x7f1954001110 con 0x7f19640ab470 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1962ffd700 1 -- 172.21.15.154:0/1238946041 >> v1:172.21.15.191:6789/0 conn(0x7f1964155fd0 legacy=0x7f19640aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1962ffd700 1 -- 172.21.15.154:0/1238946041 >> v1:172.21.15.154:6789/0 conn(0x7f19640ab470 legacy=0x7f19641635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1962ffd700 1 -- 172.21.15.154:0/1238946041 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640aa560 con 0x7f1964159840 2024-03-14T09:01:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1962ffd700 1 --2- 172.21.15.154:0/1238946041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964159840 0x7f1964070750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1958009840 tx=0x7f1958009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ec71297444b07a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f20cffff700 1 -- 172.21.15.154:0/837779803 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d000bad0 con 0x7f20d8155fd0 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f20cffff700 1 -- 172.21.15.154:0/837779803 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d000be80 con 0x7f20d8155fd0 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f3d5cc60700 1 -- 172.21.15.154:0/637255070 shutdown_connections 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/637255070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d58152c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/637255070 >> v2:172.21.15.191:3300/0 conn(0x7f3d5806a770 0x7f3d5806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f3d5cc60700 1 -- 172.21.15.154:0/637255070 >> 172.21.15.154:0/637255070 conn(0x7f3d580c3880 msgr2=0x7f3d580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1961ffb700 1 -- 172.21.15.154:0/1238946041 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011830 con 0x7f1964159840 2024-03-14T09:01:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1961ffb700 1 -- 172.21.15.154:0/1238946041 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958011990 con 0x7f1964159840 2024-03-14T09:01:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f1961ffb700 1 -- 172.21.15.154:0/1238946041 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958011d20 con 0x7f1964159840 2024-03-14T09:01:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.149+0000 7f3c7bd52700 1 -- 172.21.15.154:0/1999138038 wait complete. 2024-03-14T09:01:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f3d5cc60700 1 -- 172.21.15.154:0/637255070 shutdown_connections 2024-03-14T09:01:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f3c7bd52700 1 Processor -- start 2024-03-14T09:01:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f3c7bd52700 1 -- start start 2024-03-14T09:01:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 -- 172.21.15.154:0/1238946041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964159840 msgr2=0x7f1964070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 --2- 172.21.15.154:0/1238946041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964159840 0x7f1964070750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1958009840 tx=0x7f1958009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 -- 172.21.15.154:0/837779803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d8155fd0 msgr2=0x7f20d8156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 --2- 172.21.15.154:0/837779803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d8155fd0 0x7f20d8156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009de0 tx=0x7f20d000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 -- 172.21.15.154:0/837779803 shutdown_connections 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 --2- 172.21.15.154:0/837779803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d8155fd0 0x7f20d8156480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 -- 172.21.15.154:0/837779803 >> 172.21.15.154:0/837779803 conn(0x7f20d80c6130 msgr2=0x7f20d80c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f20df642700 1 -- 172.21.15.154:0/837779803 shutdown_connections 2024-03-14T09:01:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 -- 172.21.15.154:0/1238946041 shutdown_connections 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 --2- 172.21.15.154:0/1238946041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964159840 0x7f1964070750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 -- 172.21.15.154:0/1238946041 >> 172.21.15.154:0/1238946041 conn(0x7f19640c6130 msgr2=0x7f19640c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f196952b700 1 -- 172.21.15.154:0/1238946041 shutdown_connections 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.145+0000 7fa74952b700 1 --2- 172.21.15.154:0/1569656049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 0x7fa74415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.145+0000 7fa73bfff700 1 -- 172.21.15.154:0/1569656049 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174033096 0 0) 0x7fa744071030 con 0x7fa7440701e0 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa73bfff700 1 -- 172.21.15.154:0/1569656049 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730003590 con 0x7fa7440701e0 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa74952b700 1 -- 172.21.15.154:0/1569656049 >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 msgr2=0x7fa744070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa74952b700 1 --2- 172.21.15.154:0/1569656049 >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa74952b700 1 -- 172.21.15.154:0/1569656049 >> v1:172.21.15.154:6789/0 conn(0x7fa7440701e0 legacy=0x7fa744161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa74952b700 1 -- 172.21.15.154:0/1569656049 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744072520 con 0x7fa74406c1d0 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.147+0000 7fa74952b700 1 --2- 172.21.15.154:0/1569656049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 0x7fa74415ec50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00eed0 tx=0x7fa73c007710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa905121c73facfb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7fa73bfff700 1 -- 172.21.15.154:0/1569656049 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00e250 con 0x7fa74406c1d0 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7fa73bfff700 1 -- 172.21.15.154:0/1569656049 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c00e3b0 con 0x7fa74406c1d0 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.148+0000 7fa73bfff700 1 -- 172.21.15.154:0/1569656049 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c018860 con 0x7fa74406c1d0 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 -- 172.21.15.154:0/1569656049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 msgr2=0x7fa74415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 --2- 172.21.15.154:0/1569656049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 0x7fa74415ec50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00eed0 tx=0x7fa73c007710 comp rx=0 tx=0).stop 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa749d2c700 1 --2- 172.21.15.154:0/1569656049 >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 -- 172.21.15.154:0/1569656049 shutdown_connections 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 --2- 172.21.15.154:0/1569656049 >> v2:172.21.15.154:3300/0 conn(0x7fa744161d70 0x7fa744070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 --2- 172.21.15.154:0/1569656049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa74406c1d0 0x7fa74415ec50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7fa74b78f700 1 -- 172.21.15.154:0/1569656049 >> 172.21.15.154:0/1569656049 conn(0x7fa7440c2970 msgr2=0x7fa7440c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7fa74b78f700 1 -- 172.21.15.154:0/1569656049 shutdown_connections 2024-03-14T09:01:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7fa74b78f700 1 -- 172.21.15.154:0/1569656049 wait complete. 2024-03-14T09:01:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f815874d700 1 Processor -- start 2024-03-14T09:01:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f815874d700 1 -- start start 2024-03-14T09:01:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.150+0000 7f3d5cc60700 1 -- 172.21.15.154:0/637255070 wait complete. 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c74072410 0x7f3c74158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 0x7f3c741efb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74159750 0x7f3c741f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741ee700 con 0x7f3c74156080 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741ee9c0 con 0x7f3c74072410 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f3c7bd52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741eeca0 con 0x7f3c74159750 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f3c7a2ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74159750 0x7f3c741f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c792ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 0x7f3c741efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c79aee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c74072410 0x7f3c74158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c792ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 0x7f3c741efb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48682/0 (socket says 172.21.15.154:48682) 2024-03-14T09:01:25.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c792ed700 1 -- 172.21.15.154:0/3778032248 learned_addr learned my addr 172.21.15.154:0/3778032248 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f196952b700 1 -- 172.21.15.154:0/1238946041 wait complete. 2024-03-14T09:01:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f196952b700 1 Processor -- start 2024-03-14T09:01:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f3d5cc60700 1 Processor -- start 2024-03-14T09:01:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f3d5cc60700 1 -- start start 2024-03-14T09:01:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7fa74b78f700 1 Processor -- start 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 0x7f8150156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81500adcc0 0x7f81500ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f815015c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500ad830 con 0x7f81500ad200 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500b0040 con 0x7f81500adcc0 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f815874d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81501569a0 con 0x7f815015c270 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f8156cea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f815015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f81564e9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 0x7f8150156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f8155ce8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81500adcc0 0x7f81500ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f8156cea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f815015c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:45994/0 (socket says 172.21.15.154:45994) 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f8155ce8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81500adcc0 0x7f81500ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38344/0 (socket says 172.21.15.154:38344) 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f8156cea700 1 -- 172.21.15.154:0/1659666859 learned_addr learned my addr 172.21.15.154:0/1659666859 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.152+0000 7f81564e9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 0x7f8150156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48698/0 (socket says 172.21.15.154:48698) 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f8155ce8700 1 -- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 msgr2=0x7f815015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f8155ce8700 1 --2- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f815015c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f8155ce8700 1 -- 172.21.15.154:0/1659666859 >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 msgr2=0x7f8150156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f8155ce8700 1 --2- 172.21.15.154:0/1659666859 >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 0x7f8150156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f8155ce8700 1 -- 172.21.15.154:0/1659666859 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500ae640 con 0x7f81500adcc0 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.151+0000 7f20df642700 1 -- 172.21.15.154:0/837779803 wait complete. 2024-03-14T09:01:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f20df642700 1 Processor -- start 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 0x7f3d58144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5806d420 0x7f3d580c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d581ee820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581f1d30 con 0x7f3d58158c10 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581f2010 con 0x7f3d5806a770 2024-03-14T09:01:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5cc60700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581f22f0 con 0x7f3d5806d420 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7fa74b78f700 1 -- start start 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7fa74b78f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa74406c1d0 0x7fa74415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74b78f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7440701e0 0x7fa7441fab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74b78f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa744161d70 0x7fa7440a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74b78f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441f7550 con 0x7fa744161d70 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74b78f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441fb0a0 con 0x7fa74406c1d0 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74b78f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74415fab0 con 0x7fa7440701e0 2024-03-14T09:01:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f8155ce8700 1 --2- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500adcc0 0x7f81500ae100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81500c9ca0 tx=0x7f814000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6218e6f31b1d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 0x7f3d58144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 0x7f3d58144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38346/0 (socket says 172.21.15.154:38346) 2024-03-14T09:01:25.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 -- 172.21.15.154:0/1529131185 learned_addr learned my addr 172.21.15.154:0/1529131185 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d56d9d700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d581ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c74072410 msgr2=0x7f3c74158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c74072410 0x7f3c74158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 msgr2=0x7f3c741efb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 0x7f3c741efb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640094a0 con 0x7f3c74159750 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f3c7a2ef700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74159750 0x7f3c741f00b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c64015000 tx=0x7f3c64004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3c792ed700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c74156080 0x7f3c741efb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f196952b700 1 -- start start 2024-03-14T09:01:25.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 0x7f19641f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f81554e7700 1 -- 172.21.15.154:0/1659666859 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000e070 con 0x7f81500adcc0 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964155fd0 0x7f19640bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1964159840 0x7f19641fab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19641e4ed0 con 0x7f1964155fd0 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19641f7250 con 0x7f1964159840 2024-03-14T09:01:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f196952b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19640bff30 con 0x7f19640ab470 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f81554e7700 1 -- 172.21.15.154:0/1659666859 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814000bb20 con 0x7f81500adcc0 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f81554e7700 1 -- 172.21.15.154:0/1659666859 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140012430 con 0x7f81500adcc0 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 0x7f19641f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 0x7f19641f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46018/0 (socket says 172.21.15.154:46018) 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 -- 172.21.15.154:0/2631204097 learned_addr learned my addr 172.21.15.154:0/2631204097 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f19627fc700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964155fd0 0x7f19640bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa748d2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7440701e0 0x7fa7441fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa748d2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7440701e0 0x7fa7441fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46010/0 (socket says 172.21.15.154:46010) 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa748d2a700 1 -- 172.21.15.154:0/2327817829 learned_addr learned my addr 172.21.15.154:0/2327817829 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa74406c1d0 0x7fa74415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 -- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5806d420 msgr2=0x7f3d580c5d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d5806d420 0x7f3d580c5d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 -- 172.21.15.154:0/1529131185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 msgr2=0x7f3d581ee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d58158c10 0x7f3d581ee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3d5659c700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009550 con 0x7f3d5806a770 2024-03-14T09:01:25.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64003da0 con 0x7f3c74159750 2024-03-14T09:01:25.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6401f070 con 0x7f3c74159750 2024-03-14T09:01:25.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64011870 con 0x7f3c74159750 2024-03-14T09:01:25.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c741eef80 con 0x7f3c74159750 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 -- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7440701e0 msgr2=0x7fa7441fab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7440701e0 0x7fa7441fab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 -- 172.21.15.154:0/2327817829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa744161d70 msgr2=0x7fa7440a8740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa744161d70 0x7fa7440a8740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7fa74952b700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c00eb30 con 0x7fa74406c1d0 2024-03-14T09:01:25.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 -- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1964159840 msgr2=0x7f19641fab20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1964159840 0x7f19641fab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 -- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964155fd0 msgr2=0x7f19640bf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964155fd0 0x7f19640bf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f1962ffd700 1 -- 172.21.15.154:0/2631204097 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580094a0 con 0x7f19640ab470 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3d5659c700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 0x7f3d58144b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d580c43c0 tx=0x7f3d4c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.153+0000 7f20df642700 1 -- start start 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f20df642700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d80aad60 0x7f20d81e2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f20df642700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20d8155fd0 0x7f20d81f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20df642700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d8159840 0x7f20d81fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20df642700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81fa5c0 con 0x7f20d8159840 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20df642700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d815b8b0 con 0x7f20d80aad60 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20df642700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81fa220 con 0x7f20d8155fd0 2024-03-14T09:01:25.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7fa74952b700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa74406c1d0 0x7fa74415f4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa74406feb0 tx=0x7fa73c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20dcbdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20d8155fd0 0x7f20d81f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20dcbdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20d8155fd0 0x7f20d81f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46020/0 (socket says 172.21.15.154:46020) 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20dcbdd700 1 -- 172.21.15.154:0/325209076 learned_addr learned my addr 172.21.15.154:0/325209076 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20dd3de700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d80aad60 0x7f20d81e2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c741585d0 con 0x7f3c74159750 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f19627fc700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1964155fd0 0x7f19640bf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:25.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f1962ffd700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 0x7f19641f6d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19580090b0 tx=0x7f1958004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f20ddbdf700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d8159840 0x7f20d81fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 -- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20d8155fd0 msgr2=0x7f20d81f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20d8155fd0 0x7f20d81f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 -- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d80aad60 msgr2=0x7f20d81e2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d80aad60 0x7f20d81e2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0009a70 con 0x7f20d8159840 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20dd3de700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20d80aad60 0x7f20d81e2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f815874d700 1 -- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500adcc0 msgr2=0x7f81500ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f815874d700 1 --2- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500adcc0 0x7f81500ae100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81500c9ca0 tx=0x7f814000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011630 con 0x7f3d5806a770 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d581f25d0 con 0x7f3d5806a770 2024-03-14T09:01:25.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d581505f0 con 0x7f3d5806a770 2024-03-14T09:01:25.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d580c1f90 con 0x7f3d5806a770 2024-03-14T09:01:25.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f20ddbdf700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d8159840 0x7f20d81fa9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c540 tx=0x7f20d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c74156510 con 0x7f3c74159750 2024-03-14T09:01:25.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.154+0000 7f461f46d700 1 Processor -- start 2024-03-14T09:01:25.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7f461f46d700 1 -- start start 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580036a0 con 0x7f19640ab470 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195801f070 con 0x7f19640ab470 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958003a40 con 0x7f19640ab470 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196415cb20 con 0x7f19640ab470 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196415d0e0 con 0x7f19640ab470 2024-03-14T09:01:25.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f461f46d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4618156440 0x7f46181593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f461f46d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4618159910 0x7f46180c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f461f46d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461815d160 con 0x7f46180c1dc0 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f461f46d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461806e530 con 0x7f4618159910 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f461f46d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4618158bf0 con 0x7f4618156440 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c011790 con 0x7f3d5806a770 2024-03-14T09:01:25.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011b40 con 0x7f3d5806a770 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40054e0 con 0x7f20d8159840 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d4003680 con 0x7f20d8159840 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4003a30 con 0x7f20d8159840 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d81e2e80 con 0x7f20d8159840 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d815a1c0 con 0x7f20d8159840 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f815874d700 1 -- 172.21.15.154:0/1659666859 shutdown_connections 2024-03-14T09:01:25.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f815874d700 1 --2- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f815015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f815874d700 1 --2- 172.21.15.154:0/1659666859 >> v2:172.21.15.154:3300/0 conn(0x7f81500ad200 0x7f8150156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f815874d700 1 --2- 172.21.15.154:0/1659666859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500adcc0 0x7f81500ae100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f815874d700 1 -- 172.21.15.154:0/1659666859 >> 172.21.15.154:0/1659666859 conn(0x7f81500ca070 msgr2=0x7f81500ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f461d209700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4618156440 0x7f46181593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f461d209700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4618156440 0x7f46181593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38368/0 (socket says 172.21.15.154:38368) 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f461d209700 1 -- 172.21.15.154:0/1410430009 learned_addr learned my addr 172.21.15.154:0/1410430009 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c021450 con 0x7fa74406c1d0 2024-03-14T09:01:25.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c0215b0 con 0x7fa74406c1d0 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.155+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c021940 con 0x7fa74406c1d0 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7441f6e90 con 0x7fa74406c1d0 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.156+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7440a8c80 con 0x7fa74406c1d0 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.157+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa744072000 con 0x7fa74406c1d0 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f461d209700 1 -- 172.21.15.154:0/1410430009 >> v2:172.21.15.154:3300/0 conn(0x7f4618159910 msgr2=0x7f46180c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f461d209700 1 --2- 172.21.15.154:0/1410430009 >> v2:172.21.15.154:3300/0 conn(0x7f4618159910 0x7f46180c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f461d209700 1 -- 172.21.15.154:0/1410430009 >> v1:172.21.15.154:6789/0 conn(0x7f46180c1dc0 legacy=0x7f46180701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f461d209700 1 -- 172.21.15.154:0/1410430009 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618071f20 con 0x7f4618156440 2024-03-14T09:01:25.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f461d209700 1 --2- 172.21.15.154:0/1410430009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4618156440 0x7f46181593d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46180c3780 tx=0x7f4608009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc2753972350d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.158+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1958003ba0 con 0x7f19640ab470 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f194b7fe700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f195006a930 0x7f195006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1958021070 con 0x7f19640ab470 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f196952b700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7f19440015e0 0x7f1944003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 --> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1944006bd0 con 0x7f19440015e0 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f19627fc700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f195006a930 0x7f195006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20d80a96e0 con 0x7f20d8159840 2024-03-14T09:01:25.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa73c008040 con 0x7fa74406c1d0 2024-03-14T09:01:25.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3d4c021070 con 0x7f3d5806a770 2024-03-14T09:01:25.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f460ffff700 1 -- 172.21.15.154:0/1410430009 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608004230 con 0x7f4618156440 2024-03-14T09:01:25.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f460ffff700 1 -- 172.21.15.154:0/1410430009 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608004390 con 0x7f4618156440 2024-03-14T09:01:25.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f460ffff700 1 -- 172.21.15.154:0/1410430009 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080036a0 con 0x7f4618156440 2024-03-14T09:01:25.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f19637fe700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7f19440015e0 0x7f1944003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d46ffd700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d3806a860 0x7f3d3806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.159+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3c640119d0 con 0x7f3c74159750 2024-03-14T09:01:25.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c6a7fc700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c5806a900 0x7f3c5806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c64021070 con 0x7f3c74159750 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c6a7fc700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f3c58072b20 0x7f3c58074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c580755f0 con 0x7f3c58072b20 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6408ce00 con 0x7f3c74159750 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c79aee700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c5806a900 0x7f3c5806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3c792ed700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f3c58072b20 0x7f3c58074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f3c792ed700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f3c58072b20 0x7f3c58074f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f461f46d700 1 -- 172.21.15.154:0/1410430009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4618156440 msgr2=0x7f46181593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f461f46d700 1 --2- 172.21.15.154:0/1410430009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4618156440 0x7f46181593d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f46180c3780 tx=0x7f4608009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f461f46d700 1 -- 172.21.15.154:0/1410430009 shutdown_connections 2024-03-14T09:01:25.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f461f46d700 1 --2- 172.21.15.154:0/1410430009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4618156440 0x7f46181593d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f461f46d700 1 --2- 172.21.15.154:0/1410430009 >> v2:172.21.15.154:3300/0 conn(0x7f4618159910 0x7f46180c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f461f46d700 1 -- 172.21.15.154:0/1410430009 >> 172.21.15.154:0/1410430009 conn(0x7f46180c2970 msgr2=0x7f46180c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f19627fc700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f195006a930 0x7f195006cdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1964158610 tx=0x7f195c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f20d4003b90 con 0x7f20d8159840 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20cdffb700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20bc06a900 0x7f20bc06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20d4088740 con 0x7f20d8159840 2024-03-14T09:01:25.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20dd3de700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20bc06a900 0x7f20bc06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20cdffb700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7f20bc072b20 0x7f20bc074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20bc0755f0 con 0x7f20bc072b20 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20d4088af0 con 0x7f20d8159840 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20dcbdd700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7f20bc072b20 0x7f20bc074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f20dd3de700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20bc06a900 0x7f20bc06cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20c8007c20 tx=0x7f20c8007b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d4c08a990 con 0x7f3d5806a770 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d46ffd700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f3d38072a00 0x7f3d38074e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d380754d0 con 0x7f3d38072a00 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d4c08ad00 con 0x7f3d5806a770 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f3d55d9b700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d3806a860 0x7f3d3806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f3d55d9b700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d3806a860 0x7f3d3806cd20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006fd0 tx=0x7f3d48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7f3d56d9d700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f3d38072a00 0x7f3d38074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f3d56d9d700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f3d38072a00 0x7f3d38074e20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa739ffb700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa72806a880 0x7fa72806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa73c08b2a0 con 0x7fa74406c1d0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa739ffb700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7fa728072aa0 0x7fa728074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa728075570 con 0x7fa728072aa0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa73c08b740 con 0x7fa74406c1d0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7fa749d2c700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7fa728072aa0 0x7fa728074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7fa749d2c700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7fa728072aa0 0x7fa728074ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.161+0000 7fa748d2a700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa72806a880 0x7fa72806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7fa748d2a700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa72806a880 0x7fa72806cd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa74415ae90 tx=0x7fa740009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f815874d700 1 -- 172.21.15.154:0/1659666859 shutdown_connections 2024-03-14T09:01:25.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.160+0000 7f815874d700 1 -- 172.21.15.154:0/1659666859 wait complete. 2024-03-14T09:01:25.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f815874d700 1 Processor -- start 2024-03-14T09:01:25.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f461f46d700 1 -- 172.21.15.154:0/1410430009 shutdown_connections 2024-03-14T09:01:25.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== osd.5 v2:172.21.15.191:6816/47551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa728075570 con 0x7fa728072aa0 2024-03-14T09:01:25.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f461f46d700 1 -- 172.21.15.154:0/1410430009 wait complete. 2024-03-14T09:01:25.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f20dcbdd700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7f20bc072b20 0x7f20bc074f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.162+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== osd.0 v2:172.21.15.154:6816/47282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3c580755f0 con 0x7f3c58072b20 2024-03-14T09:01:25.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.163+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== osd.4 v2:172.21.15.191:6808/47548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f3d380754d0 con 0x7f3d38072a00 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.163+0000 7f461f46d700 1 Processor -- start 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.163+0000 7f461f46d700 1 -- start start 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46180c1dc0 0x7f46180aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4618156440 0x7f46180c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 0x7f461814bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46181ef720 con 0x7f4618156440 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180a9a70 con 0x7f46181ef1a0 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461f46d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180a9d50 con 0x7f46180c1dc0 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 0x7f461814bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 0x7f461814bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38382/0 (socket says 172.21.15.154:38382) 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 -- 172.21.15.154:0/2350960655 learned_addr learned my addr 172.21.15.154:0/2350960655 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461ca08700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4618156440 0x7f46180c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461d209700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46180c1dc0 0x7f46180aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 -- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46180c1dc0 msgr2=0x7f46180aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46180c1dc0 0x7f46180aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 -- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4618156440 msgr2=0x7f46180c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4618156440 0x7f46180c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f461da0a700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608009550 con 0x7f46181ef1a0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f461da0a700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 0x7f461814bee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f461000c7a0 tx=0x7f461000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003e10 con 0x7f46181ef1a0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610005070 con 0x7f46181ef1a0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610005400 con 0x7f46181ef1a0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46180aa030 con 0x7f46181ef1a0 2024-03-14T09:01:25.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46180c5670 con 0x7f46181ef1a0 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4618158eb0 con 0x7f46181ef1a0 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.167+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4610005560 con 0x7f46181ef1a0 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.164+0000 7f815874d700 1 -- start start 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500ad200 0x7f8150159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500adcc0 0x7f81501f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f81501fa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8150159780 con 0x7f81500adcc0 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81501fa860 con 0x7f81500ad200 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f815874d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81501fab40 con 0x7f815015c270 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f81501fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f81501fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46026/0 (socket says 172.21.15.154:46026) 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 -- 172.21.15.154:0/357104650 learned_addr learned my addr 172.21.15.154:0/357104650 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 -- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500ad200 msgr2=0x7f8150159240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81500ad200 0x7f8150159240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 -- 172.21.15.154:0/357104650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500adcc0 msgr2=0x7f81501f9d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500adcc0 0x7f81501f9d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140009a70 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.165+0000 7f8156cea700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f81501fa260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f814400e900 tx=0x7f814400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400c850 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814400c9b0 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8150156b70 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400cd60 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.166+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8150157130 con 0x7f815015c270 2024-03-14T09:01:25.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.167+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81501facb0 con 0x7f815015c270 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f460dffb700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f45fc06a880 0x7f45fc06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4610088750 con 0x7f46181ef1a0 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f460dffb700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7f45fc072aa0 0x7f45fc074ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45fc075570 con 0x7f45fc072aa0 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4610088bf0 con 0x7f46181ef1a0 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8144013470 con 0x7f815015c270 2024-03-14T09:01:25.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f461ca08700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7f45fc072aa0 0x7f45fc074ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f461d209700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f45fc06a880 0x7f45fc06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f814effd700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c08c910 0x7f813c08edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== osd.3 v2:172.21.15.154:6800/47281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f20bc0755f0 con 0x7f20bc072b20 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81440891c0 con 0x7f815015c270 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f814effd700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f813c094b30 0x7f813c096f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f813c097600 con 0x7f813c094b30 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8144089570 con 0x7f815015c270 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f461d209700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f45fc06a880 0x7f45fc06cd40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4608009c10 tx=0x7f4608008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f461ca08700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7f45fc072aa0 0x7f45fc074ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.168+0000 7f19637fe700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7f19440015e0 0x7f1944003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== osd.2 v2:172.21.15.154:6802/47280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1944006bd0 con 0x7f19440015e0 2024-03-14T09:01:25.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.169+0000 7f3c79aee700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c5806a900 0x7f3c5806cdc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c70001380 tx=0x7f3c70000f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.170+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== osd.1 v2:172.21.15.154:6824/47283 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f45fc075570 con 0x7f45fc072aa0 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.171+0000 7fa8ecf92700 1 Processor -- start 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.171+0000 7fa8ecf92700 1 -- start start 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8ecf92700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8e80c2860 0x7fa8e80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8ecf92700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa8e8156040 0x7fa8e81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8ecf92700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e8159a40 con 0x7fa8e80c46b0 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8ecf92700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80c3740 con 0x7fa8e80c2860 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8ecf92700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80a7290 con 0x7fa8e8156040 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8e659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8e80c2860 0x7fa8e80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8e659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa8e80c2860 0x7fa8e80c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48756/0 (socket says 172.21.15.154:48756) 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8e659c700 1 -- 172.21.15.154:0/3404745538 learned_addr learned my addr 172.21.15.154:0/3404745538 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.173+0000 7fa8e5d9b700 1 --2- 172.21.15.154:0/3404745538 >> v2:172.21.15.191:3300/0 conn(0x7fa8e8156040 0x7fa8e81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e559a700 1 -- 172.21.15.154:0/3404745538 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177349265 0 0) 0x7fa8e8159a40 con 0x7fa8e80c46b0 2024-03-14T09:01:25.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e559a700 1 -- 172.21.15.154:0/3404745538 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d00035b0 con 0x7fa8e80c46b0 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e659c700 1 -- 172.21.15.154:0/3404745538 >> v2:172.21.15.191:3300/0 conn(0x7fa8e8156040 msgr2=0x7fa8e81564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e659c700 1 --2- 172.21.15.154:0/3404745538 >> v2:172.21.15.191:3300/0 conn(0x7fa8e8156040 0x7fa8e81564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e659c700 1 -- 172.21.15.154:0/3404745538 >> v1:172.21.15.191:6789/0 conn(0x7fa8e80c46b0 legacy=0x7fa8e80c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e659c700 1 -- 172.21.15.154:0/3404745538 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80a74a0 con 0x7fa8e80c2860 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.174+0000 7fa8e659c700 1 --2- 172.21.15.154:0/3404745538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c2860 0x7fa8e80c7df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80a6da0 tx=0x7fa8d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bdfce6ef565722 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.175+0000 7fa8e559a700 1 -- 172.21.15.154:0/3404745538 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8011830 con 0x7fa8e80c2860 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.175+0000 7fa8e559a700 1 -- 172.21.15.154:0/3404745538 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8011990 con 0x7fa8e80c2860 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.175+0000 7fa8e559a700 1 -- 172.21.15.154:0/3404745538 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8011d20 con 0x7fa8e80c2860 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 -- 172.21.15.154:0/3404745538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c2860 msgr2=0x7fa8e80c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/3404745538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c2860 0x7fa8e80c7df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80a6da0 tx=0x7fa8d8009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 -- 172.21.15.154:0/3404745538 shutdown_connections 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/3404745538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c2860 0x7fa8e80c7df0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/3404745538 >> v2:172.21.15.191:3300/0 conn(0x7fa8e8156040 0x7fa8e81564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 -- 172.21.15.154:0/3404745538 >> 172.21.15.154:0/3404745538 conn(0x7fa8e806a770 msgr2=0x7fa8e806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 -- 172.21.15.154:0/3404745538 shutdown_connections 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.176+0000 7fa8ecf92700 1 -- 172.21.15.154:0/3404745538 wait complete. 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.178+0000 7fa8ecf92700 1 Processor -- start 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.178+0000 7fa8ecf92700 1 -- start start 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 0x7fa8e8148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c46b0 0x7fa8e813cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e8156040 0x7fa8e813d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81575c0 con 0x7fa8e80c46b0 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8157e80 con 0x7fa8e80c2860 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.179+0000 7fa8ecf92700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81442c0 con 0x7fa8e8156040 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 0x7fa8e8148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c46b0 0x7fa8e813cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 0x7fa8e8148460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38396/0 (socket says 172.21.15.154:38396) 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 -- 172.21.15.154:0/1929539642 learned_addr learned my addr 172.21.15.154:0/1929539642 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e6d9d700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e8156040 0x7fa8e813d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 -- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e8156040 msgr2=0x7fa8e813d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e8156040 0x7fa8e813d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 -- 172.21.15.154:0/1929539642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c46b0 msgr2=0x7fa8e813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8e80c46b0 0x7fa8e813cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 -- 172.21.15.154:0/1929539642 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009550 con 0x7fa8e80c2860 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.180+0000 7fa8e659c700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 0x7fa8e8148460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8004450 con 0x7fa8e80c2860 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d80045b0 con 0x7fa8e80c2860 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d80038e0 con 0x7fa8e80c2860 2024-03-14T09:01:25.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e8143d60 con 0x7fa8e80c2860 2024-03-14T09:01:25.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7f81564e9700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c08c910 0x7f813c08edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7f8155ce8700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f813c094b30 0x7f813c096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.181+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e8157b10 con 0x7fa8e80c2860 2024-03-14T09:01:25.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.184+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa8d8003a40 con 0x7fa8e80c2860 2024-03-14T09:01:25.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.187+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 --> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3c74159230 con 0x7f3c58072b20 2024-03-14T09:01:25.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.187+0000 7f3c6a7fc700 1 -- 172.21.15.154:0/3778032248 <== osd.0 v2:172.21.15.154:6816/47282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3c6c001e50 con 0x7f3c58072b20 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f3c58072b20 msgr2=0x7f3c58074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f3c58072b20 0x7f3c58074f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c5806a900 msgr2=0x7f3c5806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c5806a900 0x7f3c5806cdc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c70001380 tx=0x7f3c70000f10 comp rx=0 tx=0).stop 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74159750 msgr2=0x7f3c741f00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 --2- 172.21.15.154:0/3778032248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c74159750 0x7f3c741f00b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c64015000 tx=0x7f3c64004750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7a2ef700 1 -- 172.21.15.154:0/3778032248 reap_dead start 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 shutdown_connections 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 >> 172.21.15.154:0/3778032248 conn(0x7f3c740c5e70 msgr2=0x7f3c740c0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 shutdown_connections 2024-03-14T09:01:25.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.189+0000 7f3c7bd52700 1 -- 172.21.15.154:0/3778032248 wait complete. 2024-03-14T09:01:25.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.189+0000 7fa8d6ffd700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa8c806a900 0x7fa8c806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.189+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8d8089e10 con 0x7fa8e80c2860 2024-03-14T09:01:25.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.189+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7fa8cc0015e0 0x7fa8cc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.189+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 --> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8cc006bd0 con 0x7fa8cc0015e0 2024-03-14T09:01:25.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.190+0000 7fa8e5d9b700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa8c806a900 0x7fa8c806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.190+0000 7fa8e6d9d700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7fa8cc0015e0 0x7fa8cc003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.190+0000 7fa8e5d9b700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa8c806a900 0x7fa8c806cdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806d740 tx=0x7fa8dc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.190+0000 7fa8e6d9d700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7fa8cc0015e0 0x7fa8cc003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.187+0000 7f81564e9700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c08c910 0x7f813c08edd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8148009950 tx=0x7f8148009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.187+0000 7f8155ce8700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f813c094b30 0x7f813c096f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.188+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== osd.6 v2:172.21.15.191:6817/47571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f813c097600 con 0x7f813c094b30 2024-03-14T09:01:25.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.191+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== osd.7 v2:172.21.15.191:6800/47550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa8cc006bd0 con 0x7fa8cc0015e0 2024-03-14T09:01:25.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.191+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 --> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f461815cbf0 con 0x7f45fc072aa0 2024-03-14T09:01:25.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f460dffb700 1 -- 172.21.15.154:0/2350960655 <== osd.1 v2:172.21.15.154:6824/47283 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46140095c0 con 0x7f45fc072aa0 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7f45fc072aa0 msgr2=0x7f45fc074ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7f45fc072aa0 0x7f45fc074ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f45fc06a880 msgr2=0x7f45fc06cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f45fc06a880 0x7f45fc06cd40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4608009c10 tx=0x7f4608008030 comp rx=0 tx=0).stop 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 msgr2=0x7f461814bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461f46d700 1 --2- 172.21.15.154:0/2350960655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46181ef1a0 0x7f461814bee0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f461000c7a0 tx=0x7f461000cab0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.192+0000 7f461da0a700 1 -- 172.21.15.154:0/2350960655 reap_dead start 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.193+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 shutdown_connections 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.193+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 >> 172.21.15.154:0/2350960655 conn(0x7f46180c2970 msgr2=0x7f4618157290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.193+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 shutdown_connections 2024-03-14T09:01:25.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.193+0000 7f461f46d700 1 -- 172.21.15.154:0/2350960655 wait complete. 2024-03-14T09:01:25.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.194+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 --> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa744070d90 con 0x7fa728072aa0 2024-03-14T09:01:25.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.194+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 --> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d580c1cb0 con 0x7f3d38072a00 2024-03-14T09:01:25.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.195+0000 7fa739ffb700 1 -- 172.21.15.154:0/2327817829 <== osd.5 v2:172.21.15.191:6816/47551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa73400a040 con 0x7fa728072aa0 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d46ffd700 1 -- 172.21.15.154:0/1529131185 <== osd.4 v2:172.21.15.191:6808/47548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d50005600 con 0x7f3d38072a00 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f3d38072a00 msgr2=0x7f3d38074e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f3d38072a00 0x7f3d38074e20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d3806a860 msgr2=0x7f3d3806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d3806a860 0x7f3d3806cd20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006fd0 tx=0x7f3d48008040 comp rx=0 tx=0).stop 2024-03-14T09:01:25.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 msgr2=0x7f3d58144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 --2- 172.21.15.154:0/1529131185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d5806a770 0x7f3d58144b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d580c43c0 tx=0x7f3d4c004750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d56d9d700 1 -- 172.21.15.154:0/1529131185 reap_dead start 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 shutdown_connections 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.199+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 >> 172.21.15.154:0/1529131185 conn(0x7f3d580c3880 msgr2=0x7f3d58071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.200+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 shutdown_connections 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.201+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 --> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1944005cb0 con 0x7f19440015e0 2024-03-14T09:01:25.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.202+0000 7f3d5cc60700 1 -- 172.21.15.154:0/1529131185 wait complete. 2024-03-14T09:01:25.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.204+0000 7f194b7fe700 1 -- 172.21.15.154:0/2631204097 <== osd.2 v2:172.21.15.154:6802/47280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1944005cb0 con 0x7f19440015e0 2024-03-14T09:01:25.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.204+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 --> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20d81fa080 con 0x7f20bc072b20 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.204+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7f19440015e0 msgr2=0x7f1944003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.204+0000 7f196952b700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7f19440015e0 0x7f1944003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f195006a930 msgr2=0x7f195006cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f195006a930 0x7f195006cdf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1964158610 tx=0x7f195c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 msgr2=0x7f19641f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 --2- 172.21.15.154:0/2631204097 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19640ab470 0x7f19641f6d10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19580090b0 tx=0x7f1958004750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f19637fe700 1 -- 172.21.15.154:0/2631204097 reap_dead start 2024-03-14T09:01:25.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 shutdown_connections 2024-03-14T09:01:25.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 >> 172.21.15.154:0/2631204097 conn(0x7f19640c6130 msgr2=0x7f19640c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 shutdown_connections 2024-03-14T09:01:25.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.205+0000 7f196952b700 1 -- 172.21.15.154:0/2631204097 wait complete. 2024-03-14T09:01:25.219 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-14T09:01:25.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.207+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7fa728072aa0 msgr2=0x7fa728074ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.207+0000 7fa74b78f700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7fa728072aa0 0x7fa728074ec0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa72806a880 msgr2=0x7fa72806cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa72806a880 0x7fa72806cd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa74415ae90 tx=0x7fa740009040 comp rx=0 tx=0).stop 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa74406c1d0 msgr2=0x7fa74415f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 --2- 172.21.15.154:0/2327817829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa74406c1d0 0x7fa74415f4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa74406feb0 tx=0x7fa73c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa749d2c700 1 -- 172.21.15.154:0/2327817829 reap_dead start 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 shutdown_connections 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.210+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 >> 172.21.15.154:0/2327817829 conn(0x7fa7440c2970 msgr2=0x7fa744159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.213+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 shutdown_connections 2024-03-14T09:01:25.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.213+0000 7fa74b78f700 1 -- 172.21.15.154:0/2327817829 wait complete. 2024-03-14T09:01:25.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20cdffb700 1 -- 172.21.15.154:0/325209076 <== osd.3 v2:172.21.15.154:6800/47281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f20d81fa080 con 0x7f20bc072b20 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7f20bc072b20 msgr2=0x7f20bc074f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20df642700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7f20bc072b20 0x7f20bc074f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20bc06a900 msgr2=0x7f20bc06cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20df642700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20bc06a900 0x7f20bc06cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20c8007c20 tx=0x7f20c8007b70 comp rx=0 tx=0).stop 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.215+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d8159840 msgr2=0x7f20d81fa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.216+0000 7f20df642700 1 --2- 172.21.15.154:0/325209076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20d8159840 0x7f20d81fa9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c540 tx=0x7f20d400c850 comp rx=0 tx=0).stop 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.216+0000 7f20ddbdf700 1 -- 172.21.15.154:0/325209076 reap_dead start 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.217+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 shutdown_connections 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.217+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 >> 172.21.15.154:0/325209076 conn(0x7f20d80c6130 msgr2=0x7f20d80c1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.217+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 shutdown_connections 2024-03-14T09:01:25.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.217+0000 7f20df642700 1 -- 172.21.15.154:0/325209076 wait complete. 2024-03-14T09:01:25.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.220+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 --> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81500c5280 con 0x7f813c094b30 2024-03-14T09:01:25.224 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.225 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-14T09:01:25.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.222+0000 7f814effd700 1 -- 172.21.15.154:0/357104650 <== osd.6 v2:172.21.15.191:6817/47571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8140003a20 con 0x7f813c094b30 2024-03-14T09:01:25.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.222+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f813c094b30 msgr2=0x7f813c096f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.222+0000 7f815874d700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f813c094b30 0x7f813c096f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.223+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 --> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8cc005cb0 con 0x7fa8cc0015e0 2024-03-14T09:01:25.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.223+0000 7fa8d6ffd700 1 -- 172.21.15.154:0/1929539642 <== osd.7 v2:172.21.15.191:6800/47550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8cc005cb0 con 0x7fa8cc0015e0 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c08c910 msgr2=0x7f813c08edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c08c910 0x7f813c08edd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8148009950 tx=0x7f8148009210 comp rx=0 tx=0).stop 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 msgr2=0x7f81501fa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 --2- 172.21.15.154:0/357104650 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f815015c270 0x7f81501fa260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f814400e900 tx=0x7f814400ecc0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f8156cea700 1 -- 172.21.15.154:0/357104650 reap_dead start 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 shutdown_connections 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 >> 172.21.15.154:0/357104650 conn(0x7f81500ca070 msgr2=0x7f81500c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 shutdown_connections 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.224+0000 7f815874d700 1 -- 172.21.15.154:0/357104650 wait complete. 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7fa8cc0015e0 msgr2=0x7fa8cc003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7fa8cc0015e0 0x7fa8cc003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa8c806a900 msgr2=0x7fa8c806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa8c806a900 0x7fa8c806cdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806d740 tx=0x7fa8dc009210 comp rx=0 tx=0).stop 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 msgr2=0x7fa8e8148460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8ecf92700 1 --2- 172.21.15.154:0/1929539642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e80c2860 0x7fa8e8148460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8000c00 tx=0x7fa8d8004750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.225+0000 7fa8e6d9d700 1 -- 172.21.15.154:0/1929539642 reap_dead start 2024-03-14T09:01:25.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.226+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 shutdown_connections 2024-03-14T09:01:25.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.226+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 >> 172.21.15.154:0/1929539642 conn(0x7fa8e806a770 msgr2=0x7fa8e806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.226+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 shutdown_connections 2024-03-14T09:01:25.230 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-14T09:01:25.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.227+0000 7fa8ecf92700 1 -- 172.21.15.154:0/1929539642 wait complete. 2024-03-14T09:01:25.231 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.232 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-14T09:01:25.236 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.236 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-14T09:01:25.237 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-14T09:01:25.247 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.248 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-14T09:01:25.250 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:25.251 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-14T09:01:25.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.423+0000 7fcb7f7f9700 1 Processor -- start 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.423+0000 7fcb7f7f9700 1 -- start start 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.424+0000 7fcb7f7f9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb78159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.424+0000 7fcb7f7f9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb78156080 0x7fcb780c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.424+0000 7fcb7f7f9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb78072200 con 0x7fcb780c0db0 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.424+0000 7fcb7f7f9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb78158cb0 con 0x7fcb78156080 2024-03-14T09:01:25.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.424+0000 7fcb7f7f9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb78071f20 con 0x7fcb78072410 2024-03-14T09:01:25.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb78156080 0x7fcb780c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb78156080 0x7fcb780c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48780/0 (socket says 172.21.15.154:48780) 2024-03-14T09:01:25.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 -- 172.21.15.154:0/256839891 learned_addr learned my addr 172.21.15.154:0/256839891 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 -- 172.21.15.154:0/256839891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 msgr2=0x7fcb78159750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T09:01:25.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 --2- 172.21.15.154:0/256839891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb78159750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 -- 172.21.15.154:0/256839891 >> v1:172.21.15.191:6789/0 conn(0x7fcb780c0db0 legacy=0x7fcb78071320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T09:01:25.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 -- 172.21.15.154:0/256839891 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78158f50 con 0x7fcb78156080 2024-03-14T09:01:25.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.426+0000 7fcb7cd94700 1 --2- 172.21.15.154:0/256839891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78156080 0x7fcb780c0870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009e50 tx=0x7fcb7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9029fc44a83c58b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.430+0000 7fcb6ffff700 1 -- 172.21.15.154:0/256839891 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7000e070 con 0x7fcb78156080 2024-03-14T09:01:25.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.430+0000 7fcb6ffff700 1 -- 172.21.15.154:0/256839891 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70003a20 con 0x7fcb78156080 2024-03-14T09:01:25.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.430+0000 7fcb6ffff700 1 -- 172.21.15.154:0/256839891 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003dd0 con 0x7fcb78156080 2024-03-14T09:01:25.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.432+0000 7f0e64d29700 1 Processor -- start 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.432+0000 7f0e64d29700 1 -- start start 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e64d29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600723b0 0x7f0e600c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e64d29700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e600aac00 0x7f0e600ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e64d29700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6006c8e0 con 0x7f0e600701e0 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e64d29700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e60072790 con 0x7f0e600aac00 2024-03-14T09:01:25.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e64d29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006cbc0 con 0x7f0e600723b0 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.433+0000 7f0e5dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e600aac00 0x7f0e600ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e600aac00 0x7f0e600ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48784/0 (socket says 172.21.15.154:48784) 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/388627756 learned_addr learned my addr 172.21.15.154:0/388627756 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/256839891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78156080 msgr2=0x7fcb780c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/256839891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78156080 0x7fcb780c0870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009e50 tx=0x7fcb7000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/256839891 shutdown_connections 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/256839891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb78159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/256839891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78156080 0x7fcb780c0870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/256839891 >> 172.21.15.154:0/256839891 conn(0x7fcb780c5e70 msgr2=0x7fcb780c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/388627756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600723b0 msgr2=0x7f0e600c1af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/388627756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600723b0 0x7f0e600c1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/388627756 >> v1:172.21.15.191:6789/0 conn(0x7f0e600701e0 legacy=0x7f0e60070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.434+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/388627756 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6006cea0 con 0x7f0e600aac00 2024-03-14T09:01:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.435+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/388627756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600aac00 0x7f0e600ab080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098f0 tx=0x7f0e50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a6cf8d96701a68 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.435+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/256839891 shutdown_connections 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/256839891 wait complete. 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.435+0000 7f0e5d59a700 1 -- 172.21.15.154:0/388627756 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50004230 con 0x7f0e600aac00 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.435+0000 7f0e5d59a700 1 -- 172.21.15.154:0/388627756 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50004390 con 0x7f0e600aac00 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.435+0000 7f0e5d59a700 1 -- 172.21.15.154:0/388627756 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000f740 con 0x7f0e600aac00 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 -- 172.21.15.154:0/388627756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600aac00 msgr2=0x7f0e600ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 --2- 172.21.15.154:0/388627756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600aac00 0x7f0e600ab080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098f0 tx=0x7f0e50009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 -- 172.21.15.154:0/388627756 shutdown_connections 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 --2- 172.21.15.154:0/388627756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600723b0 0x7f0e600c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 --2- 172.21.15.154:0/388627756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600aac00 0x7f0e600ab080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 -- 172.21.15.154:0/388627756 >> 172.21.15.154:0/388627756 conn(0x7f0e600c2970 msgr2=0x7f0e600c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 -- 172.21.15.154:0/388627756 shutdown_connections 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.437+0000 7fcb7f7f9700 1 Processor -- start 2024-03-14T09:01:25.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.437+0000 7fcb7f7f9700 1 -- start start 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.436+0000 7f0e64d29700 1 -- 172.21.15.154:0/388627756 wait complete. 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7f0e64d29700 1 Processor -- start 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb7814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb78156080 0x7fcb7814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 0x7fcb7813e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7815cbf0 con 0x7fcb7814ed10 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7814cee0 con 0x7fcb78156080 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.438+0000 7fcb7f7f9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7813e110 con 0x7fcb78072410 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7cd94700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb78156080 0x7fcb7814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7d595700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb7814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7d595700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb7814d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46036/0 (socket says 172.21.15.154:46036) 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7d595700 1 -- 172.21.15.154:0/28170057 learned_addr learned my addr 172.21.15.154:0/28170057 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7dd96700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 0x7fcb7813e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7f0e64d29700 1 -- start start 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7dd96700 1 -- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 msgr2=0x7fcb7814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7dd96700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb7814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7dd96700 1 -- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb78156080 msgr2=0x7fcb7814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7fcb7dd96700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb78156080 0x7fcb7814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7fcb7dd96700 1 -- 172.21.15.154:0/28170057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70009b30 con 0x7fcb7814ed10 2024-03-14T09:01:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7fcb7dd96700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 0x7fcb7813e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400b390 tx=0x7fcb7400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600701e0 0x7f0e601f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 0x7f0e600bb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 0x7f0e60154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600bd1f0 con 0x7f0e600723b0 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600baf20 con 0x7f0e600aac00 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e64d29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60151a20 con 0x7f0e600701e0 2024-03-14T09:01:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7f477e7d2700 1 Processor -- start 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.439+0000 7f477e7d2700 1 -- start start 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f477e7d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 0x7f47781593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f477e7d2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4778159910 0x7f47780c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f477e7d2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477815d160 con 0x7f47780c1dc0 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f477e7d2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477806e530 con 0x7f4778159910 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f477e7d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4778158bf0 con 0x7f4778156440 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f477cd6f700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f47780c1dc0 0x7f47780701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38354/0 (socket says 172.21.15.154:38354) 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f477cd6f700 1 -- 172.21.15.154:0/3009798691 learned_addr learned my addr 172.21.15.154:0/3009798691 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.440+0000 7f0e5ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 0x7f0e60154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 0x7f0e60154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38426/0 (socket says 172.21.15.154:38426) 2024-03-14T09:01:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5ed9d700 1 -- 172.21.15.154:0/542627953 learned_addr learned my addr 172.21.15.154:0/542627953 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 0x7f0e600bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f4777fff700 1 --2- 172.21.15.154:0/3009798691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 0x7f47781593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600701e0 msgr2=0x7f0e601f6dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600701e0 0x7f0e601f6dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 msgr2=0x7f0e60154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 0x7f0e60154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 -- 172.21.15.154:0/542627953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50009550 con 0x7f0e600723b0 2024-03-14T09:01:25.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f47777fe700 1 --2- 172.21.15.154:0/3009798691 >> v2:172.21.15.154:3300/0 conn(0x7f4778159910 0x7f47780c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e5dd9b700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 0x7f0e600bb190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e50017000 tx=0x7f0e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50024070 con 0x7f0e600723b0 2024-03-14T09:01:25.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500045e0 con 0x7f0e600723b0 2024-03-14T09:01:25.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500038b0 con 0x7f0e600723b0 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.442+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e60151d00 con 0x7f0e600723b0 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.442+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601522c0 con 0x7f0e600723b0 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f4777fff700 1 -- 172.21.15.154:0/3009798691 >> v2:172.21.15.154:3300/0 conn(0x7f4778159910 msgr2=0x7f47780c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f4777fff700 1 --2- 172.21.15.154:0/3009798691 >> v2:172.21.15.154:3300/0 conn(0x7f4778159910 0x7f47780c1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f4777fff700 1 -- 172.21.15.154:0/3009798691 >> v1:172.21.15.191:6789/0 conn(0x7f47780c1dc0 legacy=0x7f47780701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.441+0000 7f4777fff700 1 -- 172.21.15.154:0/3009798691 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778071f20 con 0x7f4778156440 2024-03-14T09:01:25.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.442+0000 7f4777fff700 1 --2- 172.21.15.154:0/3009798691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 0x7f47781593d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4768009840 tx=0x7f4768009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d118ed31b2984bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.443+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74011620 con 0x7fcb7814ed10 2024-03-14T09:01:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.443+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb740117a0 con 0x7fcb7814ed10 2024-03-14T09:01:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.443+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74011b50 con 0x7fcb7814ed10 2024-03-14T09:01:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.443+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7813fc10 con 0x7fcb7814ed10 2024-03-14T09:01:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.443+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7814cb70 con 0x7fcb7814ed10 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.444+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0e50003a10 con 0x7f0e600723b0 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.444+0000 7f0e4effd700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 0x7f0e4806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.445+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e50013070 con 0x7f0e600723b0 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.445+0000 7f0e5e59c700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 0x7f0e4806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.446+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440052f0 con 0x7f0e600723b0 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.446+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb580052f0 con 0x7fcb7814ed10 2024-03-14T09:01:25.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.446+0000 7fb24bc0f700 1 Processor -- start 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.447+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcb74011cb0 con 0x7fcb7814ed10 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.447+0000 7fb24bc0f700 1 -- start start 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24bc0f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2440c3250 0x7fb24413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24bc0f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb2440c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24bc0f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24413ade0 con 0x7fb2440a8540 2024-03-14T09:01:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24bc0f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb24406fb30 con 0x7fb2440c3250 2024-03-14T09:01:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24bc0f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24406ec30 con 0x7fb2440c1780 2024-03-14T09:01:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.447+0000 7f4776ffd700 1 -- 172.21.15.154:0/3009798691 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768011830 con 0x7f4778156440 2024-03-14T09:01:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7f4776ffd700 1 -- 172.21.15.154:0/3009798691 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768011990 con 0x7f4778156440 2024-03-14T09:01:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7f4776ffd700 1 -- 172.21.15.154:0/3009798691 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768011d00 con 0x7f4778156440 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb2491aa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2440c3250 0x7fb24413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb24a1ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb2440c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb2491aa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb2440c3250 0x7fb24413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38442/0 (socket says 172.21.15.154:38442) 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.448+0000 7fb2491aa700 1 -- 172.21.15.154:0/183768870 learned_addr learned my addr 172.21.15.154:0/183768870 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2491aa700 1 -- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 msgr2=0x7fb2440c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2491aa700 1 --2- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb2440c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2491aa700 1 -- 172.21.15.154:0/183768870 >> v1:172.21.15.154:6789/0 conn(0x7fb2440a8540 legacy=0x7fb244154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2491aa700 1 -- 172.21.15.154:0/183768870 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24406ef10 con 0x7fb2440c3250 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 -- 172.21.15.154:0/3009798691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 msgr2=0x7f47781593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 --2- 172.21.15.154:0/3009798691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 0x7f47781593d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4768009840 tx=0x7f4768009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 -- 172.21.15.154:0/3009798691 shutdown_connections 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 --2- 172.21.15.154:0/3009798691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4778156440 0x7f47781593d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 --2- 172.21.15.154:0/3009798691 >> v2:172.21.15.154:3300/0 conn(0x7f4778159910 0x7f47780c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7f477e7d2700 1 -- 172.21.15.154:0/3009798691 >> 172.21.15.154:0/3009798691 conn(0x7f47780c2970 msgr2=0x7f47780c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2491aa700 1 --2- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440c3250 0x7fb24413a8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb24000bb90 tx=0x7fb24000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4184a911d52f43 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2489a9700 1 -- 172.21.15.154:0/183768870 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24000dd30 con 0x7fb2440c3250 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2489a9700 1 -- 172.21.15.154:0/183768870 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24000de90 con 0x7fb2440c3250 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.449+0000 7fb2489a9700 1 -- 172.21.15.154:0/183768870 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240010690 con 0x7fb2440c3250 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 -- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440c3250 msgr2=0x7fb24413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440c3250 0x7fb24413a8a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb24000bb90 tx=0x7fb24000bea0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 -- 172.21.15.154:0/183768870 shutdown_connections 2024-03-14T09:01:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb2440c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/183768870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440c3250 0x7fb24413a8a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 -- 172.21.15.154:0/183768870 >> 172.21.15.154:0/183768870 conn(0x7fb2440c3880 msgr2=0x7fb2440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.450+0000 7fb24bc0f700 1 -- 172.21.15.154:0/183768870 shutdown_connections 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.451+0000 7fb24bc0f700 1 -- 172.21.15.154:0/183768870 wait complete. 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7fcb6dffb700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 0x7fcb5c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.451+0000 7f0e5e59c700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 0x7f0e4806cf70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009710 tx=0x7f0e54006c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7f477e7d2700 1 -- 172.21.15.154:0/3009798691 shutdown_connections 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7fb24bc0f700 1 Processor -- start 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7fb24bc0f700 1 -- start start 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb244153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2440c3250 0x7fb2441e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440ab000 con 0x7fb2440c3250 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440ac530 con 0x7fb2440a8540 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24bc0f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440ac760 con 0x7fb2440c1780 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb2491aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb244153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb2499ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb2499ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38446/0 (socket says 172.21.15.154:38446) 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb2499ab700 1 -- 172.21.15.154:0/3633832777 learned_addr learned my addr 172.21.15.154:0/3633832777 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.453+0000 7fb24a1ac700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2440c3250 0x7fb2441e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 -- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 msgr2=0x7fb244153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb244153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 -- 172.21.15.154:0/3633832777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2440c3250 msgr2=0x7fb2441e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2440c3250 0x7fb2441e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 -- 172.21.15.154:0/3633832777 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24000b7f0 con 0x7fb2440a8540 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7f477e7d2700 1 -- 172.21.15.154:0/3009798691 wait complete. 2024-03-14T09:01:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7f477e7d2700 1 Processor -- start 2024-03-14T09:01:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.452+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb74088b50 con 0x7fcb7814ed10 2024-03-14T09:01:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2499ab700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb23800c450 tx=0x7fb23800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2380052d0 con 0x7fb2440a8540 2024-03-14T09:01:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb238005430 con 0x7fb2440a8540 2024-03-14T09:01:25.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2380038c0 con 0x7fb2440a8540 2024-03-14T09:01:25.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2440aca40 con 0x7fb2440a8540 2024-03-14T09:01:25.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2441e9360 con 0x7fb2440a8540 2024-03-14T09:01:25.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7fcb7d595700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 0x7fcb5c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7fcb7d595700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 0x7fcb5c06cd40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb781638e0 tx=0x7fcb68009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.454+0000 7f477e7d2700 1 -- start start 2024-03-14T09:01:25.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47780c1dc0 0x7f477815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4778156440 0x7f47780a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47780ab1f0 con 0x7f47781506e0 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477815c8b0 con 0x7f4778156440 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f477e7d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477814cb20 con 0x7f47780c1dc0 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.455+0000 7f4777fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47780c1dc0 0x7f477815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48816/0 (socket says 172.21.15.154:48816) 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 -- 172.21.15.154:0/2456793474 learned_addr learned my addr 172.21.15.154:0/2456793474 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 -- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47780c1dc0 msgr2=0x7f477815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47780c1dc0 0x7f477815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 -- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4778156440 msgr2=0x7f47780a95b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4778156440 0x7f47780a95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 -- 172.21.15.154:0/2456793474 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680094a0 con 0x7f47781506e0 2024-03-14T09:01:25.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7f477cd6f700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f477000cbc0 tx=0x7f477000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.457+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0e50055440 con 0x7f0e600723b0 2024-03-14T09:01:25.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.456+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb238003a20 con 0x7fb2440a8540 2024-03-14T09:01:25.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.457+0000 7fb2367fc700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 0x7fb22806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.457+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb238087f90 con 0x7fb2440a8540 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.457+0000 7fb2491aa700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 0x7fb22806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.457+0000 7fb2491aa700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 0x7fb22806ccf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.459+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24413ade0 con 0x7fb2440a8540 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.463+0000 7feef3d31700 1 Processor -- start 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.463+0000 7feef3d31700 1 -- start start 2024-03-14T09:01:25.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb24413ade0 con 0x7fb2440a8540 2024-03-14T09:01:25.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7feef3d31700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feeec159840 0x7feeec070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7feef3d31700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7feef3d31700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec159c20 con 0x7feeec0aad60 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7feef3d31700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0aa010 con 0x7feeec159840 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.464+0000 7feef3d31700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0aa280 con 0x7feeec155fd0 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef1acd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feeec159840 0x7feeec070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46062/0 (socket says 172.21.15.154:46062) 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 -- 172.21.15.154:0/1634773963 learned_addr learned my addr 172.21.15.154:0/1634773963 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef0acb700 1 -- 172.21.15.154:0/1634773963 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488918696 0 0) 0x7feedc001110 con 0x7feeec0aad60 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 -- 172.21.15.154:0/1634773963 >> v2:172.21.15.191:3300/0 conn(0x7feeec159840 msgr2=0x7feeec070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 --2- 172.21.15.154:0/1634773963 >> v2:172.21.15.191:3300/0 conn(0x7feeec159840 0x7feeec070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 -- 172.21.15.154:0/1634773963 >> v1:172.21.15.191:6789/0 conn(0x7feeec0aad60 legacy=0x7feeec0ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.465+0000 7feef12cc700 1 -- 172.21.15.154:0/1634773963 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0aa560 con 0x7feeec155fd0 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcb74053fe0 con 0x7fcb7814ed10 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000ab50 con 0x7f47781506e0 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477000acb0 con 0x7f47781506e0 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770012600 con 0x7f47781506e0 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7feef12cc700 1 --2- 172.21.15.154:0/1634773963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feee4009ea0 tx=0x7feee400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4594e5694ea52cf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7feef0acb700 1 -- 172.21.15.154:0/1634773963 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400e070 con 0x7feeec155fd0 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7feef0acb700 1 -- 172.21.15.154:0/1634773963 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400b9c0 con 0x7feeec155fd0 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7feef0acb700 1 -- 172.21.15.154:0/1634773963 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400bd70 con 0x7feeec155fd0 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47780aab30 con 0x7f47781506e0 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.466+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477814cf30 con 0x7f47781506e0 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.467+0000 7feef3d31700 1 -- 172.21.15.154:0/1634773963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 msgr2=0x7feeec156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.467+0000 7feef3d31700 1 --2- 172.21.15.154:0/1634773963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feee4009ea0 tx=0x7feee400b330 comp rx=0 tx=0).stop 2024-03-14T09:01:25.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.468+0000 7feef3d31700 1 -- 172.21.15.154:0/1634773963 shutdown_connections 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.468+0000 7feef3d31700 1 --2- 172.21.15.154:0/1634773963 >> v2:172.21.15.191:3300/0 conn(0x7feeec159840 0x7feeec070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.468+0000 7feef3d31700 1 --2- 172.21.15.154:0/1634773963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec156450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.468+0000 7feef3d31700 1 -- 172.21.15.154:0/1634773963 >> 172.21.15.154:0/1634773963 conn(0x7feeec0c6130 msgr2=0x7feeec0c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.469+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f47700113a0 con 0x7f47781506e0 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.469+0000 7f4774ff9700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 0x7f476006cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.469+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47700886e0 con 0x7f47781506e0 2024-03-14T09:01:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.480+0000 7feef3d31700 1 -- 172.21.15.154:0/1634773963 shutdown_connections 2024-03-14T09:01:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.480+0000 7f4777fff700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 0x7f476006cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.480+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47580052f0 con 0x7f47781506e0 2024-03-14T09:01:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.480+0000 7feef3d31700 1 -- 172.21.15.154:0/1634773963 wait complete. 2024-03-14T09:01:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.482+0000 7feef3d31700 1 Processor -- start 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.482+0000 7feef3d31700 1 -- start start 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0aad60 0x7feeec1ee570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec0c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1e3980 con 0x7feeec159840 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1e4670 con 0x7feeec0aad60 2024-03-14T09:01:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.483+0000 7feef3d31700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1eec00 con 0x7feeec155fd0 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48820/0 (socket says 172.21.15.154:48820) 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 -- 172.21.15.154:0/3258711267 learned_addr learned my addr 172.21.15.154:0/3258711267 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 -- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 msgr2=0x7feeec0c4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec0c4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 -- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0aad60 msgr2=0x7feeec1ee570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0aad60 0x7feeec1ee570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 -- 172.21.15.154:0/3258711267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009b30 con 0x7feeec159840 2024-03-14T09:01:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7f4777fff700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 0x7f476006cf00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4778155db0 tx=0x7f4768008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.484+0000 7feef22ce700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feedc001eb0 tx=0x7feedc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.486+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0054e0 con 0x7feeec159840 2024-03-14T09:01:25.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.486+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc003680 con 0x7feeec159840 2024-03-14T09:01:25.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.486+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0039f0 con 0x7feeec159840 2024-03-14T09:01:25.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.486+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec1eeee0 con 0x7feeec159840 2024-03-14T09:01:25.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.487+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec1ef4a0 con 0x7feeec159840 2024-03-14T09:01:25.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.486+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4770053b70 con 0x7f47781506e0 2024-03-14T09:01:25.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.494+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7feedc003b50 con 0x7feeec159840 2024-03-14T09:01:25.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.494+0000 7feee27fc700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 0x7feed006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.494+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feedc087ee0 con 0x7feeec159840 2024-03-14T09:01:25.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.494+0000 7feef1acd700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 0x7feed006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.495+0000 7feef1acd700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 0x7feed006cdc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feeec1e4230 tx=0x7feee8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.501+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed40052f0 con 0x7feeec159840 2024-03-14T09:01:25.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.510+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7feedc053420 con 0x7feeec159840 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.518+0000 7fde13b32700 1 Processor -- start 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.518+0000 7fde13b32700 1 -- start start 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.519+0000 7fde13b32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c156440 0x7fde0c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.519+0000 7fde13b32700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde0c159910 0x7fde0c0c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde13b32700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c15d160 con 0x7fde0c0c1dc0 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde13b32700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c06e530 con 0x7fde0c159910 2024-03-14T09:01:25.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde13b32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c158bf0 con 0x7fde0c156440 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde120cf700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fde0c0c1dc0 0x7fde0c0701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38372/0 (socket says 172.21.15.154:38372) 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde120cf700 1 -- 172.21.15.154:0/3190800086 learned_addr learned my addr 172.21.15.154:0/3190800086 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.520+0000 7fde118ce700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c156440 0x7fde0c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 --2- 172.21.15.154:0/3190800086 >> v2:172.21.15.154:3300/0 conn(0x7fde0c159910 0x7fde0c0c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 -- 172.21.15.154:0/3190800086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c156440 msgr2=0x7fde0c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c156440 0x7fde0c1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 -- 172.21.15.154:0/3190800086 >> v1:172.21.15.191:6789/0 conn(0x7fde0c0c1dc0 legacy=0x7fde0c0701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 -- 172.21.15.154:0/3190800086 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c071f20 con 0x7fde0c159910 2024-03-14T09:01:25.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.521+0000 7fde110cd700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c159910 0x7fde0c0c1880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009840 tx=0x7fddfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f850b29092b7601d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.523+0000 7fde108cc700 1 -- 172.21.15.154:0/3190800086 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011830 con 0x7fde0c159910 2024-03-14T09:01:25.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.523+0000 7fde108cc700 1 -- 172.21.15.154:0/3190800086 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc011990 con 0x7fde0c159910 2024-03-14T09:01:25.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.523+0000 7fde108cc700 1 -- 172.21.15.154:0/3190800086 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011d00 con 0x7fde0c159910 2024-03-14T09:01:25.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 -- 172.21.15.154:0/3190800086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c159910 msgr2=0x7fde0c0c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c159910 0x7fde0c0c1880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009840 tx=0x7fddfc009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:25.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 -- 172.21.15.154:0/3190800086 shutdown_connections 2024-03-14T09:01:25.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c156440 0x7fde0c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190800086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c159910 0x7fde0c0c1880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.524+0000 7fde13b32700 1 -- 172.21.15.154:0/3190800086 >> 172.21.15.154:0/3190800086 conn(0x7fde0c0c2970 msgr2=0x7fde0c0c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.525+0000 7fbb34c62700 1 Processor -- start 2024-03-14T09:01:25.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.525+0000 7fbb34c62700 1 -- start start 2024-03-14T09:01:25.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fde13b32700 1 -- 172.21.15.154:0/3190800086 shutdown_connections 2024-03-14T09:01:25.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fbb34c62700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 0x7fbb300ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fbb34c62700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3006c8e0 con 0x7fbb300723b0 2024-03-14T09:01:25.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fbb34c62700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3006cbc0 con 0x7fbb300701e0 2024-03-14T09:01:25.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fbb34c62700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30072790 con 0x7fbb300aac00 2024-03-14T09:01:25.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.526+0000 7fde13b32700 1 -- 172.21.15.154:0/3190800086 wait complete. 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb2ed9d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbb300701e0 0x7fbb30070670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38376/0 (socket says 172.21.15.154:38376) 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb2ed9d700 1 -- 172.21.15.154:0/3711048070 learned_addr learned my addr 172.21.15.154:0/3711048070 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb27fff700 1 --2- 172.21.15.154:0/3711048070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 0x7fbb300ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb2dd9b700 1 -- 172.21.15.154:0/3711048070 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297764123 0 0) 0x7fbb3006cbc0 con 0x7fbb300701e0 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb2dd9b700 1 -- 172.21.15.154:0/3711048070 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1c0035b0 con 0x7fbb300701e0 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb27fff700 1 -- 172.21.15.154:0/3711048070 >> v1:172.21.15.191:6789/0 conn(0x7fbb300701e0 legacy=0x7fbb30070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb27fff700 1 -- 172.21.15.154:0/3711048070 >> v1:172.21.15.154:6789/0 conn(0x7fbb300723b0 legacy=0x7fbb300c1af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T09:01:25.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fbb27fff700 1 -- 172.21.15.154:0/3711048070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3006cea0 con 0x7fbb300aac00 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.527+0000 7fde13b32700 1 Processor -- start 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.528+0000 7fbb27fff700 1 --2- 172.21.15.154:0/3711048070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 0x7fbb300ab060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009870 tx=0x7fbb18009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64b02977404fe0f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.528+0000 7fbb2dd9b700 1 -- 172.21.15.154:0/3711048070 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004230 con 0x7fbb300aac00 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.528+0000 7fbb2dd9b700 1 -- 172.21.15.154:0/3711048070 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18004390 con 0x7fbb300aac00 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.528+0000 7fbb2dd9b700 1 -- 172.21.15.154:0/3711048070 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800f740 con 0x7fbb300aac00 2024-03-14T09:01:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fbb34c62700 1 -- 172.21.15.154:0/3711048070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 msgr2=0x7fbb300ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fbb34c62700 1 --2- 172.21.15.154:0/3711048070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 0x7fbb300ab060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009870 tx=0x7fbb18009b80 comp rx=0 tx=0).stop 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.528+0000 7fde13b32700 1 -- start start 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c0c1dc0 0x7fde0c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde0c156440 0x7fde0c13d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c15a760 con 0x7fde0c1554a0 2024-03-14T09:01:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0c3de0 con 0x7fde0c156440 2024-03-14T09:01:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde13b32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c13cce0 con 0x7fde0c0c1dc0 2024-03-14T09:01:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fbb34c62700 1 -- 172.21.15.154:0/3711048070 shutdown_connections 2024-03-14T09:01:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fbb34c62700 1 --2- 172.21.15.154:0/3711048070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb300aac00 0x7fbb300ab060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fbb34c62700 1 -- 172.21.15.154:0/3711048070 >> 172.21.15.154:0/3711048070 conn(0x7fbb300c2970 msgr2=0x7fbb300c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde120cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde120cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48852/0 (socket says 172.21.15.154:48852) 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde120cf700 1 -- 172.21.15.154:0/3190981413 learned_addr learned my addr 172.21.15.154:0/3190981413 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.529+0000 7fde110cd700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde0c156440 0x7fde0c13d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 -- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c0c1dc0 msgr2=0x7fde0c154f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c0c1dc0 0x7fde0c154f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 -- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde0c156440 msgr2=0x7fde0c13d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde0c156440 0x7fde0c13d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 -- 172.21.15.154:0/3190981413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0094a0 con 0x7fde0c1554a0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.530+0000 7fde120cf700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c480 tx=0x7fde0800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.531+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08005b70 con 0x7fde0c1554a0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.531+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08005cd0 con 0x7fde0c1554a0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.531+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde080072d0 con 0x7fde0c1554a0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.531+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c13cfc0 con 0x7fde0c1554a0 2024-03-14T09:01:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.531+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c13d450 con 0x7fde0c1554a0 2024-03-14T09:01:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.534+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fde08007430 con 0x7fde0c1554a0 2024-03-14T09:01:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.534+0000 7fde027fc700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 0x7fddf006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.534+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde08087e60 con 0x7fde0c1554a0 2024-03-14T09:01:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.535+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf40052f0 con 0x7fde0c1554a0 2024-03-14T09:01:25.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.538+0000 7fbb34c62700 1 -- 172.21.15.154:0/3711048070 shutdown_connections 2024-03-14T09:01:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.540+0000 7fde118ce700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 0x7fddf006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.540+0000 7fbb34c62700 1 -- 172.21.15.154:0/3711048070 wait complete. 2024-03-14T09:01:25.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.542+0000 7fbb34c62700 1 Processor -- start 2024-03-14T09:01:25.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.542+0000 7fbb34c62700 1 -- start start 2024-03-14T09:01:25.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 0x7fbb30154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb300aac00 0x7fbb3008a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 0x7fbb30153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30147040 con 0x7fbb300701e0 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300c0520 con 0x7fbb300aac00 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.543+0000 7fbb34c62700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300aa0e0 con 0x7fbb30153780 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.544+0000 7fbb2ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 0x7fbb30153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.544+0000 7fbb2ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 0x7fbb30153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46116/0 (socket says 172.21.15.154:46116) 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.544+0000 7fbb2ed9d700 1 -- 172.21.15.154:0/2844060704 learned_addr learned my addr 172.21.15.154:0/2844060704 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.544+0000 7fbb27fff700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb300aac00 0x7fbb3008a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fde080533a0 con 0x7fde0c1554a0 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.544+0000 7fbb2e59c700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 0x7fbb30154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 -- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 msgr2=0x7fbb30153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 0x7fbb30153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 -- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb300aac00 msgr2=0x7fbb3008a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb300aac00 0x7fbb3008a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 -- 172.21.15.154:0/2844060704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180094a0 con 0x7fbb300701e0 2024-03-14T09:01:25.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.545+0000 7fbb2e59c700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 0x7fbb30154850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006feb0 tx=0x7fbb2000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.547 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.546+0000 7fde118ce700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 0x7fddf006cdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde04009e70 tx=0x7fde04009830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.548+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20011620 con 0x7fbb300701e0 2024-03-14T09:01:25.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.548+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb200117a0 con 0x7fbb300701e0 2024-03-14T09:01:25.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.548+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20011b50 con 0x7fbb300701e0 2024-03-14T09:01:25.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.548+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb300aa3c0 con 0x7fbb300701e0 2024-03-14T09:01:25.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.548+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb301502c0 con 0x7fbb300701e0 2024-03-14T09:01:25.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.553+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbb20011cb0 con 0x7fbb300701e0 2024-03-14T09:01:25.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.553+0000 7fbb26ffd700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 0x7fbb1c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.553+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb20088af0 con 0x7fbb300701e0 2024-03-14T09:01:25.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.554+0000 7fbb27fff700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 0x7fbb1c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.554+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb140052f0 con 0x7fbb300701e0 2024-03-14T09:01:25.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.564+0000 7fbb27fff700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 0x7fbb1c06d270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097c0 tx=0x7fbb18008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.564+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbb20053f80 con 0x7fbb300701e0 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.567+0000 7f433551c700 1 Processor -- start 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.567+0000 7f433551c700 1 -- start start 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.568+0000 7f433551c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43300708f0 0x7f43300ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.568+0000 7f433551c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433006c8e0 con 0x7f43300723b0 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.568+0000 7f433551c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433006cbc0 con 0x7f43300aac00 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.568+0000 7f433551c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4330072790 con 0x7f43300708f0 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.569+0000 7f432e7fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f43300aac00 0x7f43300701e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38380/0 (socket says 172.21.15.154:38380) 2024-03-14T09:01:25.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.569+0000 7f432e7fc700 1 -- 172.21.15.154:0/3168688545 learned_addr learned my addr 172.21.15.154:0/3168688545 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:25.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.570+0000 7f432f7fe700 1 --2- 172.21.15.154:0/3168688545 >> v2:172.21.15.154:3300/0 conn(0x7f43300708f0 0x7f43300ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.570+0000 7f432f7fe700 1 -- 172.21.15.154:0/3168688545 >> v1:172.21.15.191:6789/0 conn(0x7f43300aac00 legacy=0x7f43300701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.570+0000 7f432f7fe700 1 -- 172.21.15.154:0/3168688545 >> v1:172.21.15.154:6789/0 conn(0x7f43300723b0 legacy=0x7f43300c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.570+0000 7f432f7fe700 1 -- 172.21.15.154:0/3168688545 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433006cea0 con 0x7f43300708f0 2024-03-14T09:01:25.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.571+0000 7f432f7fe700 1 --2- 172.21.15.154:0/3168688545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300708f0 0x7f43300ab060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4328009c80 tx=0x7f4328009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650cdb3349e425a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.572+0000 7f432dffb700 1 -- 172.21.15.154:0/3168688545 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800cab0 con 0x7f43300708f0 2024-03-14T09:01:25.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.572+0000 7f432dffb700 1 -- 172.21.15.154:0/3168688545 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800cc10 con 0x7f43300708f0 2024-03-14T09:01:25.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.572+0000 7f432dffb700 1 -- 172.21.15.154:0/3168688545 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328011400 con 0x7f43300708f0 2024-03-14T09:01:25.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.573+0000 7f433551c700 1 -- 172.21.15.154:0/3168688545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300708f0 msgr2=0x7f43300ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.573+0000 7f433551c700 1 --2- 172.21.15.154:0/3168688545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300708f0 0x7f43300ab060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4328009c80 tx=0x7f4328009f90 comp rx=0 tx=0).stop 2024-03-14T09:01:25.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:01:25.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.576+0000 7f433551c700 1 -- 172.21.15.154:0/3168688545 shutdown_connections 2024-03-14T09:01:25.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.576+0000 7f433551c700 1 --2- 172.21.15.154:0/3168688545 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300708f0 0x7f43300ab060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.576+0000 7f433551c700 1 -- 172.21.15.154:0/3168688545 >> 172.21.15.154:0/3168688545 conn(0x7f43300c2970 msgr2=0x7f43300c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.576+0000 7f433551c700 1 -- 172.21.15.154:0/3168688545 shutdown_connections 2024-03-14T09:01:25.591 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.576+0000 7f433551c700 1 -- 172.21.15.154:0/3168688545 wait complete. 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.578+0000 7f433551c700 1 Processor -- start 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.579+0000 7f433551c700 1 -- start start 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43300708f0 0x7f433013de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300723b0 0x7f433013bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433015cab0 con 0x7f43300aac00 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433014ddf0 con 0x7f43300723b0 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f433551c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433014e040 con 0x7f43300708f0 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.580+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48872/0 (socket says 172.21.15.154:48872) 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 -- 172.21.15.154:0/4204804668 learned_addr learned my addr 172.21.15.154:0/4204804668 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 -- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43300708f0 msgr2=0x7f433013de30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43300708f0 0x7f433013de30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 -- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300723b0 msgr2=0x7f433013bde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300723b0 0x7f433013bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 -- 172.21.15.154:0/4204804668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432800d040 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.581+0000 7f432f7fe700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f433006b750 tx=0x7f4328004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.583+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800cab0 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.584+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43280046e0 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.584+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432801a450 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.584+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433015d1d0 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.584+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433013e370 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.589+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f432801a5b0 con 0x7f43300aac00 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.589+0000 7f431f7fe700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 0x7f431406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:25.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.589+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f432808abc0 con 0x7f43300aac00 2024-03-14T09:01:25.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.591+0000 7f432effd700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 0x7f431406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:25.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.597+0000 7f432effd700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 0x7f431406cdc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4320001580 tx=0x7f4320007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.599+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c0052f0 con 0x7f43300aac00 2024-03-14T09:01:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.608+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4328056050 con 0x7f43300aac00 2024-03-14T09:01:25.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.871+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0e440055a0 con 0x7f0e600723b0 2024-03-14T09:01:25.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.872+0000 7f0e4effd700 1 -- 172.21.15.154:0/542627953 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e50058a60 con 0x7f0e600723b0 2024-03-14T09:01:25.877 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 msgr2=0x7f0e4806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 0x7f0e4806cf70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009710 tx=0x7f0e54006c60 comp rx=0 tx=0).stop 2024-03-14T09:01:25.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 msgr2=0x7f0e600bb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 0x7f0e600bb190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e50017000 tx=0x7f0e50004750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 shutdown_connections 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e600aac00 0x7f0e60154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e600701e0 0x7f0e601f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0e4806aab0 0x7f0e4806cf70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 --2- 172.21.15.154:0/542627953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e600723b0 0x7f0e600bb190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 >> 172.21.15.154:0/542627953 conn(0x7f0e600c2970 msgr2=0x7f0e600c4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.879 INFO:teuthology.orchestra.run.smithi154.stdout:55834574853 2024-03-14T09:01:25.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.877+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f47580055a0 con 0x7f47781506e0 2024-03-14T09:01:25.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f4774ff9700 1 -- 172.21.15.154:0/2456793474 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4770017090 con 0x7f47781506e0 2024-03-14T09:01:25.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 shutdown_connections 2024-03-14T09:01:25.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.878+0000 7f0e64d29700 1 -- 172.21.15.154:0/542627953 wait complete. 2024-03-14T09:01:25.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcb580055a0 con 0x7fcb7814ed10 2024-03-14T09:01:25.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.884+0000 7fcb6dffb700 1 -- 172.21.15.154:0/28170057 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb74015020 con 0x7fcb7814ed10 2024-03-14T09:01:25.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.884+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb24413ade0 con 0x7fb2440a8540 2024-03-14T09:01:25.885 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.886+0000 7fb2367fc700 1 -- 172.21.15.154:0/3633832777 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb24413ade0 con 0x7fb2440a8540 2024-03-14T09:01:25.888 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 msgr2=0x7f476006cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 0x7f476006cf00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4778155db0 tx=0x7f4768008030 comp rx=0 tx=0).stop 2024-03-14T09:01:25.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 msgr2=0x7f47780a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f477000cbc0 tx=0x7f477000cf80 comp rx=0 tx=0).stop 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 shutdown_connections 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f476006aa40 0x7f476006cf00 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4778155db0 tx=0x7f4768008030 comp rx=0 tx=0).stop 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47781506e0 0x7f47780a9af0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47780c1dc0 0x7f477815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 --2- 172.21.15.154:0/2456793474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4778156440 0x7f47780a95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.883+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 >> 172.21.15.154:0/2456793474 conn(0x7f47780c2970 msgr2=0x7f4778157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.884+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 shutdown_connections 2024-03-14T09:01:25.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.884+0000 7f477e7d2700 1 -- 172.21.15.154:0/2456793474 wait complete. 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 msgr2=0x7fb22806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 0x7fb22806ccf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240013000 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 msgr2=0x7fb2440abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb23800c450 tx=0x7fb23800c760 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 shutdown_connections 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb22806a830 0x7fb22806ccf0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb240000c00 tx=0x7fb240013000 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2440c3250 0x7fb2441e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2440a8540 0x7fb2440abd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 --2- 172.21.15.154:0/3633832777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440c1780 0x7fb244153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 >> 172.21.15.154:0/3633832777 conn(0x7fb2440c3880 msgr2=0x7fb2440c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 msgr2=0x7fcb5c06cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 0x7fcb5c06cd40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb781638e0 tx=0x7fcb68009210 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 msgr2=0x7fcb7813e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 0x7fcb7813e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb7400b390 tx=0x7fcb7400b6a0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 shutdown_connections 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcb5c06a880 0x7fcb5c06cd40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb7814ed10 0x7fcb7813e630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb78072410 0x7fcb7814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 --2- 172.21.15.154:0/28170057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb78156080 0x7fcb7814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 >> 172.21.15.154:0/28170057 conn(0x7fcb780c5e70 msgr2=0x7fcb78154fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.892+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 shutdown_connections 2024-03-14T09:01:25.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.895+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 shutdown_connections 2024-03-14T09:01:25.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.896+0000 7fb24bc0f700 1 -- 172.21.15.154:0/3633832777 wait complete. 2024-03-14T09:01:25.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.896+0000 7fcb7f7f9700 1 -- 172.21.15.154:0/28170057 wait complete. 2024-03-14T09:01:25.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-03-14T09:01:25.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-14T09:01:25.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-03-14T09:01:25.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-03-14T09:01:25.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.927+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feed40055a0 con 0x7feeec159840 2024-03-14T09:01:25.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.927+0000 7feee27fc700 1 -- 172.21.15.154:0/3258711267 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feedc017070 con 0x7feeec159840 2024-03-14T09:01:25.927 INFO:teuthology.orchestra.run.smithi154.stdout:55834574853 2024-03-14T09:01:25.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 msgr2=0x7feed006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 0x7feed006cdc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feeec1e4230 tx=0x7feee8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:25.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 msgr2=0x7feeec151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feedc001eb0 tx=0x7feedc00c7e0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 shutdown_connections 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7feed006a900 0x7feed006cdc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeec159840 0x7feeec151230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0aad60 0x7feeec1ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 --2- 172.21.15.154:0/3258711267 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec155fd0 0x7feeec0c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.930+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 >> 172.21.15.154:0/3258711267 conn(0x7feeec0c6130 msgr2=0x7feeec0c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.931+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 shutdown_connections 2024-03-14T09:01:25.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.931+0000 7feef3d31700 1 -- 172.21.15.154:0/3258711267 wait complete. 2024-03-14T09:01:25.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-03-14T09:01:25.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.953+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fddf40055a0 con 0x7fde0c1554a0 2024-03-14T09:01:25.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.954+0000 7fde027fc700 1 -- 172.21.15.154:0/3190981413 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde08017070 con 0x7fde0c1554a0 2024-03-14T09:01:25.953 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 msgr2=0x7fddf006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 0x7fddf006cdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fde04009e70 tx=0x7fde04009830 comp rx=0 tx=0).stop 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 msgr2=0x7fde0c13dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c480 tx=0x7fde0800c790 comp rx=0 tx=0).stop 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 shutdown_connections 2024-03-14T09:01:25.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fddf006a900 0x7fddf006cdc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c1554a0 0x7fde0c13dcf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde0c0c1dc0 0x7fde0c154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 --2- 172.21.15.154:0/3190981413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde0c156440 0x7fde0c13d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 >> 172.21.15.154:0/3190981413 conn(0x7fde0c0c2970 msgr2=0x7fde0c06bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 shutdown_connections 2024-03-14T09:01:25.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.956+0000 7fde13b32700 1 -- 172.21.15.154:0/3190981413 wait complete. 2024-03-14T09:01:25.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-03-14T09:01:25.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.981+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbb140055a0 con 0x7fbb300701e0 2024-03-14T09:01:25.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.981+0000 7fbb26ffd700 1 -- 172.21.15.154:0/2844060704 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb200575a0 con 0x7fbb300701e0 2024-03-14T09:01:25.981 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:25.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 msgr2=0x7fbb1c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 0x7fbb1c06d270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097c0 tx=0x7fbb18008030 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 msgr2=0x7fbb30154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 0x7fbb30154850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006feb0 tx=0x7fbb2000b750 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 shutdown_connections 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb30153780 0x7fbb30153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb1c06adb0 0x7fbb1c06d270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb300701e0 0x7fbb30154850 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 --2- 172.21.15.154:0/2844060704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb300aac00 0x7fbb3008a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.983+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 >> 172.21.15.154:0/2844060704 conn(0x7fbb300c2970 msgr2=0x7fbb300c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.984+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 shutdown_connections 2024-03-14T09:01:25.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:25.984+0000 7fbb34c62700 1 -- 172.21.15.154:0/2844060704 wait complete. 2024-03-14T09:01:26.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-03-14T09:01:26.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.012+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f430c0055a0 con 0x7f43300aac00 2024-03-14T09:01:26.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.012+0000 7f431f7fe700 1 -- 172.21.15.154:0/4204804668 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4328015070 con 0x7f43300aac00 2024-03-14T09:01:26.012 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 msgr2=0x7f431406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 0x7f431406cdc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4320001580 tx=0x7f4320007a20 comp rx=0 tx=0).stop 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 msgr2=0x7f433013c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f433006b750 tx=0x7f4328004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 shutdown_connections 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f431406a900 0x7f431406cdc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:26.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43300aac00 0x7f433013c320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:26.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43300708f0 0x7f433013de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:26.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 --2- 172.21.15.154:0/4204804668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43300723b0 0x7f433013bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:26.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 >> 172.21.15.154:0/4204804668 conn(0x7f43300c2970 msgr2=0x7f43301490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:26.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 shutdown_connections 2024-03-14T09:01:26.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:26.014+0000 7f433551c700 1 -- 172.21.15.154:0/4204804668 wait complete. 2024-03-14T09:01:26.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-03-14T09:01:26.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-14T09:01:26.911 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-14T09:01:26.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-14T09:01:26.920 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-14T09:01:26.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-14T09:01:26.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-14T09:01:27.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-14T09:01:27.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-14T09:01:27.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.083+0000 7fc086e96700 1 Processor -- start 2024-03-14T09:01:27.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.083+0000 7fc086e96700 1 -- start start 2024-03-14T09:01:27.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.084+0000 7fc086e96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 0x7fc080156460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.084+0000 7fc086e96700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0800adcc0 0x7fc0800ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.084+0000 7fc086e96700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0800ad830 con 0x7fc08015c270 2024-03-14T09:01:27.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.084+0000 7fc086e96700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08015ce10 con 0x7fc0800adcc0 2024-03-14T09:01:27.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.084+0000 7fc086e96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0800b0040 con 0x7fc0800ad200 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc084c32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 0x7fc080156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc085433700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc08015c270 0x7fc08015c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34546/0 (socket says 172.21.15.154:34546) 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc085433700 1 -- 172.21.15.154:0/1583183958 learned_addr learned my addr 172.21.15.154:0/1583183958 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc07ffff700 1 --2- 172.21.15.154:0/1583183958 >> v2:172.21.15.154:3300/0 conn(0x7fc0800adcc0 0x7fc0800ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc084c32700 1 -- 172.21.15.154:0/1583183958 >> v2:172.21.15.154:3300/0 conn(0x7fc0800adcc0 msgr2=0x7fc0800ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc084c32700 1 --2- 172.21.15.154:0/1583183958 >> v2:172.21.15.154:3300/0 conn(0x7fc0800adcc0 0x7fc0800ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.085+0000 7fc084c32700 1 -- 172.21.15.154:0/1583183958 >> v1:172.21.15.154:6789/0 conn(0x7fc08015c270 legacy=0x7fc08015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.086+0000 7fc084c32700 1 -- 172.21.15.154:0/1583183958 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0801569a0 con 0x7fc0800ad200 2024-03-14T09:01:27.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.086+0000 7fc084c32700 1 --2- 172.21.15.154:0/1583183958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 0x7fc080156460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc070009d90 tx=0x7fc07000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b55232e379ebdb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.086+0000 7fc07f7fe700 1 -- 172.21.15.154:0/1583183958 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07000e070 con 0x7fc0800ad200 2024-03-14T09:01:27.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.087+0000 7fc07f7fe700 1 -- 172.21.15.154:0/1583183958 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07000bbd0 con 0x7fc0800ad200 2024-03-14T09:01:27.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.087+0000 7fc07f7fe700 1 -- 172.21.15.154:0/1583183958 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070012430 con 0x7fc0800ad200 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 -- 172.21.15.154:0/1583183958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 msgr2=0x7fc080156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 --2- 172.21.15.154:0/1583183958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 0x7fc080156460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc070009d90 tx=0x7fc07000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 -- 172.21.15.154:0/1583183958 shutdown_connections 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 --2- 172.21.15.154:0/1583183958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800ad200 0x7fc080156460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 --2- 172.21.15.154:0/1583183958 >> v2:172.21.15.154:3300/0 conn(0x7fc0800adcc0 0x7fc0800ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7fc086e96700 1 -- 172.21.15.154:0/1583183958 >> 172.21.15.154:0/1583183958 conn(0x7fc0800ca070 msgr2=0x7fc0800ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.086+0000 7f637b0a7700 1 Processor -- start 2024-03-14T09:01:27.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.088+0000 7f637b0a7700 1 -- start start 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.089+0000 7f637b0a7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f63740723b0 0x7f63740c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.089+0000 7f637b0a7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637406c8e0 con 0x7f63740aac00 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.089+0000 7f637b0a7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637406cbc0 con 0x7f63740708f0 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.089+0000 7f637b0a7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6374072790 con 0x7f63740723b0 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6378e43700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f63740723b0 0x7f63740c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6379644700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f63740708f0 0x7f637415b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38388/0 (socket says 172.21.15.154:38388) 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6379644700 1 -- 172.21.15.154:0/1702079181 learned_addr learned my addr 172.21.15.154:0/1702079181 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6378e43700 1 -- 172.21.15.154:0/1702079181 >> v1:172.21.15.191:6789/0 conn(0x7f63740708f0 legacy=0x7f637415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6378e43700 1 -- 172.21.15.154:0/1702079181 >> v1:172.21.15.154:6789/0 conn(0x7f63740aac00 legacy=0x7f63740701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6378e43700 1 -- 172.21.15.154:0/1702079181 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637406cea0 con 0x7f63740723b0 2024-03-14T09:01:27.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f6378e43700 1 --2- 172.21.15.154:0/1702079181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740723b0 0x7f63740c1af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f636c00bfa0 tx=0x7f636c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c742b9983e45cad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f63737fe700 1 -- 172.21.15.154:0/1702079181 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c010070 con 0x7f63740723b0 2024-03-14T09:01:27.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f63737fe700 1 -- 172.21.15.154:0/1702079181 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c00b2e0 con 0x7f63740723b0 2024-03-14T09:01:27.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.090+0000 7f63737fe700 1 -- 172.21.15.154:0/1702079181 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00dcb0 con 0x7f63740723b0 2024-03-14T09:01:27.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.091+0000 7f637b0a7700 1 -- 172.21.15.154:0/1702079181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740723b0 msgr2=0x7f63740c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.091+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1702079181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740723b0 0x7f63740c1af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f636c00bfa0 tx=0x7f636c00d330 comp rx=0 tx=0).stop 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.092+0000 7f637b0a7700 1 -- 172.21.15.154:0/1702079181 shutdown_connections 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.092+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1702079181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740723b0 0x7f63740c1af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.092+0000 7f637b0a7700 1 -- 172.21.15.154:0/1702079181 >> 172.21.15.154:0/1702079181 conn(0x7f63740c2970 msgr2=0x7f63740c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.092+0000 7f637b0a7700 1 -- 172.21.15.154:0/1702079181 shutdown_connections 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.092+0000 7f637b0a7700 1 -- 172.21.15.154:0/1702079181 wait complete. 2024-03-14T09:01:27.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.093+0000 7f637b0a7700 1 Processor -- start 2024-03-14T09:01:27.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7fc086e96700 1 -- 172.21.15.154:0/1583183958 shutdown_connections 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7fc086e96700 1 -- 172.21.15.154:0/1583183958 wait complete. 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7fc086e96700 1 Processor -- start 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.093+0000 7f637b0a7700 1 -- start start 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63740723b0 0x7f63741438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740aac00 0x7f6374143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741ee3c0 con 0x7f63740708f0 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63740a9f40 con 0x7f63740aac00 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f637b0a7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63741ee680 con 0x7f63740723b0 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f6378e43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f6378e43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48894/0 (socket says 172.21.15.154:48894) 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.094+0000 7f6378e43700 1 -- 172.21.15.154:0/1442520661 learned_addr learned my addr 172.21.15.154:0/1442520661 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6379644700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740aac00 0x7f6374143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6378e43700 1 -- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63740723b0 msgr2=0x7f63741438a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6378e43700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63740723b0 0x7f63741438a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6378e43700 1 -- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740aac00 msgr2=0x7f6374143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6378e43700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740aac00 0x7f6374143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.095+0000 7f6378e43700 1 -- 172.21.15.154:0/1442520661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c00bc80 con 0x7f63740708f0 2024-03-14T09:01:27.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f6378e43700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f636c017040 tx=0x7f636c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7fc086e96700 1 -- start start 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 0x7fc0800ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800adcc0 0x7fc080150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 0x7fc08014ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801f7e30 con 0x7fc0800ad200 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801f78c0 con 0x7fc08015c270 2024-03-14T09:01:27.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc086e96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801f7b50 con 0x7fc0800adcc0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c004440 con 0x7f63740708f0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c0045e0 con 0x7f63740708f0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c009290 con 0x7f63740708f0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63741ee960 con 0x7f63740708f0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.096+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6374149410 con 0x7f63740708f0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc084c32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 0x7fc0800ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc084c32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 0x7fc0800ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48898/0 (socket says 172.21.15.154:48898) 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.097+0000 7fc084c32700 1 -- 172.21.15.154:0/263162810 learned_addr learned my addr 172.21.15.154:0/263162810 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 0x7fc08014ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc07ffff700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800adcc0 0x7fc080150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 -- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800adcc0 msgr2=0x7fc080150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800adcc0 0x7fc080150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 -- 172.21.15.154:0/263162810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 msgr2=0x7fc0800ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 0x7fc0800ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 -- 172.21.15.154:0/263162810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070009a70 con 0x7fc08015c270 2024-03-14T09:01:27.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.098+0000 7fc085433700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 0x7fc08014ef10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c390 tx=0x7fc07800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.100+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f636c01b070 con 0x7f63740708f0 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.100+0000 7f63717fa700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 0x7f636006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.100+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f636c08a090 con 0x7f63740708f0 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.101+0000 7f6373fff700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 0x7f636006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078005b40 con 0x7fc08015c270 2024-03-14T09:01:27.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078005ca0 con 0x7fc08015c270 2024-03-14T09:01:27.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078007310 con 0x7fc08015c270 2024-03-14T09:01:27.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0800aaa10 con 0x7fc08015c270 2024-03-14T09:01:27.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0800aafd0 con 0x7fc08015c270 2024-03-14T09:01:27.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.102+0000 7f6373fff700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 0x7f636006cdc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6368006fd0 tx=0x7f6368006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.104+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc078007470 con 0x7fc08015c270 2024-03-14T09:01:27.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.104+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0800ab1f0 con 0x7fc08015c270 2024-03-14T09:01:27.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.104+0000 7fc07d7fa700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 0x7fc06c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.104+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc078087f80 con 0x7fc08015c270 2024-03-14T09:01:27.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.105+0000 7fc084c32700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 0x7fc06c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.107+0000 7fc084c32700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 0x7fc06c06cdc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc070000f80 tx=0x7fc070011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.115+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc078053410 con 0x7fc08015c270 2024-03-14T09:01:27.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.116+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637404ea80 con 0x7f63740708f0 2024-03-14T09:01:27.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.126+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f636c055520 con 0x7f63740708f0 2024-03-14T09:01:27.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.163+0000 7fe4673e8700 1 Processor -- start 2024-03-14T09:01:27.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.163+0000 7f5df4886700 1 Processor -- start 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.163+0000 7f5df4886700 1 -- start start 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7f5df4886700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7f5df4886700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5df0161d70 0x7f5df0070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7f5df4886700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df0071030 con 0x7f5df00701e0 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7f5df4886700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df006e0d0 con 0x7f5df0161d70 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7f5df4886700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0070ba0 con 0x7f5df006c1d0 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7f5dee59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7f5dee59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46136/0 (socket says 172.21.15.154:46136) 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7f5dee59c700 1 -- 172.21.15.154:0/3048044076 learned_addr learned my addr 172.21.15.154:0/3048044076 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 --2- 172.21.15.154:0/3048044076 >> v2:172.21.15.154:3300/0 conn(0x7f5df0161d70 0x7f5df0070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 -- 172.21.15.154:0/3048044076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 msgr2=0x7f5df015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 --2- 172.21.15.154:0/3048044076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 -- 172.21.15.154:0/3048044076 >> v1:172.21.15.154:6789/0 conn(0x7f5df00701e0 legacy=0x7f5df0161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 -- 172.21.15.154:0/3048044076 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0072520 con 0x7f5df0161d70 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.164+0000 7fe4673e8700 1 -- start start 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe4673e8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe460072410 0x7fe460159750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe4673e8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4600717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe4673e8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe460072200 con 0x7fe460156080 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe4673e8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe460158cb0 con 0x7fe460072410 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe4673e8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe460156530 con 0x7fe460071320 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe464983700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fe460156080 0x7fe4600c0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38396/0 (socket says 172.21.15.154:38396) 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe464983700 1 -- 172.21.15.154:0/3493990048 learned_addr learned my addr 172.21.15.154:0/3493990048 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.165+0000 7fe465985700 1 --2- 172.21.15.154:0/3493990048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4600717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465184700 1 --2- 172.21.15.154:0/3493990048 >> v2:172.21.15.191:3300/0 conn(0x7fe460072410 0x7fe460159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465985700 1 -- 172.21.15.154:0/3493990048 >> v2:172.21.15.191:3300/0 conn(0x7fe460072410 msgr2=0x7fe460159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465985700 1 --2- 172.21.15.154:0/3493990048 >> v2:172.21.15.191:3300/0 conn(0x7fe460072410 0x7fe460159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465985700 1 -- 172.21.15.154:0/3493990048 >> v1:172.21.15.191:6789/0 conn(0x7fe460156080 legacy=0x7fe4600c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465985700 1 -- 172.21.15.154:0/3493990048 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4600c0f80 con 0x7fe460071320 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe465985700 1 --2- 172.21.15.154:0/3493990048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4600717b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009840 tx=0x7fe45c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c670be690e114f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.166+0000 7fe457fff700 1 -- 172.21.15.154:0/3493990048 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c004230 con 0x7fe460071320 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7fe457fff700 1 -- 172.21.15.154:0/3493990048 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c004390 con 0x7fe460071320 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7fe457fff700 1 -- 172.21.15.154:0/3493990048 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c0036a0 con 0x7fe460071320 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 -- 172.21.15.154:0/3493990048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 msgr2=0x7fe4600717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 --2- 172.21.15.154:0/3493990048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4600717b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009840 tx=0x7fe45c009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.167+0000 7f5deed9d700 1 --2- 172.21.15.154:0/3048044076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df0161d70 0x7f5df0070660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5de0010bb0 tx=0x7f5de0010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f9abbb1b10162a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7f5ded59a700 1 -- 172.21.15.154:0/3048044076 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de000be90 con 0x7f5df0161d70 2024-03-14T09:01:27.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7f5ded59a700 1 -- 172.21.15.154:0/3048044076 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0004030 con 0x7f5df0161d70 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7f5ded59a700 1 -- 172.21.15.154:0/3048044076 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de00043a0 con 0x7f5df0161d70 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 -- 172.21.15.154:0/3493990048 shutdown_connections 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 --2- 172.21.15.154:0/3493990048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4600717b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 --2- 172.21.15.154:0/3493990048 >> v2:172.21.15.191:3300/0 conn(0x7fe460072410 0x7fe460159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.168+0000 7fe4673e8700 1 -- 172.21.15.154:0/3493990048 >> 172.21.15.154:0/3493990048 conn(0x7fe4600c5e70 msgr2=0x7fe4600c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.169+0000 7f5df4886700 1 -- 172.21.15.154:0/3048044076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df0161d70 msgr2=0x7f5df0070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.169+0000 7f5df4886700 1 --2- 172.21.15.154:0/3048044076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df0161d70 0x7f5df0070660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5de0010bb0 tx=0x7f5de0010ec0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.169+0000 7fe4673e8700 1 -- 172.21.15.154:0/3493990048 shutdown_connections 2024-03-14T09:01:27.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.169+0000 7fe4673e8700 1 -- 172.21.15.154:0/3493990048 wait complete. 2024-03-14T09:01:27.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 -- 172.21.15.154:0/3048044076 shutdown_connections 2024-03-14T09:01:27.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 --2- 172.21.15.154:0/3048044076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df0161d70 0x7f5df0070660 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 --2- 172.21.15.154:0/3048044076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 -- 172.21.15.154:0/3048044076 >> 172.21.15.154:0/3048044076 conn(0x7f5df00c2970 msgr2=0x7f5df00c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 -- 172.21.15.154:0/3048044076 shutdown_connections 2024-03-14T09:01:27.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.171+0000 7fe4673e8700 1 Processor -- start 2024-03-14T09:01:27.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.171+0000 7fe4673e8700 1 -- start start 2024-03-14T09:01:27.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.170+0000 7f5df4886700 1 -- 172.21.15.154:0/3048044076 wait complete. 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4601fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 0x7fe4601582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 0x7fe4600a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600a9630 con 0x7fe460156080 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460138410 con 0x7fe460072410 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe4673e8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe460158ad0 con 0x7fe460071320 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe464983700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 0x7fe4601582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe464983700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 0x7fe4601582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:38536/0 (socket says 172.21.15.154:38536) 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe464983700 1 -- 172.21.15.154:0/2024772830 learned_addr learned my addr 172.21.15.154:0/2024772830 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.172+0000 7fe465985700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 0x7fe4600a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7f5df4886700 1 Processor -- start 2024-03-14T09:01:27.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 -- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 msgr2=0x7fe4601fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4601fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 -- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 msgr2=0x7fe4601582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 0x7fe4601582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 -- 172.21.15.154:0/2024772830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0094a0 con 0x7fe460156080 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7fe465985700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 0x7fe4600a89d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe45c015000 tx=0x7fe45c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.173+0000 7f5df4886700 1 -- start start 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 0x7f5df015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c003c80 con 0x7fe460156080 2024-03-14T09:01:27.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c01f070 con 0x7fe460156080 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011630 con 0x7fe460156080 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df0161d70 0x7f5df0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df00aa290 con 0x7f5df00701e0 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df00a9680 con 0x7f5df0161d70 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5df4886700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df00a9960 con 0x7f5df006c1d0 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5deed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df0161d70 0x7f5df0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5dee59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5dedd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 0x7f5df015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5dedd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 0x7f5df015d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48918/0 (socket says 172.21.15.154:48918) 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7f5dedd9b700 1 -- 172.21.15.154:0/2473208182 learned_addr learned my addr 172.21.15.154:0/2473208182 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f5dee59c700 1 -- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df0161d70 msgr2=0x7f5df0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f5dee59c700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df0161d70 0x7f5df0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f5dee59c700 1 -- 172.21.15.154:0/2473208182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 msgr2=0x7f5df015d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f5dee59c700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 0x7f5df015d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f5dee59c700 1 -- 172.21.15.154:0/2473208182 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0010810 con 0x7f5df006c1d0 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4600a84d0 con 0x7fe460156080 2024-03-14T09:01:27.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.174+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4600c2ee0 con 0x7fe460156080 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.176+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4400052f0 con 0x7fe460156080 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f81fbf4c700 1 Processor -- start 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.175+0000 7f81fbf4c700 1 -- start start 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.176+0000 7f81fbf4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4156440 0x7f81f41593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.177+0000 7f81fbf4c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f415d160 con 0x7f81f4159910 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.177+0000 7f81fbf4c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f406e530 con 0x7f81f415cbf0 2024-03-14T09:01:27.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.177+0000 7f81fbf4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4158bf0 con 0x7f81f4156440 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.177+0000 7f81fa4e9700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f81f415cbf0 0x7f81f4163570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38406/0 (socket says 172.21.15.154:38406) 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.177+0000 7f81fa4e9700 1 -- 172.21.15.154:0/998437691 learned_addr learned my addr 172.21.15.154:0/998437691 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7f81f9ce8700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4156440 0x7f81f41593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe45c021030 con 0x7fe460156080 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7fe455ffb700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 0x7fe44406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe45c08a420 con 0x7fe460156080 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557689430 0 0) 0x7f81f406e530 con 0x7f81f415cbf0 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e00035b0 con 0x7f81f415cbf0 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925337151 0 0) 0x7f81e00035b0 con 0x7f81f415cbf0 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4156440 msgr2=0x7f81f41593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4156440 0x7f81f41593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 >> v1:172.21.15.154:6789/0 conn(0x7f81f4159910 legacy=0x7f81f40c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4071f20 con 0x7f81f415cbf0 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f81f0001670 con 0x7f81f415cbf0 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 >> v1:172.21.15.191:6789/0 conn(0x7f81f415cbf0 legacy=0x7f81f4163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0003a00 0x7f81e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81e0004850 0x7f81e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e0005210 0x7f81e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4158bf0 con 0x7f81e0005210 2024-03-14T09:01:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e000c110 con 0x7f81e0003a00 2024-03-14T09:01:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e000c2d0 con 0x7f81e0004850 2024-03-14T09:01:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7fe465184700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 0x7fe44406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81e0004850 0x7f81e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 -- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0003a00 msgr2=0x7f81e0003e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0003a00 0x7f81e0003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 -- 172.21.15.154:0/998437691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e0005210 msgr2=0x7f81e000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81e0005210 0x7f81e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 -- 172.21.15.154:0/998437691 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4071f20 con 0x7f81e0004850 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f94e7700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81e0004850 0x7f81e0004cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f81e40016c0 tx=0x7f81e0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d73f9529c352d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7fe465184700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 0x7fe44406cdc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097f0 tx=0x7fe450008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.180+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40052d0 con 0x7f81e0004850 2024-03-14T09:01:27.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.178+0000 7f5dee59c700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ad40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5de800dc50 tx=0x7f5de800df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800bb70 con 0x7f5df006c1d0 2024-03-14T09:01:27.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800bcd0 con 0x7f5df006c1d0 2024-03-14T09:01:27.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.179+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8012660 con 0x7f5df006c1d0 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.181+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4005450 con 0x7f81e0004850 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.181+0000 7f81f8ce6700 1 -- 172.21.15.154:0/998437691 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40038e0 con 0x7f81e0004850 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.181+0000 7f81fbf4c700 1 -- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81e0004850 msgr2=0x7f81e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.181+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/998437691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81e0004850 0x7f81e0004cd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f81e40016c0 tx=0x7f81e0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.181+0000 7f81fa4e9700 1 -- 172.21.15.154:0/998437691 reap_dead start 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.182+0000 7f81fbf4c700 1 -- 172.21.15.154:0/998437691 shutdown_connections 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.182+0000 7f81fbf4c700 1 -- 172.21.15.154:0/998437691 >> 172.21.15.154:0/998437691 conn(0x7f81f40c2970 msgr2=0x7f81f40c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.182+0000 7f81fbf4c700 1 -- 172.21.15.154:0/998437691 shutdown_connections 2024-03-14T09:01:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.182+0000 7f81fbf4c700 1 -- 172.21.15.154:0/998437691 wait complete. 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.183+0000 7f81fbf4c700 1 Processor -- start 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.183+0000 7f81fbf4c700 1 -- start start 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0004850 0x7f81f41fb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4154a40 0x7f81f414ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f41519c0 con 0x7f81f4151100 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40c5b60 con 0x7f81e0004850 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81fbf4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f40c55a0 con 0x7f81f4154a40 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81f94e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81f94e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48922/0 (socket says 172.21.15.154:48922) 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.184+0000 7f81f94e7700 1 -- 172.21.15.154:0/953729080 learned_addr learned my addr 172.21.15.154:0/953729080 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 -- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4154a40 msgr2=0x7f81f414ba60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4154a40 0x7f81f414ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 -- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0004850 msgr2=0x7f81f41fb010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:27.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0004850 0x7f81f41fb010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 -- 172.21.15.154:0/953729080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40c5880 con 0x7f81f4151100 2024-03-14T09:01:27.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f81f94e7700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c460 tx=0x7f81e4000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df00a9c40 con 0x7f5df006c1d0 2024-03-14T09:01:27.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe45c055a10 con 0x7fe460156080 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.185+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df0162900 con 0x7f5df006c1d0 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df004ea80 con 0x7f5df006c1d0 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e401e600 con 0x7f81f4151100 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e40052e0 con 0x7f81f4151100 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e401e9a0 con 0x7f81f4151100 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f4153d00 con 0x7f81f4151100 2024-03-14T09:01:27.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.187+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f4151580 con 0x7f81f4151100 2024-03-14T09:01:27.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.190+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5de8010070 con 0x7f5df006c1d0 2024-03-14T09:01:27.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.189+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f81e4019030 con 0x7f81f4151100 2024-03-14T09:01:27.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.190+0000 7f81ea7fc700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 0x7f81e006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.190+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81e408a200 con 0x7f81f4151100 2024-03-14T09:01:27.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.190+0000 7f5ddeffd700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 0x7f5dd006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.190+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de80896c0 con 0x7f5df006c1d0 2024-03-14T09:01:27.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.191+0000 7f5dedd9b700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 0x7f5dd006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.191+0000 7f5dedd9b700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 0x7f5dd006cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5df008ab40 tx=0x7f5de4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.191+0000 7f81f9ce8700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 0x7f81e006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.192+0000 7f81f9ce8700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 0x7f81e006e7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81f40c3780 tx=0x7f81ec00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.194+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5de8054ac0 con 0x7f5df006c1d0 2024-03-14T09:01:27.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.206+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc0052f0 con 0x7f81f4151100 2024-03-14T09:01:27.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.216+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f81e4055730 con 0x7f81f4151100 2024-03-14T09:01:27.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.217+0000 7f148bdce700 1 Processor -- start 2024-03-14T09:01:27.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.217+0000 7f148bdce700 1 -- start start 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.218+0000 7f148bdce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14840708f0 0x7f148415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.218+0000 7f148bdce700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1484161660 0x7f1484161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.218+0000 7f148bdce700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1484071030 con 0x7f148406c1d0 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.218+0000 7f148bdce700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148406e0d0 con 0x7f1484161660 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.218+0000 7f148bdce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148406c5b0 con 0x7f14840708f0 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.219+0000 7f1489369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14840708f0 0x7f148415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.219+0000 7f148a36b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1484161660 0x7f1484161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.219+0000 7f1489369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14840708f0 0x7f148415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46190/0 (socket says 172.21.15.154:46190) 2024-03-14T09:01:27.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.219+0000 7f1489369700 1 -- 172.21.15.154:0/319828963 learned_addr learned my addr 172.21.15.154:0/319828963 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.220+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089143174 0 0) 0x7f1484071030 con 0x7f148406c1d0 2024-03-14T09:01:27.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14700034b0 con 0x7f148406c1d0 2024-03-14T09:01:27.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735414264 0 0) 0x7f14700034b0 con 0x7f148406c1d0 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14840708f0 msgr2=0x7f148415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14840708f0 0x7f148415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 >> v2:172.21.15.191:3300/0 conn(0x7f1484161660 msgr2=0x7f1484161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 --2- 172.21.15.154:0/319828963 >> v2:172.21.15.191:3300/0 conn(0x7f1484161660 0x7f1484161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.226+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484072520 con 0x7f148406c1d0 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f147c0019f0 con 0x7f148406c1d0 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 >> v1:172.21.15.154:6789/0 conn(0x7f148406c1d0 legacy=0x7f14840701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1470004ae0 0x7f1470004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14700036f0 0x7f1470003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470004220 0x7f147001c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148406e0d0 con 0x7f14700036f0 2024-03-14T09:01:27.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148406c5b0 con 0x7f1470004220 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147001d0c0 con 0x7f1470004ae0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1489b6a700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1470004ae0 0x7f1470004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.227+0000 7f1489369700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14700036f0 0x7f1470003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 -- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1470004ae0 msgr2=0x7f1470004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1470004ae0 0x7f1470004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 -- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470004220 msgr2=0x7f147001c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470004220 0x7f147001c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 -- 172.21.15.154:0/319828963 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484072520 con 0x7f14700036f0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.228+0000 7f1489369700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14700036f0 0x7f1470003b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f148406bf00 tx=0x7f1470004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae24e2e9b6c9d3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.229+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148000f040 con 0x7f14700036f0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.229+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800132d0 con 0x7f14700036f0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.229+0000 7f1488b68700 1 -- 172.21.15.154:0/319828963 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480013660 con 0x7f14700036f0 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.229+0000 7f148bdce700 1 -- 172.21.15.154:0/319828963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14700036f0 msgr2=0x7f1470003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.230+0000 7f148bdce700 1 --2- 172.21.15.154:0/319828963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14700036f0 0x7f1470003b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f148406bf00 tx=0x7f1470004770 comp rx=0 tx=0).stop 2024-03-14T09:01:27.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.230+0000 7f148a36b700 1 -- 172.21.15.154:0/319828963 reap_dead start 2024-03-14T09:01:27.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.231+0000 7f148bdce700 1 -- 172.21.15.154:0/319828963 shutdown_connections 2024-03-14T09:01:27.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.231+0000 7f148bdce700 1 -- 172.21.15.154:0/319828963 >> 172.21.15.154:0/319828963 conn(0x7f14840c2970 msgr2=0x7f14840c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.231+0000 7f148bdce700 1 -- 172.21.15.154:0/319828963 shutdown_connections 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.231+0000 7f148bdce700 1 -- 172.21.15.154:0/319828963 wait complete. 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.233+0000 7f148bdce700 1 Processor -- start 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.233+0000 7f148bdce700 1 -- start start 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14700036f0 0x7f14840c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14841f7990 0x7f14840c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841fb1b0 con 0x7f14840a8970 2024-03-14T09:01:27.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484164560 con 0x7f14841f7990 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f148bdce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1484164840 con 0x7f14700036f0 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f1489369700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f1489369700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48940/0 (socket says 172.21.15.154:48940) 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.234+0000 7f1489369700 1 -- 172.21.15.154:0/1840161026 learned_addr learned my addr 172.21.15.154:0/1840161026 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f148a36b700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14841f7990 0x7f14840c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 -- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14700036f0 msgr2=0x7f14840c2fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14700036f0 0x7f14840c2fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 -- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14841f7990 msgr2=0x7f14840c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14841f7990 0x7f14840c3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 -- 172.21.15.154:0/1840161026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840c3500 con 0x7f14840a8970 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.235+0000 7f1489369700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f147001d010 tx=0x7f14800082b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.236+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148000f050 con 0x7f14840a8970 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.236+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480004220 con 0x7f14840a8970 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.236+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480013830 con 0x7f14840a8970 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.236+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1484164b20 con 0x7f14840a8970 2024-03-14T09:01:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.236+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14840c4ec0 con 0x7f14840a8970 2024-03-14T09:01:27.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.238+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0052f0 con 0x7f14840a8970 2024-03-14T09:01:27.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.239+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f14800038b0 con 0x7f14840a8970 2024-03-14T09:01:27.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.240+0000 7f147a7fc700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 0x7f146806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.240+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1480089fd0 con 0x7f14840a8970 2024-03-14T09:01:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.244+0000 7f1489b6a700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 0x7f146806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.247+0000 7f1489b6a700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 0x7f146806cdc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f147c001330 tx=0x7f147c00a060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.247+0000 7f2427ac5700 1 Processor -- start 2024-03-14T09:01:27.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.247+0000 7f2427ac5700 1 -- start start 2024-03-14T09:01:27.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.248+0000 7f2427ac5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24200aac00 0x7f24200ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.248+0000 7f2427ac5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f24200701e0 0x7f2420070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.248+0000 7f2427ac5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242006c8e0 con 0x7f24200723b0 2024-03-14T09:01:27.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.248+0000 7f2427ac5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2420072790 con 0x7f24200aac00 2024-03-14T09:01:27.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.248+0000 7f2427ac5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242006cbc0 con 0x7f24200701e0 2024-03-14T09:01:27.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1480055510 con 0x7f14840a8970 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.249+0000 7f2425060700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24200aac00 0x7f24200ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.249+0000 7f2425060700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24200aac00 0x7f24200ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48950/0 (socket says 172.21.15.154:48950) 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.249+0000 7f2425060700 1 -- 172.21.15.154:0/1626286424 learned_addr learned my addr 172.21.15.154:0/1626286424 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f2425060700 1 -- 172.21.15.154:0/1626286424 >> v2:172.21.15.191:3300/0 conn(0x7f24200701e0 msgr2=0x7f2420070610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f2425060700 1 --2- 172.21.15.154:0/1626286424 >> v2:172.21.15.191:3300/0 conn(0x7f24200701e0 0x7f2420070610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f2425060700 1 -- 172.21.15.154:0/1626286424 >> v1:172.21.15.154:6789/0 conn(0x7f24200723b0 legacy=0x7f24200c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f2425060700 1 -- 172.21.15.154:0/1626286424 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242006cea0 con 0x7f24200aac00 2024-03-14T09:01:27.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.250+0000 7f2425060700 1 --2- 172.21.15.154:0/1626286424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200aac00 0x7f24200ab060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24200c27d0 tx=0x7f2418010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff084284f2a6367 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.251+0000 7f242485f700 1 -- 172.21.15.154:0/1626286424 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418010270 con 0x7f24200aac00 2024-03-14T09:01:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.251+0000 7f242485f700 1 -- 172.21.15.154:0/1626286424 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418019a60 con 0x7f24200aac00 2024-03-14T09:01:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.251+0000 7f242485f700 1 -- 172.21.15.154:0/1626286424 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418019e10 con 0x7f24200aac00 2024-03-14T09:01:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.252+0000 7f2427ac5700 1 -- 172.21.15.154:0/1626286424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200aac00 msgr2=0x7f24200ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.252+0000 7f2427ac5700 1 --2- 172.21.15.154:0/1626286424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200aac00 0x7f24200ab060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24200c27d0 tx=0x7f2418010800 comp rx=0 tx=0).stop 2024-03-14T09:01:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.253+0000 7f2427ac5700 1 -- 172.21.15.154:0/1626286424 shutdown_connections 2024-03-14T09:01:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.253+0000 7f2427ac5700 1 --2- 172.21.15.154:0/1626286424 >> v2:172.21.15.191:3300/0 conn(0x7f24200701e0 0x7f2420070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.253+0000 7f2427ac5700 1 --2- 172.21.15.154:0/1626286424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200aac00 0x7f24200ab060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.253+0000 7f2427ac5700 1 -- 172.21.15.154:0/1626286424 >> 172.21.15.154:0/1626286424 conn(0x7f24200c2970 msgr2=0x7f24200c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.254+0000 7f2427ac5700 1 -- 172.21.15.154:0/1626286424 shutdown_connections 2024-03-14T09:01:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.254+0000 7f2427ac5700 1 -- 172.21.15.154:0/1626286424 wait complete. 2024-03-14T09:01:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.255+0000 7f2427ac5700 1 Processor -- start 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.256+0000 7f2427ac5700 1 -- start start 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24200723b0 0x7f24201e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24200aac00 0x7f24201e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201544e0 con 0x7f24200701e0 2024-03-14T09:01:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420153300 con 0x7f24200aac00 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2427ac5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24201e9eb0 con 0x7f24200723b0 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2425861700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2425861700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48966/0 (socket says 172.21.15.154:48966) 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.257+0000 7f2425861700 1 -- 172.21.15.154:0/2407570657 learned_addr learned my addr 172.21.15.154:0/2407570657 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.258+0000 7f2425861700 1 -- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24200723b0 msgr2=0x7f24201e5b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-14T09:01:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.258+0000 7f2425861700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24200723b0 0x7f24201e5b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.258+0000 7f2425861700 1 -- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24200aac00 msgr2=0x7f24201e60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-14T09:01:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.258+0000 7f2425861700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24200aac00 0x7f24201e60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.258+0000 7f2425861700 1 -- 172.21.15.154:0/2407570657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418007790 con 0x7f24200701e0 2024-03-14T09:01:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.259+0000 7f2425861700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c4e0 tx=0x7f241c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.263+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0052d0 con 0x7f24200701e0 2024-03-14T09:01:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.263+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c005430 con 0x7f24200701e0 2024-03-14T09:01:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.263+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0038c0 con 0x7f24200701e0 2024-03-14T09:01:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.263+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24201546c0 con 0x7f24200701e0 2024-03-14T09:01:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.263+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200bfe90 con 0x7f24200701e0 2024-03-14T09:01:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.265+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242004ea80 con 0x7f24200701e0 2024-03-14T09:01:27.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.267+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f241c003a20 con 0x7f24200701e0 2024-03-14T09:01:27.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.267+0000 7f24167fc700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 0x7f240406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.268+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f241c0881a0 con 0x7f24200701e0 2024-03-14T09:01:27.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.275+0000 7f2425060700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 0x7f240406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.277+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f241c053630 con 0x7f24200701e0 2024-03-14T09:01:27.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.287+0000 7f4056840700 1 Processor -- start 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.287+0000 7f4056840700 1 -- start start 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.288+0000 7f4056840700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40500aac00 0x7f40500ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.288+0000 7f4056840700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500701e0 0x7f4050070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.288+0000 7f4056840700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405006c8e0 con 0x7f40500723b0 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.288+0000 7f4056840700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4050072790 con 0x7f40500aac00 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.288+0000 7f4056840700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405006cbc0 con 0x7f40500701e0 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.289+0000 7f4054ddd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500701e0 0x7f4050070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f404f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40500aac00 0x7f40500ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f404ffff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f40500723b0 0x7f40500c1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:38416/0 (socket says 172.21.15.154:38416) 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f404ffff700 1 -- 172.21.15.154:0/2635753696 learned_addr learned my addr 172.21.15.154:0/2635753696 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393606110 0 0) 0x7f405006c8e0 con 0x7f40500723b0 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40380034d0 con 0x7f40500723b0 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.290+0000 7f2425060700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 0x7f240406cd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2420154490 tx=0x7f2418010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.291+0000 7f404f7fe700 1 -- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500701e0 msgr2=0x7f4050070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.291+0000 7f404f7fe700 1 --2- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500701e0 0x7f4050070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.291+0000 7f404f7fe700 1 -- 172.21.15.154:0/2635753696 >> v1:172.21.15.191:6789/0 conn(0x7f40500723b0 legacy=0x7f40500c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.291+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681846379 0 0) 0x7f40380034d0 con 0x7f40500723b0 2024-03-14T09:01:27.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.291+0000 7f404f7fe700 1 -- 172.21.15.154:0/2635753696 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405006cea0 con 0x7f40500aac00 2024-03-14T09:01:27.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.302+0000 7f404f7fe700 1 --2- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500aac00 0x7f40500ab060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4040009840 tx=0x7f4040009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64383d41bf9be87a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.306+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004d60 con 0x7f40500aac00 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.306+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040004030 con 0x7f40500aac00 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.306+0000 7f404effd700 1 -- 172.21.15.154:0/2635753696 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400043c0 con 0x7f40500aac00 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 -- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500aac00 msgr2=0x7f40500ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 --2- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500aac00 0x7f40500ab060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4040009840 tx=0x7f4040009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 -- 172.21.15.154:0/2635753696 shutdown_connections 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 --2- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500701e0 0x7f4050070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 --2- 172.21.15.154:0/2635753696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500aac00 0x7f40500ab060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 -- 172.21.15.154:0/2635753696 >> 172.21.15.154:0/2635753696 conn(0x7f40500c2970 msgr2=0x7f40500c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 -- 172.21.15.154:0/2635753696 shutdown_connections 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.307+0000 7f4056840700 1 -- 172.21.15.154:0/2635753696 wait complete. 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.309+0000 7f4056840700 1 Processor -- start 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.310+0000 7f4056840700 1 -- start start 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500701e0 0x7f40500a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40500723b0 0x7f40501f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500bfaf0 con 0x7f40500723b0 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500bea40 con 0x7f40500701e0 2024-03-14T09:01:27.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4056840700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40501f4860 con 0x7f40500aac00 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4054ddd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f404ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500701e0 0x7f40500a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4054ddd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:46232/0 (socket says 172.21.15.154:46232) 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.311+0000 7f4054ddd700 1 -- 172.21.15.154:0/390023870 learned_addr learned my addr 172.21.15.154:0/390023870 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f404f7fe700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40500723b0 0x7f40501f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 -- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500701e0 msgr2=0x7f40500a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500701e0 0x7f40500a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 -- 172.21.15.154:0/390023870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40500723b0 msgr2=0x7f40501f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40500723b0 0x7f40501f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 -- 172.21.15.154:0/390023870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40400094a0 con 0x7f40500aac00 2024-03-14T09:01:27.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.312+0000 7f4054ddd700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4048007e40 tx=0x7f404800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.313+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800ede0 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.314+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048014420 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.314+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480147d0 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.314+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40501fb010 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.314+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501fa530 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.317+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4048014930 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.318+0000 7f404cff9700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 0x7f403c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.318+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4048089400 con 0x7f40500aac00 2024-03-14T09:01:27.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.318+0000 7f404ffff700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 0x7f403c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.319+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40300052f0 con 0x7f40500aac00 2024-03-14T09:01:27.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.319+0000 7f404ffff700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 0x7f403c06cdf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40501514c0 tx=0x7f4044008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.324+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4048054800 con 0x7f40500aac00 2024-03-14T09:01:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.503+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc0800697d0 con 0x7fc08015c270 2024-03-14T09:01:27.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.503+0000 7fc07d7fa700 1 -- 172.21.15.154:0/263162810 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc078017030 con 0x7fc08015c270 2024-03-14T09:01:27.506 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.507+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 msgr2=0x7fc06c06cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.507+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 0x7fc06c06cdc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc070000f80 tx=0x7fc070011000 comp rx=0 tx=0).stop 2024-03-14T09:01:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.507+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 msgr2=0x7fc08014ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.507+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 0x7fc08014ef10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c390 tx=0x7fc07800c6a0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 shutdown_connections 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc08015c270 0x7fc08014ef10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc06c06a900 0x7fc06c06cdc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0800ad200 0x7fc0800ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 --2- 172.21.15.154:0/263162810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0800adcc0 0x7fc080150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 >> 172.21.15.154:0/263162810 conn(0x7fc0800ca070 msgr2=0x7fc080153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 shutdown_connections 2024-03-14T09:01:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.508+0000 7fc086e96700 1 -- 172.21.15.154:0/263162810 wait complete. 2024-03-14T09:01:27.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-03-14T09:01:27.527 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.531+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6374071710 con 0x7f63740708f0 2024-03-14T09:01:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.532+0000 7f63717fa700 1 -- 172.21.15.154:0/1442520661 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f636c019020 con 0x7f63740708f0 2024-03-14T09:01:27.531 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.535+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 msgr2=0x7f636006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.535+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 0x7f636006cdc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6368006fd0 tx=0x7f6368006bc0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.535+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 msgr2=0x7f6374148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.535+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f636c017040 tx=0x7f636c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 shutdown_connections 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63740aac00 0x7f6374143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f636006a900 0x7f636006cdc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63740708f0 0x7f6374148ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 --2- 172.21.15.154:0/1442520661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63740723b0 0x7f63741438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 >> 172.21.15.154:0/1442520661 conn(0x7f63740c2970 msgr2=0x7f63740c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 shutdown_connections 2024-03-14T09:01:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.536+0000 7f637b0a7700 1 -- 172.21.15.154:0/1442520661 wait complete. 2024-03-14T09:01:27.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-14T09:01:27.560 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.588+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe440005870 con 0x7fe460156080 2024-03-14T09:01:27.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.588+0000 7fe455ffb700 1 -- 172.21.15.154:0/2024772830 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe45c058ff0 con 0x7fe460156080 2024-03-14T09:01:27.589 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 msgr2=0x7fe44406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 0x7fe44406cdc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097f0 tx=0x7fe450008040 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 msgr2=0x7fe4600a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 0x7fe4600a89d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe45c015000 tx=0x7fe45c004750 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 shutdown_connections 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe44406a900 0x7fe44406cdc0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097f0 tx=0x7fe450008040 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe460156080 0x7fe4600a89d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe460071320 0x7fe4601fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 --2- 172.21.15.154:0/2024772830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe460072410 0x7fe4601582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.592+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 >> 172.21.15.154:0/2024772830 conn(0x7fe4600c5e70 msgr2=0x7fe460154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.595+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 shutdown_connections 2024-03-14T09:01:27.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.595+0000 7fe4673e8700 1 -- 172.21.15.154:0/2024772830 wait complete. 2024-03-14T09:01:27.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.607+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5df00697d0 con 0x7f5df006c1d0 2024-03-14T09:01:27.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.608+0000 7f5ddeffd700 1 -- 172.21.15.154:0/2473208182 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5de80580e0 con 0x7f5df006c1d0 2024-03-14T09:01:27.607 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.611+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 msgr2=0x7f5dd006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.611+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 0x7f5dd006cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5df008ab40 tx=0x7f5de4009040 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.611+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 msgr2=0x7f5df015ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.611+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ad40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5de800dc50 tx=0x7f5de800df60 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 shutdown_connections 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5df0161d70 0x7f5df0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5df006c1d0 0x7f5df015ad40 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5de800dc50 tx=0x7f5de800df60 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dd006a900 0x7f5dd006cdc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 --2- 172.21.15.154:0/2473208182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5df00701e0 0x7f5df015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.612+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 >> 172.21.15.154:0/2473208182 conn(0x7f5df00c2970 msgr2=0x7f5df00c0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-03-14T09:01:27.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.614+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 shutdown_connections 2024-03-14T09:01:27.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.614+0000 7f5df4886700 1 -- 172.21.15.154:0/2473208182 wait complete. 2024-03-14T09:01:27.614 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.619+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81dc0055a0 con 0x7f81f4151100 2024-03-14T09:01:27.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.619+0000 7f81ea7fc700 1 -- 172.21.15.154:0/953729080 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f81e4017090 con 0x7f81f4151100 2024-03-14T09:01:27.619 INFO:teuthology.orchestra.run.smithi154.stdout:55834574854 2024-03-14T09:01:27.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.622+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 msgr2=0x7f81e006e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.622+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 0x7f81e006e7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81f40c3780 tx=0x7f81ec00b040 comp rx=0 tx=0).stop 2024-03-14T09:01:27.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.622+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 msgr2=0x7f81f4154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.622+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c460 tx=0x7f81e4000fb0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 shutdown_connections 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81f4154a40 0x7f81f414ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81e0004850 0x7f81f41fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e006c320 0x7f81e006e7e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 --2- 172.21.15.154:0/953729080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81f4151100 0x7f81f4154500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 >> 172.21.15.154:0/953729080 conn(0x7f81f40c2970 msgr2=0x7f81f40c17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 shutdown_connections 2024-03-14T09:01:27.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.623+0000 7f81fbf4c700 1 -- 172.21.15.154:0/953729080 wait complete. 2024-03-14T09:01:27.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.629+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f24200697d0 con 0x7f24200701e0 2024-03-14T09:01:27.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.629+0000 7f24167fc700 1 -- 172.21.15.154:0/2407570657 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f241c056c50 con 0x7f24200701e0 2024-03-14T09:01:27.630 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.632+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 msgr2=0x7f240406cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.632+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 0x7f240406cd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2420154490 tx=0x7f2418010000 comp rx=0 tx=0).stop 2024-03-14T09:01:27.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.632+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 msgr2=0x7f242014ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.632+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c4e0 tx=0x7f241c00c8a0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-14T09:01:27.633 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 shutdown_connections 2024-03-14T09:01:27.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24200aac00 0x7f24201e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240406a880 0x7f240406cd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24200701e0 0x7f242014ff80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 --2- 172.21.15.154:0/2407570657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24200723b0 0x7f24201e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 >> 172.21.15.154:0/2407570657 conn(0x7f24200c2970 msgr2=0x7f24200a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 shutdown_connections 2024-03-14T09:01:27.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.633+0000 7f2427ac5700 1 -- 172.21.15.154:0/2407570657 wait complete. 2024-03-14T09:01:27.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-14T09:01:27.638 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.639+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f40300055a0 con 0x7f40500aac00 2024-03-14T09:01:27.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.640+0000 7f404cff9700 1 -- 172.21.15.154:0/390023870 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4048057e20 con 0x7f40500aac00 2024-03-14T09:01:27.639 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 msgr2=0x7f403c06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 0x7f403c06cdf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40501514c0 tx=0x7f4044008040 comp rx=0 tx=0).stop 2024-03-14T09:01:27.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 msgr2=0x7f40501f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4048007e40 tx=0x7f404800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 shutdown_connections 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40500aac00 0x7f40501f4320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40500701e0 0x7f40500a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f403c06a930 0x7f403c06cdf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 --2- 172.21.15.154:0/390023870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40500723b0 0x7f40501f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 >> 172.21.15.154:0/390023870 conn(0x7f40500c2970 msgr2=0x7f405014dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 shutdown_connections 2024-03-14T09:01:27.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.642+0000 7f4056840700 1 -- 172.21.15.154:0/390023870 wait complete. 2024-03-14T09:01:27.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-03-14T09:01:27.646 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.648+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f146c005870 con 0x7f14840a8970 2024-03-14T09:01:27.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.649+0000 7f147a7fc700 1 -- 172.21.15.154:0/1840161026 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f148002c090 con 0x7f14840a8970 2024-03-14T09:01:27.649 INFO:teuthology.orchestra.run.smithi154.stdout:55834574855 2024-03-14T09:01:27.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 msgr2=0x7f146806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 0x7f146806cdc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f147c001330 tx=0x7f147c00a060 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 msgr2=0x7f14841f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f147001d010 tx=0x7f14800082b0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 shutdown_connections 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14841f7990 0x7f14840c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14700036f0 0x7f14840c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f146806a900 0x7f146806cdc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 --2- 172.21.15.154:0/1840161026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14840a8970 0x7f14841f7450 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 >> 172.21.15.154:0/1840161026 conn(0x7f14840c2970 msgr2=0x7f14840c0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 shutdown_connections 2024-03-14T09:01:27.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.651+0000 7f148bdce700 1 -- 172.21.15.154:0/1840161026 wait complete. 2024-03-14T09:01:27.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-14T09:01:27.656 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-03-14T09:01:27.663 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:27.664 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T09:01:27.664 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:01:27.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.825+0000 7f69fae75700 1 Processor -- start 2024-03-14T09:01:27.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.825+0000 7f69fae75700 1 -- start start 2024-03-14T09:01:27.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69fae75700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f69f40c3250 0x7f69f413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69fae75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69f40c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69fae75700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f413ade0 con 0x7f69f40a8540 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69fae75700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f406fb30 con 0x7f69f40c3250 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69fae75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f406ec30 con 0x7f69f40c1780 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f8c11700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f69f40a8540 0x7f69f4154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34602/0 (socket says 172.21.15.154:34602) 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f8c11700 1 -- 172.21.15.154:0/1511958934 learned_addr learned my addr 172.21.15.154:0/1511958934 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019981524 0 0) 0x7f69e8001110 con 0x7f69f40a8540 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e40035b0 con 0x7f69f40a8540 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f3fff700 1 --2- 172.21.15.154:0/1511958934 >> v2:172.21.15.191:3300/0 conn(0x7f69f40c3250 0x7f69f413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f9412700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69f40c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1448020631 0 0) 0x7f69e40035b0 con 0x7f69f40a8540 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 msgr2=0x7f69f40c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69f40c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 >> v2:172.21.15.191:3300/0 conn(0x7f69f40c3250 msgr2=0x7f69f413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 --2- 172.21.15.154:0/1511958934 >> v2:172.21.15.191:3300/0 conn(0x7f69f40c3250 0x7f69f413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f406ef10 con 0x7f69f40a8540 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f69e8001800 con 0x7f69f40a8540 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 >> v1:172.21.15.154:6789/0 conn(0x7f69f40a8540 legacy=0x7f69f4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4003a00 0x7f69e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e4004850 0x7f69e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f406fb30 con 0x7f69e4004850 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f406ec30 con 0x7f69e4003a00 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e400bac0 con 0x7f69f40c1780 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f3fff700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e4004850 0x7f69e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.827+0000 7f69f8c11700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4003a00 0x7f69e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f9412700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 -- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 msgr2=0x7f69e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 -- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4003a00 msgr2=0x7f69e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4003a00 0x7f69e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 -- 172.21.15.154:0/1511958934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40035b0 con 0x7f69e4004850 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f9412700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f40c1780 0x7f69e4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f8c11700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4003a00 0x7f69e4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f3fff700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e4004850 0x7f69e4004cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009b60 tx=0x7f69e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f7fcded816f29c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000e6c0 con 0x7f69e4004850 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e000e820 con 0x7f69e4004850 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f37fe700 1 -- 172.21.15.154:0/1511958934 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000eb90 con 0x7f69e4004850 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69fae75700 1 -- 172.21.15.154:0/1511958934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e4004850 msgr2=0x7f69e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69fae75700 1 --2- 172.21.15.154:0/1511958934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69e4004850 0x7f69e4004cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009b60 tx=0x7f69e4005880 comp rx=0 tx=0).stop 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69f9412700 1 -- 172.21.15.154:0/1511958934 reap_dead start 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69fae75700 1 -- 172.21.15.154:0/1511958934 shutdown_connections 2024-03-14T09:01:27.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.828+0000 7f69fae75700 1 -- 172.21.15.154:0/1511958934 >> 172.21.15.154:0/1511958934 conn(0x7f69f40c3880 msgr2=0x7f69f40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.829+0000 7f69fae75700 1 -- 172.21.15.154:0/1511958934 shutdown_connections 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.829+0000 7f69fae75700 1 -- 172.21.15.154:0/1511958934 wait complete. 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 Processor -- start 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 -- start start 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 0x7f69f414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f41339e0 0x7f69f41e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41dad00 con 0x7f69f4151830 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40a9d20 con 0x7f69e4004850 2024-03-14T09:01:27.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.830+0000 7f69fae75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40aa000 con 0x7f69f41339e0 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48992/0 (socket says 172.21.15.154:48992) 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 -- 172.21.15.154:0/2585383516 learned_addr learned my addr 172.21.15.154:0/2585383516 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f8c11700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 0x7f69f414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f9412700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f41339e0 0x7f69f41e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 -- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f41339e0 msgr2=0x7f69f41e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f41339e0 0x7f69f41e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 -- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 msgr2=0x7f69f414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 0x7f69f414e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 -- 172.21.15.154:0/2585383516 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f414ec90 con 0x7f69f4151830 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f8c11700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 0x7f69f414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f3fff700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e4b0 tx=0x7f69e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000e6c0 con 0x7f69f4151830 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0003ab0 con 0x7f69f4151830 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003e60 con 0x7f69f4151830 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f4133e60 con 0x7f69f4151830 2024-03-14T09:01:27.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.831+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f41ec3a0 con 0x7f69f4151830 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.833+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f4155200 con 0x7f69f4151830 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.833+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f69e0013470 con 0x7f69f4151830 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.833+0000 7f69f17fa700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 0x7f69d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.833+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69e008a660 con 0x7f69f4151830 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.833+0000 7f69f8c11700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 0x7f69d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.834+0000 7f69f8c11700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 0x7f69d806ccf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001390 tx=0x7f69e8001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:27.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:27.838+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f69e0055c30 con 0x7f69f4151830 2024-03-14T09:01:28.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.028+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f41ec590 con 0x7f69d806a830 2024-03-14T09:01:28.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.036+0000 7f69f17fa700 1 -- 172.21.15.154:0/2585383516 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264587 (secure 0 0 0) 0x7f69f41ec590 con 0x7f69d806a830 2024-03-14T09:01:28.039 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:28.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.038+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 msgr2=0x7f69d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.038+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 0x7f69d806ccf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001390 tx=0x7f69e8001e00 comp rx=0 tx=0).stop 2024-03-14T09:01:28.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.038+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 msgr2=0x7f69f41334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.038+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e4b0 tx=0x7f69e00047c0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 shutdown_connections 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f69f41339e0 0x7f69f41e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69e4004850 0x7f69f414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f69d806a830 0x7f69d806ccf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 --2- 172.21.15.154:0/2585383516 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69f4151830 0x7f69f41334a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 >> 172.21.15.154:0/2585383516 conn(0x7f69f40c3880 msgr2=0x7f69f406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 shutdown_connections 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.039+0000 7f69fae75700 1 -- 172.21.15.154:0/2585383516 wait complete. 2024-03-14T09:01:28.042 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:01:28.050 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-14T09:01:27.519287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.409468"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745804+0000","last_change":"2024-03-14T09:01:19.140304+0000","last_active":"2024-03-14T09:01:20.745804+0000","last_peered":"2024-03-14T09:01:20.745804+0000","last_clean":"2024-03-14T09:01:20.745804+0000","last_became_active":"2024-03-14T09:01:19.140051+0000","last_became_peered":"2024-03-14T09:01:19.140051+0000","last_unstale":"2024-03-14T09:01:20.745804+0000","last_undegraded":"2024-03-14T09:01:20.745804+0000","last_fullsized":"2024-03-14T09:01:20.745804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744549+0000","last_change":"2024-03-14T09:01:19.119652+0000","last_active":"2024-03-14T09:01:20.744549+0000","last_peered":"2024-03-14T09:01:20.744549+0000","last_clean":"2024-03-14T09:01:20.744549+0000","last_became_active":"2024-03-14T09:01:19.119408+0000","last_became_peered":"2024-03-14T09:01:19.119408+0000","last_unstale":"2024-03-14T09:01:20.744549+0000","last_undegraded":"2024-03-14T09:01:20.744549+0000","last_fullsized":"2024-03-14T09:01:20.744549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119387+0000","last_change":"2024-03-14T09:01:19.122167+0000","last_active":"2024-03-14T09:01:20.119387+0000","last_peered":"2024-03-14T09:01:20.119387+0000","last_clean":"2024-03-14T09:01:20.119387+0000","last_became_active":"2024-03-14T09:01:19.121201+0000","last_became_peered":"2024-03-14T09:01:19.121201+0000","last_unstale":"2024-03-14T09:01:20.119387+0000","last_undegraded":"2024-03-14T09:01:20.119387+0000","last_fullsized":"2024-03-14T09:01:20.119387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746011+0000","last_change":"2024-03-14T09:01:19.129747+0000","last_active":"2024-03-14T09:01:20.746011+0000","last_peered":"2024-03-14T09:01:20.746011+0000","last_clean":"2024-03-14T09:01:20.746011+0000","last_became_active":"2024-03-14T09:01:19.129254+0000","last_became_peered":"2024-03-14T09:01:19.129254+0000","last_unstale":"2024-03-14T09:01:20.746011+0000","last_undegraded":"2024-03-14T09:01:20.746011+0000","last_fullsized":"2024-03-14T09:01:20.746011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745535+0000","last_change":"2024-03-14T09:01:19.138324+0000","last_active":"2024-03-14T09:01:20.745535+0000","last_peered":"2024-03-14T09:01:20.745535+0000","last_clean":"2024-03-14T09:01:20.745535+0000","last_became_active":"2024-03-14T09:01:19.138018+0000","last_became_peered":"2024-03-14T09:01:19.138018+0000","last_unstale":"2024-03-14T09:01:20.745535+0000","last_undegraded":"2024-03-14T09:01:20.745535+0000","last_fullsized":"2024-03-14T09:01:20.745535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:51.127253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744951+0000","last_change":"2024-03-14T09:01:19.129902+0000","last_active":"2024-03-14T09:01:20.744951+0000","last_peered":"2024-03-14T09:01:20.744951+0000","last_clean":"2024-03-14T09:01:20.744951+0000","last_became_active":"2024-03-14T09:01:19.128311+0000","last_became_peered":"2024-03-14T09:01:19.128311+0000","last_unstale":"2024-03-14T09:01:20.744951+0000","last_undegraded":"2024-03-14T09:01:20.744951+0000","last_fullsized":"2024-03-14T09:01:20.744951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745141+0000","last_change":"2024-03-14T09:01:19.136657+0000","last_active":"2024-03-14T09:01:20.745141+0000","last_peered":"2024-03-14T09:01:20.745141+0000","last_clean":"2024-03-14T09:01:20.745141+0000","last_became_active":"2024-03-14T09:01:19.136253+0000","last_became_peered":"2024-03-14T09:01:19.136253+0000","last_unstale":"2024-03-14T09:01:20.745141+0000","last_undegraded":"2024-03-14T09:01:20.745141+0000","last_fullsized":"2024-03-14T09:01:20.745141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122779+0000","last_change":"2024-03-14T09:01:19.123809+0000","last_active":"2024-03-14T09:01:20.122779+0000","last_peered":"2024-03-14T09:01:20.122779+0000","last_clean":"2024-03-14T09:01:20.122779+0000","last_became_active":"2024-03-14T09:01:19.122471+0000","last_became_peered":"2024-03-14T09:01:19.122471+0000","last_unstale":"2024-03-14T09:01:20.122779+0000","last_undegraded":"2024-03-14T09:01:20.122779+0000","last_fullsized":"2024-03-14T09:01:20.122779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745747+0000","last_change":"2024-03-14T09:01:19.142452+0000","last_active":"2024-03-14T09:01:20.745747+0000","last_peered":"2024-03-14T09:01:20.745747+0000","last_clean":"2024-03-14T09:01:20.745747+0000","last_became_active":"2024-03-14T09:01:19.142164+0000","last_became_peered":"2024-03-14T09:01:19.142164+0000","last_unstale":"2024-03-14T09:01:20.745747+0000","last_undegraded":"2024-03-14T09:01:20.745747+0000","last_fullsized":"2024-03-14T09:01:20.745747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745182+0000","last_change":"2024-03-14T09:01:19.131516+0000","last_active":"2024-03-14T09:01:20.745182+0000","last_peered":"2024-03-14T09:01:20.745182+0000","last_clean":"2024-03-14T09:01:20.745182+0000","last_became_active":"2024-03-14T09:01:19.131190+0000","last_became_peered":"2024-03-14T09:01:19.131190+0000","last_unstale":"2024-03-14T09:01:20.745182+0000","last_undegraded":"2024-03-14T09:01:20.745182+0000","last_fullsized":"2024-03-14T09:01:20.745182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745021+0000","last_change":"2024-03-14T09:01:19.137722+0000","last_active":"2024-03-14T09:01:20.745021+0000","last_peered":"2024-03-14T09:01:20.745021+0000","last_clean":"2024-03-14T09:01:20.745021+0000","last_became_active":"2024-03-14T09:01:19.137447+0000","last_became_peered":"2024-03-14T09:01:19.137447+0000","last_unstale":"2024-03-14T09:01:20.745021+0000","last_undegraded":"2024-03-14T09:01:20.745021+0000","last_fullsized":"2024-03-14T09:01:20.745021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123173+0000","last_change":"2024-03-14T09:01:19.127732+0000","last_active":"2024-03-14T09:01:20.123173+0000","last_peered":"2024-03-14T09:01:20.123173+0000","last_clean":"2024-03-14T09:01:20.123173+0000","last_became_active":"2024-03-14T09:01:19.126961+0000","last_became_peered":"2024-03-14T09:01:19.126961+0000","last_unstale":"2024-03-14T09:01:20.123173+0000","last_undegraded":"2024-03-14T09:01:20.123173+0000","last_fullsized":"2024-03-14T09:01:20.123173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122936+0000","last_change":"2024-03-14T09:01:19.131805+0000","last_active":"2024-03-14T09:01:20.122936+0000","last_peered":"2024-03-14T09:01:20.122936+0000","last_clean":"2024-03-14T09:01:20.122936+0000","last_became_active":"2024-03-14T09:01:19.131545+0000","last_became_peered":"2024-03-14T09:01:19.131545+0000","last_unstale":"2024-03-14T09:01:20.122936+0000","last_undegraded":"2024-03-14T09:01:20.122936+0000","last_fullsized":"2024-03-14T09:01:20.122936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119342+0000","last_change":"2024-03-14T09:01:19.121888+0000","last_active":"2024-03-14T09:01:20.119342+0000","last_peered":"2024-03-14T09:01:20.119342+0000","last_clean":"2024-03-14T09:01:20.119342+0000","last_became_active":"2024-03-14T09:01:19.121604+0000","last_became_peered":"2024-03-14T09:01:19.121604+0000","last_unstale":"2024-03-14T09:01:20.119342+0000","last_undegraded":"2024-03-14T09:01:20.119342+0000","last_fullsized":"2024-03-14T09:01:20.119342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743969+0000","last_change":"2024-03-14T09:01:19.125119+0000","last_active":"2024-03-14T09:01:20.743969+0000","last_peered":"2024-03-14T09:01:20.743969+0000","last_clean":"2024-03-14T09:01:20.743969+0000","last_became_active":"2024-03-14T09:01:19.124829+0000","last_became_peered":"2024-03-14T09:01:19.124829+0000","last_unstale":"2024-03-14T09:01:20.743969+0000","last_undegraded":"2024-03-14T09:01:20.743969+0000","last_fullsized":"2024-03-14T09:01:20.743969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744669+0000","last_change":"2024-03-14T09:01:19.128411+0000","last_active":"2024-03-14T09:01:20.744669+0000","last_peered":"2024-03-14T09:01:20.744669+0000","last_clean":"2024-03-14T09:01:20.744669+0000","last_became_active":"2024-03-14T09:01:19.127935+0000","last_became_peered":"2024-03-14T09:01:19.127935+0000","last_unstale":"2024-03-14T09:01:20.744669+0000","last_undegraded":"2024-03-14T09:01:20.744669+0000","last_fullsized":"2024-03-14T09:01:20.744669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745529+0000","last_change":"2024-03-14T09:01:19.139316+0000","last_active":"2024-03-14T09:01:20.745529+0000","last_peered":"2024-03-14T09:01:20.745529+0000","last_clean":"2024-03-14T09:01:20.745529+0000","last_became_active":"2024-03-14T09:01:19.139045+0000","last_became_peered":"2024-03-14T09:01:19.139045+0000","last_unstale":"2024-03-14T09:01:20.745529+0000","last_undegraded":"2024-03-14T09:01:20.745529+0000","last_fullsized":"2024-03-14T09:01:20.745529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744233+0000","last_change":"2024-03-14T09:01:19.120202+0000","last_active":"2024-03-14T09:01:20.744233+0000","last_peered":"2024-03-14T09:01:20.744233+0000","last_clean":"2024-03-14T09:01:20.744233+0000","last_became_active":"2024-03-14T09:01:19.119935+0000","last_became_peered":"2024-03-14T09:01:19.119935+0000","last_unstale":"2024-03-14T09:01:20.744233+0000","last_undegraded":"2024-03-14T09:01:20.744233+0000","last_fullsized":"2024-03-14T09:01:20.744233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746144+0000","last_change":"2024-03-14T09:01:19.139436+0000","last_active":"2024-03-14T09:01:20.746144+0000","last_peered":"2024-03-14T09:01:20.746144+0000","last_clean":"2024-03-14T09:01:20.746144+0000","last_became_active":"2024-03-14T09:01:19.139165+0000","last_became_peered":"2024-03-14T09:01:19.139165+0000","last_unstale":"2024-03-14T09:01:20.746144+0000","last_undegraded":"2024-03-14T09:01:20.746144+0000","last_fullsized":"2024-03-14T09:01:20.746144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:44:30.654286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119464+0000","last_change":"2024-03-14T09:01:18.720548+0000","last_active":"2024-03-14T09:01:20.119464+0000","last_peered":"2024-03-14T09:01:20.119464+0000","last_clean":"2024-03-14T09:01:20.119464+0000","last_became_active":"2024-03-14T09:01:18.720258+0000","last_became_peered":"2024-03-14T09:01:18.720258+0000","last_unstale":"2024-03-14T09:01:20.119464+0000","last_undegraded":"2024-03-14T09:01:20.119464+0000","last_fullsized":"2024-03-14T09:01:20.119464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123199+0000","last_change":"2024-03-14T09:01:19.131625+0000","last_active":"2024-03-14T09:01:20.123199+0000","last_peered":"2024-03-14T09:01:20.123199+0000","last_clean":"2024-03-14T09:01:20.123199+0000","last_became_active":"2024-03-14T09:01:19.131308+0000","last_became_peered":"2024-03-14T09:01:19.131308+0000","last_unstale":"2024-03-14T09:01:20.123199+0000","last_undegraded":"2024-03-14T09:01:20.123199+0000","last_fullsized":"2024-03-14T09:01:20.123199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123216+0000","last_change":"2024-03-14T09:01:18.722396+0000","last_active":"2024-03-14T09:01:20.123216+0000","last_peered":"2024-03-14T09:01:20.123216+0000","last_clean":"2024-03-14T09:01:20.123216+0000","last_became_active":"2024-03-14T09:01:18.721852+0000","last_became_peered":"2024-03-14T09:01:18.721852+0000","last_unstale":"2024-03-14T09:01:20.123216+0000","last_undegraded":"2024-03-14T09:01:20.123216+0000","last_fullsized":"2024-03-14T09:01:20.123216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119416+0000","last_change":"2024-03-14T09:01:19.127918+0000","last_active":"2024-03-14T09:01:20.119416+0000","last_peered":"2024-03-14T09:01:20.119416+0000","last_clean":"2024-03-14T09:01:20.119416+0000","last_became_active":"2024-03-14T09:01:19.127663+0000","last_became_peered":"2024-03-14T09:01:19.127663+0000","last_unstale":"2024-03-14T09:01:20.119416+0000","last_undegraded":"2024-03-14T09:01:20.119416+0000","last_fullsized":"2024-03-14T09:01:20.119416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746324+0000","last_change":"2024-03-14T09:01:18.727557+0000","last_active":"2024-03-14T09:01:20.746324+0000","last_peered":"2024-03-14T09:01:20.746324+0000","last_clean":"2024-03-14T09:01:20.746324+0000","last_became_active":"2024-03-14T09:01:18.727245+0000","last_became_peered":"2024-03-14T09:01:18.727245+0000","last_unstale":"2024-03-14T09:01:20.746324+0000","last_undegraded":"2024-03-14T09:01:20.746324+0000","last_fullsized":"2024-03-14T09:01:20.746324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745761+0000","last_change":"2024-03-14T09:01:19.138918+0000","last_active":"2024-03-14T09:01:20.745761+0000","last_peered":"2024-03-14T09:01:20.745761+0000","last_clean":"2024-03-14T09:01:20.745761+0000","last_became_active":"2024-03-14T09:01:19.137549+0000","last_became_peered":"2024-03-14T09:01:19.137549+0000","last_unstale":"2024-03-14T09:01:20.745761+0000","last_undegraded":"2024-03-14T09:01:20.745761+0000","last_fullsized":"2024-03-14T09:01:20.745761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745273+0000","last_change":"2024-03-14T09:01:18.724542+0000","last_active":"2024-03-14T09:01:20.745273+0000","last_peered":"2024-03-14T09:01:20.745273+0000","last_clean":"2024-03-14T09:01:20.745273+0000","last_became_active":"2024-03-14T09:01:18.724257+0000","last_became_peered":"2024-03-14T09:01:18.724257+0000","last_unstale":"2024-03-14T09:01:20.745273+0000","last_undegraded":"2024-03-14T09:01:20.745273+0000","last_fullsized":"2024-03-14T09:01:20.745273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745073+0000","last_change":"2024-03-14T09:01:19.125917+0000","last_active":"2024-03-14T09:01:20.745073+0000","last_peered":"2024-03-14T09:01:20.745073+0000","last_clean":"2024-03-14T09:01:20.745073+0000","last_became_active":"2024-03-14T09:01:19.125654+0000","last_became_peered":"2024-03-14T09:01:19.125654+0000","last_unstale":"2024-03-14T09:01:20.745073+0000","last_undegraded":"2024-03-14T09:01:20.745073+0000","last_fullsized":"2024-03-14T09:01:20.745073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123032+0000","last_change":"2024-03-14T09:01:18.725364+0000","last_active":"2024-03-14T09:01:20.123032+0000","last_peered":"2024-03-14T09:01:20.123032+0000","last_clean":"2024-03-14T09:01:20.123032+0000","last_became_active":"2024-03-14T09:01:18.725141+0000","last_became_peered":"2024-03-14T09:01:18.725141+0000","last_unstale":"2024-03-14T09:01:20.123032+0000","last_undegraded":"2024-03-14T09:01:20.123032+0000","last_fullsized":"2024-03-14T09:01:20.123032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122963+0000","last_change":"2024-03-14T09:01:19.128226+0000","last_active":"2024-03-14T09:01:20.122963+0000","last_peered":"2024-03-14T09:01:20.122963+0000","last_clean":"2024-03-14T09:01:20.122963+0000","last_became_active":"2024-03-14T09:01:19.127601+0000","last_became_peered":"2024-03-14T09:01:19.127601+0000","last_unstale":"2024-03-14T09:01:20.122963+0000","last_undegraded":"2024-03-14T09:01:20.122963+0000","last_fullsized":"2024-03-14T09:01:20.122963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:05.796691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745089+0000","last_change":"2024-03-14T09:01:18.722204+0000","last_active":"2024-03-14T09:01:20.745089+0000","last_peered":"2024-03-14T09:01:20.745089+0000","last_clean":"2024-03-14T09:01:20.745089+0000","last_became_active":"2024-03-14T09:01:18.721901+0000","last_became_peered":"2024-03-14T09:01:18.721901+0000","last_unstale":"2024-03-14T09:01:20.745089+0000","last_undegraded":"2024-03-14T09:01:20.745089+0000","last_fullsized":"2024-03-14T09:01:20.745089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119757+0000","last_change":"2024-03-14T09:01:19.127938+0000","last_active":"2024-03-14T09:01:20.119757+0000","last_peered":"2024-03-14T09:01:20.119757+0000","last_clean":"2024-03-14T09:01:20.119757+0000","last_became_active":"2024-03-14T09:01:19.127718+0000","last_became_peered":"2024-03-14T09:01:19.127718+0000","last_unstale":"2024-03-14T09:01:20.119757+0000","last_undegraded":"2024-03-14T09:01:20.119757+0000","last_fullsized":"2024-03-14T09:01:20.119757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122920+0000","last_change":"2024-03-14T09:01:18.726440+0000","last_active":"2024-03-14T09:01:20.122920+0000","last_peered":"2024-03-14T09:01:20.122920+0000","last_clean":"2024-03-14T09:01:20.122920+0000","last_became_active":"2024-03-14T09:01:18.726143+0000","last_became_peered":"2024-03-14T09:01:18.726143+0000","last_unstale":"2024-03-14T09:01:20.122920+0000","last_undegraded":"2024-03-14T09:01:20.122920+0000","last_fullsized":"2024-03-14T09:01:20.122920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122876+0000","last_change":"2024-03-14T09:01:19.117249+0000","last_active":"2024-03-14T09:01:20.122876+0000","last_peered":"2024-03-14T09:01:20.122876+0000","last_clean":"2024-03-14T09:01:20.122876+0000","last_became_active":"2024-03-14T09:01:19.116999+0000","last_became_peered":"2024-03-14T09:01:19.116999+0000","last_unstale":"2024-03-14T09:01:20.122876+0000","last_undegraded":"2024-03-14T09:01:20.122876+0000","last_fullsized":"2024-03-14T09:01:20.122876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744666+0000","last_change":"2024-03-14T09:01:18.721268+0000","last_active":"2024-03-14T09:01:20.744666+0000","last_peered":"2024-03-14T09:01:20.744666+0000","last_clean":"2024-03-14T09:01:20.744666+0000","last_became_active":"2024-03-14T09:01:18.720984+0000","last_became_peered":"2024-03-14T09:01:18.720984+0000","last_unstale":"2024-03-14T09:01:20.744666+0000","last_undegraded":"2024-03-14T09:01:20.744666+0000","last_fullsized":"2024-03-14T09:01:20.744666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744322+0000","last_change":"2024-03-14T09:01:19.120543+0000","last_active":"2024-03-14T09:01:20.744322+0000","last_peered":"2024-03-14T09:01:20.744322+0000","last_clean":"2024-03-14T09:01:20.744322+0000","last_became_active":"2024-03-14T09:01:19.119215+0000","last_became_peered":"2024-03-14T09:01:19.119215+0000","last_unstale":"2024-03-14T09:01:20.744322+0000","last_undegraded":"2024-03-14T09:01:20.744322+0000","last_fullsized":"2024-03-14T09:01:20.744322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745938+0000","last_change":"2024-03-14T09:01:18.724696+0000","last_active":"2024-03-14T09:01:20.745938+0000","last_peered":"2024-03-14T09:01:20.745938+0000","last_clean":"2024-03-14T09:01:20.745938+0000","last_became_active":"2024-03-14T09:01:18.724393+0000","last_became_peered":"2024-03-14T09:01:18.724393+0000","last_unstale":"2024-03-14T09:01:20.745938+0000","last_undegraded":"2024-03-14T09:01:20.745938+0000","last_fullsized":"2024-03-14T09:01:20.745938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745050+0000","last_change":"2024-03-14T09:01:19.129590+0000","last_active":"2024-03-14T09:01:20.745050+0000","last_peered":"2024-03-14T09:01:20.745050+0000","last_clean":"2024-03-14T09:01:20.745050+0000","last_became_active":"2024-03-14T09:01:19.129324+0000","last_became_peered":"2024-03-14T09:01:19.129324+0000","last_unstale":"2024-03-14T09:01:20.745050+0000","last_undegraded":"2024-03-14T09:01:20.745050+0000","last_fullsized":"2024-03-14T09:01:20.745050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744921+0000","last_change":"2024-03-14T09:01:18.720316+0000","last_active":"2024-03-14T09:01:20.744921+0000","last_peered":"2024-03-14T09:01:20.744921+0000","last_clean":"2024-03-14T09:01:20.744921+0000","last_became_active":"2024-03-14T09:01:18.720030+0000","last_became_peered":"2024-03-14T09:01:18.720030+0000","last_unstale":"2024-03-14T09:01:20.744921+0000","last_undegraded":"2024-03-14T09:01:20.744921+0000","last_fullsized":"2024-03-14T09:01:20.744921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119562+0000","last_change":"2024-03-14T09:01:19.126442+0000","last_active":"2024-03-14T09:01:20.119562+0000","last_peered":"2024-03-14T09:01:20.119562+0000","last_clean":"2024-03-14T09:01:20.119562+0000","last_became_active":"2024-03-14T09:01:19.126192+0000","last_became_peered":"2024-03-14T09:01:19.126192+0000","last_unstale":"2024-03-14T09:01:20.119562+0000","last_undegraded":"2024-03-14T09:01:20.119562+0000","last_fullsized":"2024-03-14T09:01:20.119562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:57:19.311712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744536+0000","last_change":"2024-03-14T09:01:18.714712+0000","last_active":"2024-03-14T09:01:20.744536+0000","last_peered":"2024-03-14T09:01:20.744536+0000","last_clean":"2024-03-14T09:01:20.744536+0000","last_became_active":"2024-03-14T09:01:18.714304+0000","last_became_peered":"2024-03-14T09:01:18.714304+0000","last_unstale":"2024-03-14T09:01:20.744536+0000","last_undegraded":"2024-03-14T09:01:20.744536+0000","last_fullsized":"2024-03-14T09:01:20.744536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123108+0000","last_change":"2024-03-14T09:01:19.129609+0000","last_active":"2024-03-14T09:01:20.123108+0000","last_peered":"2024-03-14T09:01:20.123108+0000","last_clean":"2024-03-14T09:01:20.123108+0000","last_became_active":"2024-03-14T09:01:19.129345+0000","last_became_peered":"2024-03-14T09:01:19.129345+0000","last_unstale":"2024-03-14T09:01:20.123108+0000","last_undegraded":"2024-03-14T09:01:20.123108+0000","last_fullsized":"2024-03-14T09:01:20.123108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:57.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746528+0000","last_change":"2024-03-14T09:01:18.721291+0000","last_active":"2024-03-14T09:01:20.746528+0000","last_peered":"2024-03-14T09:01:20.746528+0000","last_clean":"2024-03-14T09:01:20.746528+0000","last_became_active":"2024-03-14T09:01:18.719243+0000","last_became_peered":"2024-03-14T09:01:18.719243+0000","last_unstale":"2024-03-14T09:01:20.746528+0000","last_undegraded":"2024-03-14T09:01:20.746528+0000","last_fullsized":"2024-03-14T09:01:20.746528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746209+0000","last_change":"2024-03-14T09:01:19.139742+0000","last_active":"2024-03-14T09:01:20.746209+0000","last_peered":"2024-03-14T09:01:20.746209+0000","last_clean":"2024-03-14T09:01:20.746209+0000","last_became_active":"2024-03-14T09:01:19.139476+0000","last_became_peered":"2024-03-14T09:01:19.139476+0000","last_unstale":"2024-03-14T09:01:20.746209+0000","last_undegraded":"2024-03-14T09:01:20.746209+0000","last_fullsized":"2024-03-14T09:01:20.746209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119687+0000","last_change":"2024-03-14T09:01:18.717256+0000","last_active":"2024-03-14T09:01:20.119687+0000","last_peered":"2024-03-14T09:01:20.119687+0000","last_clean":"2024-03-14T09:01:20.119687+0000","last_became_active":"2024-03-14T09:01:18.716989+0000","last_became_peered":"2024-03-14T09:01:18.716989+0000","last_unstale":"2024-03-14T09:01:20.119687+0000","last_undegraded":"2024-03-14T09:01:20.119687+0000","last_fullsized":"2024-03-14T09:01:20.119687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119309+0000","last_change":"2024-03-14T09:01:19.129023+0000","last_active":"2024-03-14T09:01:20.119309+0000","last_peered":"2024-03-14T09:01:20.119309+0000","last_clean":"2024-03-14T09:01:20.119309+0000","last_became_active":"2024-03-14T09:01:19.128683+0000","last_became_peered":"2024-03-14T09:01:19.128683+0000","last_unstale":"2024-03-14T09:01:20.119309+0000","last_undegraded":"2024-03-14T09:01:20.119309+0000","last_fullsized":"2024-03-14T09:01:20.119309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745348+0000","last_change":"2024-03-14T09:01:18.722021+0000","last_active":"2024-03-14T09:01:20.745348+0000","last_peered":"2024-03-14T09:01:20.745348+0000","last_clean":"2024-03-14T09:01:20.745348+0000","last_became_active":"2024-03-14T09:01:18.721794+0000","last_became_peered":"2024-03-14T09:01:18.721794+0000","last_unstale":"2024-03-14T09:01:20.745348+0000","last_undegraded":"2024-03-14T09:01:20.745348+0000","last_fullsized":"2024-03-14T09:01:20.745348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745561+0000","last_change":"2024-03-14T09:01:19.141735+0000","last_active":"2024-03-14T09:01:20.745561+0000","last_peered":"2024-03-14T09:01:20.745561+0000","last_clean":"2024-03-14T09:01:20.745561+0000","last_became_active":"2024-03-14T09:01:19.141460+0000","last_became_peered":"2024-03-14T09:01:19.141460+0000","last_unstale":"2024-03-14T09:01:20.745561+0000","last_undegraded":"2024-03-14T09:01:20.745561+0000","last_fullsized":"2024-03-14T09:01:20.745561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744488+0000","last_change":"2024-03-14T09:01:18.717974+0000","last_active":"2024-03-14T09:01:20.744488+0000","last_peered":"2024-03-14T09:01:20.744488+0000","last_clean":"2024-03-14T09:01:20.744488+0000","last_became_active":"2024-03-14T09:01:18.717677+0000","last_became_peered":"2024-03-14T09:01:18.717677+0000","last_unstale":"2024-03-14T09:01:20.744488+0000","last_undegraded":"2024-03-14T09:01:20.744488+0000","last_fullsized":"2024-03-14T09:01:20.744488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.127971+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.127683+0000","last_became_peered":"2024-03-14T09:01:19.127683+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743925+0000","last_change":"2024-03-14T09:01:18.727064+0000","last_active":"2024-03-14T09:01:20.743925+0000","last_peered":"2024-03-14T09:01:20.743925+0000","last_clean":"2024-03-14T09:01:20.743925+0000","last_became_active":"2024-03-14T09:01:18.726766+0000","last_became_peered":"2024-03-14T09:01:18.726766+0000","last_unstale":"2024-03-14T09:01:20.743925+0000","last_undegraded":"2024-03-14T09:01:20.743925+0000","last_fullsized":"2024-03-14T09:01:20.743925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745322+0000","last_change":"2024-03-14T09:01:19.136107+0000","last_active":"2024-03-14T09:01:20.745322+0000","last_peered":"2024-03-14T09:01:20.745322+0000","last_clean":"2024-03-14T09:01:20.745322+0000","last_became_active":"2024-03-14T09:01:19.135855+0000","last_became_peered":"2024-03-14T09:01:19.135855+0000","last_unstale":"2024-03-14T09:01:20.745322+0000","last_undegraded":"2024-03-14T09:01:20.745322+0000","last_fullsized":"2024-03-14T09:01:20.745322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123060+0000","last_change":"2024-03-14T09:01:18.717651+0000","last_active":"2024-03-14T09:01:20.123060+0000","last_peered":"2024-03-14T09:01:20.123060+0000","last_clean":"2024-03-14T09:01:20.123060+0000","last_became_active":"2024-03-14T09:01:18.717323+0000","last_became_peered":"2024-03-14T09:01:18.717323+0000","last_unstale":"2024-03-14T09:01:20.123060+0000","last_undegraded":"2024-03-14T09:01:20.123060+0000","last_fullsized":"2024-03-14T09:01:20.123060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:18:35.107355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745513+0000","last_change":"2024-03-14T09:01:19.141404+0000","last_active":"2024-03-14T09:01:20.745513+0000","last_peered":"2024-03-14T09:01:20.745513+0000","last_clean":"2024-03-14T09:01:20.745513+0000","last_became_active":"2024-03-14T09:01:19.141138+0000","last_became_peered":"2024-03-14T09:01:19.141138+0000","last_unstale":"2024-03-14T09:01:20.745513+0000","last_undegraded":"2024-03-14T09:01:20.745513+0000","last_fullsized":"2024-03-14T09:01:20.745513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744508+0000","last_change":"2024-03-14T09:01:18.714145+0000","last_active":"2024-03-14T09:01:20.744508+0000","last_peered":"2024-03-14T09:01:20.744508+0000","last_clean":"2024-03-14T09:01:20.744508+0000","last_became_active":"2024-03-14T09:01:18.713880+0000","last_became_peered":"2024-03-14T09:01:18.713880+0000","last_unstale":"2024-03-14T09:01:20.744508+0000","last_undegraded":"2024-03-14T09:01:20.744508+0000","last_fullsized":"2024-03-14T09:01:20.744508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745392+0000","last_change":"2024-03-14T09:01:19.120923+0000","last_active":"2024-03-14T09:01:20.745392+0000","last_peered":"2024-03-14T09:01:20.745392+0000","last_clean":"2024-03-14T09:01:20.745392+0000","last_became_active":"2024-03-14T09:01:19.120028+0000","last_became_peered":"2024-03-14T09:01:19.120028+0000","last_unstale":"2024-03-14T09:01:20.745392+0000","last_undegraded":"2024-03-14T09:01:20.745392+0000","last_fullsized":"2024-03-14T09:01:20.745392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745276+0000","last_change":"2024-03-14T09:01:18.715127+0000","last_active":"2024-03-14T09:01:20.745276+0000","last_peered":"2024-03-14T09:01:20.745276+0000","last_clean":"2024-03-14T09:01:20.745276+0000","last_became_active":"2024-03-14T09:01:18.714778+0000","last_became_peered":"2024-03-14T09:01:18.714778+0000","last_unstale":"2024-03-14T09:01:20.745276+0000","last_undegraded":"2024-03-14T09:01:20.745276+0000","last_fullsized":"2024-03-14T09:01:20.745276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122941+0000","last_change":"2024-03-14T09:01:19.130468+0000","last_active":"2024-03-14T09:01:20.122941+0000","last_peered":"2024-03-14T09:01:20.122941+0000","last_clean":"2024-03-14T09:01:20.122941+0000","last_became_active":"2024-03-14T09:01:19.130155+0000","last_became_peered":"2024-03-14T09:01:19.130155+0000","last_unstale":"2024-03-14T09:01:20.122941+0000","last_undegraded":"2024-03-14T09:01:20.122941+0000","last_fullsized":"2024-03-14T09:01:20.122941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:37.408944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744060+0000","last_change":"2024-03-14T09:01:19.118942+0000","last_active":"2024-03-14T09:01:20.744060+0000","last_peered":"2024-03-14T09:01:20.744060+0000","last_clean":"2024-03-14T09:01:20.744060+0000","last_became_active":"2024-03-14T09:01:19.118681+0000","last_became_peered":"2024-03-14T09:01:19.118681+0000","last_unstale":"2024-03-14T09:01:20.744060+0000","last_undegraded":"2024-03-14T09:01:20.744060+0000","last_fullsized":"2024-03-14T09:01:20.744060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743928+0000","last_change":"2024-03-14T09:01:18.722916+0000","last_active":"2024-03-14T09:01:20.743928+0000","last_peered":"2024-03-14T09:01:20.743928+0000","last_clean":"2024-03-14T09:01:20.743928+0000","last_became_active":"2024-03-14T09:01:18.722610+0000","last_became_peered":"2024-03-14T09:01:18.722610+0000","last_unstale":"2024-03-14T09:01:20.743928+0000","last_undegraded":"2024-03-14T09:01:20.743928+0000","last_fullsized":"2024-03-14T09:01:20.743928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123315+0000","last_change":"2024-03-14T09:01:19.124237+0000","last_active":"2024-03-14T09:01:20.123315+0000","last_peered":"2024-03-14T09:01:20.123315+0000","last_clean":"2024-03-14T09:01:20.123315+0000","last_became_active":"2024-03-14T09:01:19.123958+0000","last_became_peered":"2024-03-14T09:01:19.123958+0000","last_unstale":"2024-03-14T09:01:20.123315+0000","last_undegraded":"2024-03-14T09:01:20.123315+0000","last_fullsized":"2024-03-14T09:01:20.123315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744772+0000","last_change":"2024-03-14T09:01:18.716867+0000","last_active":"2024-03-14T09:01:20.744772+0000","last_peered":"2024-03-14T09:01:20.744772+0000","last_clean":"2024-03-14T09:01:20.744772+0000","last_became_active":"2024-03-14T09:01:18.716352+0000","last_became_peered":"2024-03-14T09:01:18.716352+0000","last_unstale":"2024-03-14T09:01:20.744772+0000","last_undegraded":"2024-03-14T09:01:20.744772+0000","last_fullsized":"2024-03-14T09:01:20.744772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-14T09:01:19.129296+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:19.129296+0000","last_peered":"2024-03-14T09:01:19.129296+0000","last_clean":"2024-03-14T09:01:19.129296+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:19.129296+0000","last_undegraded":"2024-03-14T09:01:19.129296+0000","last_fullsized":"2024-03-14T09:01:19.129296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119237+0000","last_change":"2024-03-14T09:01:19.125813+0000","last_active":"2024-03-14T09:01:20.119237+0000","last_peered":"2024-03-14T09:01:20.119237+0000","last_clean":"2024-03-14T09:01:20.119237+0000","last_became_active":"2024-03-14T09:01:19.125536+0000","last_became_peered":"2024-03-14T09:01:19.125536+0000","last_unstale":"2024-03-14T09:01:20.119237+0000","last_undegraded":"2024-03-14T09:01:20.119237+0000","last_fullsized":"2024-03-14T09:01:20.119237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745288+0000","last_change":"2024-03-14T09:01:18.728259+0000","last_active":"2024-03-14T09:01:20.745288+0000","last_peered":"2024-03-14T09:01:20.745288+0000","last_clean":"2024-03-14T09:01:20.745288+0000","last_became_active":"2024-03-14T09:01:18.728000+0000","last_became_peered":"2024-03-14T09:01:18.728000+0000","last_unstale":"2024-03-14T09:01:20.745288+0000","last_undegraded":"2024-03-14T09:01:20.745288+0000","last_fullsized":"2024-03-14T09:01:20.745288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746270+0000","last_change":"2024-03-14T09:01:18.722659+0000","last_active":"2024-03-14T09:01:20.746270+0000","last_peered":"2024-03-14T09:01:20.746270+0000","last_clean":"2024-03-14T09:01:20.746270+0000","last_became_active":"2024-03-14T09:01:18.722309+0000","last_became_peered":"2024-03-14T09:01:18.722309+0000","last_unstale":"2024-03-14T09:01:20.746270+0000","last_undegraded":"2024-03-14T09:01:20.746270+0000","last_fullsized":"2024-03-14T09:01:20.746270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744400+0000","last_change":"2024-03-14T09:01:19.119183+0000","last_active":"2024-03-14T09:01:20.744400+0000","last_peered":"2024-03-14T09:01:20.744400+0000","last_clean":"2024-03-14T09:01:20.744400+0000","last_became_active":"2024-03-14T09:01:19.118862+0000","last_became_peered":"2024-03-14T09:01:19.118862+0000","last_unstale":"2024-03-14T09:01:20.744400+0000","last_undegraded":"2024-03-14T09:01:20.744400+0000","last_fullsized":"2024-03-14T09:01:20.744400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:22.610280+0000","last_change":"2024-03-14T09:01:18.719534+0000","last_active":"2024-03-14T09:01:22.610280+0000","last_peered":"2024-03-14T09:01:22.610280+0000","last_clean":"2024-03-14T09:01:22.610280+0000","last_became_active":"2024-03-14T09:01:18.719220+0000","last_became_peered":"2024-03-14T09:01:18.719220+0000","last_unstale":"2024-03-14T09:01:22.610280+0000","last_undegraded":"2024-03-14T09:01:22.610280+0000","last_fullsized":"2024-03-14T09:01:22.610280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123213+0000","last_change":"2024-03-14T09:01:19.135946+0000","last_active":"2024-03-14T09:01:20.123213+0000","last_peered":"2024-03-14T09:01:20.123213+0000","last_clean":"2024-03-14T09:01:20.123213+0000","last_became_active":"2024-03-14T09:01:19.135763+0000","last_became_peered":"2024-03-14T09:01:19.135763+0000","last_unstale":"2024-03-14T09:01:20.123213+0000","last_undegraded":"2024-03-14T09:01:20.123213+0000","last_fullsized":"2024-03-14T09:01:20.123213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745836+0000","last_change":"2024-03-14T09:01:18.723947+0000","last_active":"2024-03-14T09:01:20.745836+0000","last_peered":"2024-03-14T09:01:20.745836+0000","last_clean":"2024-03-14T09:01:20.745836+0000","last_became_active":"2024-03-14T09:01:18.723622+0000","last_became_peered":"2024-03-14T09:01:18.723622+0000","last_unstale":"2024-03-14T09:01:20.745836+0000","last_undegraded":"2024-03-14T09:01:20.745836+0000","last_fullsized":"2024-03-14T09:01:20.745836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119439+0000","last_change":"2024-03-14T09:01:19.121831+0000","last_active":"2024-03-14T09:01:20.119439+0000","last_peered":"2024-03-14T09:01:20.119439+0000","last_clean":"2024-03-14T09:01:20.119439+0000","last_became_active":"2024-03-14T09:01:19.121672+0000","last_became_peered":"2024-03-14T09:01:19.121672+0000","last_unstale":"2024-03-14T09:01:20.119439+0000","last_undegraded":"2024-03-14T09:01:20.119439+0000","last_fullsized":"2024-03-14T09:01:20.119439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744636+0000","last_change":"2024-03-14T09:01:18.725255+0000","last_active":"2024-03-14T09:01:20.744636+0000","last_peered":"2024-03-14T09:01:20.744636+0000","last_clean":"2024-03-14T09:01:20.744636+0000","last_became_active":"2024-03-14T09:01:18.724977+0000","last_became_peered":"2024-03-14T09:01:18.724977+0000","last_unstale":"2024-03-14T09:01:20.744636+0000","last_undegraded":"2024-03-14T09:01:20.744636+0000","last_fullsized":"2024-03-14T09:01:20.744636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:54.443401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122789+0000","last_change":"2024-03-14T09:01:19.127280+0000","last_active":"2024-03-14T09:01:20.122789+0000","last_peered":"2024-03-14T09:01:20.122789+0000","last_clean":"2024-03-14T09:01:20.122789+0000","last_became_active":"2024-03-14T09:01:19.126973+0000","last_became_peered":"2024-03-14T09:01:19.126973+0000","last_unstale":"2024-03-14T09:01:20.122789+0000","last_undegraded":"2024-03-14T09:01:20.122789+0000","last_fullsized":"2024-03-14T09:01:20.122789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744026+0000","last_change":"2024-03-14T09:01:18.719084+0000","last_active":"2024-03-14T09:01:20.744026+0000","last_peered":"2024-03-14T09:01:20.744026+0000","last_clean":"2024-03-14T09:01:20.744026+0000","last_became_active":"2024-03-14T09:01:18.718785+0000","last_became_peered":"2024-03-14T09:01:18.718785+0000","last_unstale":"2024-03-14T09:01:20.744026+0000","last_undegraded":"2024-03-14T09:01:20.744026+0000","last_fullsized":"2024-03-14T09:01:20.744026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122827+0000","last_change":"2024-03-14T09:01:19.123558+0000","last_active":"2024-03-14T09:01:20.122827+0000","last_peered":"2024-03-14T09:01:20.122827+0000","last_clean":"2024-03-14T09:01:20.122827+0000","last_became_active":"2024-03-14T09:01:19.122663+0000","last_became_peered":"2024-03-14T09:01:19.122663+0000","last_unstale":"2024-03-14T09:01:20.122827+0000","last_undegraded":"2024-03-14T09:01:20.122827+0000","last_fullsized":"2024-03-14T09:01:20.122827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744437+0000","last_change":"2024-03-14T09:01:18.717632+0000","last_active":"2024-03-14T09:01:20.744437+0000","last_peered":"2024-03-14T09:01:20.744437+0000","last_clean":"2024-03-14T09:01:20.744437+0000","last_became_active":"2024-03-14T09:01:18.717322+0000","last_became_peered":"2024-03-14T09:01:18.717322+0000","last_unstale":"2024-03-14T09:01:20.744437+0000","last_undegraded":"2024-03-14T09:01:20.744437+0000","last_fullsized":"2024-03-14T09:01:20.744437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123124+0000","last_change":"2024-03-14T09:01:19.124078+0000","last_active":"2024-03-14T09:01:20.123124+0000","last_peered":"2024-03-14T09:01:20.123124+0000","last_clean":"2024-03-14T09:01:20.123124+0000","last_became_active":"2024-03-14T09:01:19.123793+0000","last_became_peered":"2024-03-14T09:01:19.123793+0000","last_unstale":"2024-03-14T09:01:20.123124+0000","last_undegraded":"2024-03-14T09:01:20.123124+0000","last_fullsized":"2024-03-14T09:01:20.123124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745643+0000","last_change":"2024-03-14T09:01:18.725304+0000","last_active":"2024-03-14T09:01:20.745643+0000","last_peered":"2024-03-14T09:01:20.745643+0000","last_clean":"2024-03-14T09:01:20.745643+0000","last_became_active":"2024-03-14T09:01:18.725064+0000","last_became_peered":"2024-03-14T09:01:18.725064+0000","last_unstale":"2024-03-14T09:01:20.745643+0000","last_undegraded":"2024-03-14T09:01:20.745643+0000","last_fullsized":"2024-03-14T09:01:20.745643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:53:28.235917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.129552+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.129287+0000","last_became_peered":"2024-03-14T09:01:19.129287+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744170+0000","last_change":"2024-03-14T09:01:18.721721+0000","last_active":"2024-03-14T09:01:20.744170+0000","last_peered":"2024-03-14T09:01:20.744170+0000","last_clean":"2024-03-14T09:01:20.744170+0000","last_became_active":"2024-03-14T09:01:18.721450+0000","last_became_peered":"2024-03-14T09:01:18.721450+0000","last_unstale":"2024-03-14T09:01:20.744170+0000","last_undegraded":"2024-03-14T09:01:20.744170+0000","last_fullsized":"2024-03-14T09:01:20.744170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746277+0000","last_change":"2024-03-14T09:01:19.142624+0000","last_active":"2024-03-14T09:01:20.746277+0000","last_peered":"2024-03-14T09:01:20.746277+0000","last_clean":"2024-03-14T09:01:20.746277+0000","last_became_active":"2024-03-14T09:01:19.142294+0000","last_became_peered":"2024-03-14T09:01:19.142294+0000","last_unstale":"2024-03-14T09:01:20.746277+0000","last_undegraded":"2024-03-14T09:01:20.746277+0000","last_fullsized":"2024-03-14T09:01:20.746277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744505+0000","last_change":"2024-03-14T09:01:18.714548+0000","last_active":"2024-03-14T09:01:20.744505+0000","last_peered":"2024-03-14T09:01:20.744505+0000","last_clean":"2024-03-14T09:01:20.744505+0000","last_became_active":"2024-03-14T09:01:18.713927+0000","last_became_peered":"2024-03-14T09:01:18.713927+0000","last_unstale":"2024-03-14T09:01:20.744505+0000","last_undegraded":"2024-03-14T09:01:20.744505+0000","last_fullsized":"2024-03-14T09:01:20.744505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745857+0000","last_change":"2024-03-14T09:01:19.139093+0000","last_active":"2024-03-14T09:01:20.745857+0000","last_peered":"2024-03-14T09:01:20.745857+0000","last_clean":"2024-03-14T09:01:20.745857+0000","last_became_active":"2024-03-14T09:01:19.138560+0000","last_became_peered":"2024-03-14T09:01:19.138560+0000","last_unstale":"2024-03-14T09:01:20.745857+0000","last_undegraded":"2024-03-14T09:01:20.745857+0000","last_fullsized":"2024-03-14T09:01:20.745857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744326+0000","last_change":"2024-03-14T09:01:18.716783+0000","last_active":"2024-03-14T09:01:20.744326+0000","last_peered":"2024-03-14T09:01:20.744326+0000","last_clean":"2024-03-14T09:01:20.744326+0000","last_became_active":"2024-03-14T09:01:18.716538+0000","last_became_peered":"2024-03-14T09:01:18.716538+0000","last_unstale":"2024-03-14T09:01:20.744326+0000","last_undegraded":"2024-03-14T09:01:20.744326+0000","last_fullsized":"2024-03-14T09:01:20.744326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745724+0000","last_change":"2024-03-14T09:01:19.141337+0000","last_active":"2024-03-14T09:01:20.745724+0000","last_peered":"2024-03-14T09:01:20.745724+0000","last_clean":"2024-03-14T09:01:20.745724+0000","last_became_active":"2024-03-14T09:01:19.141093+0000","last_became_peered":"2024-03-14T09:01:19.141093+0000","last_unstale":"2024-03-14T09:01:20.745724+0000","last_undegraded":"2024-03-14T09:01:20.745724+0000","last_fullsized":"2024-03-14T09:01:20.745724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744540+0000","last_change":"2024-03-14T09:01:19.128035+0000","last_active":"2024-03-14T09:01:20.744540+0000","last_peered":"2024-03-14T09:01:20.744540+0000","last_clean":"2024-03-14T09:01:20.744540+0000","last_became_active":"2024-03-14T09:01:19.127787+0000","last_became_peered":"2024-03-14T09:01:19.127787+0000","last_unstale":"2024-03-14T09:01:20.744540+0000","last_undegraded":"2024-03-14T09:01:20.744540+0000","last_fullsized":"2024-03-14T09:01:20.744540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743991+0000","last_change":"2024-03-14T09:01:19.121494+0000","last_active":"2024-03-14T09:01:20.743991+0000","last_peered":"2024-03-14T09:01:20.743991+0000","last_clean":"2024-03-14T09:01:20.743991+0000","last_became_active":"2024-03-14T09:01:19.120995+0000","last_became_peered":"2024-03-14T09:01:19.120995+0000","last_unstale":"2024-03-14T09:01:20.743991+0000","last_undegraded":"2024-03-14T09:01:20.743991+0000","last_fullsized":"2024-03-14T09:01:20.743991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745172+0000","last_change":"2024-03-14T09:01:19.136830+0000","last_active":"2024-03-14T09:01:20.745172+0000","last_peered":"2024-03-14T09:01:20.745172+0000","last_clean":"2024-03-14T09:01:20.745172+0000","last_became_active":"2024-03-14T09:01:19.136568+0000","last_became_peered":"2024-03-14T09:01:19.136568+0000","last_unstale":"2024-03-14T09:01:20.745172+0000","last_undegraded":"2024-03-14T09:01:20.745172+0000","last_fullsized":"2024-03-14T09:01:20.745172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119641+0000","last_change":"2024-03-14T09:01:19.118186+0000","last_active":"2024-03-14T09:01:20.119641+0000","last_peered":"2024-03-14T09:01:20.119641+0000","last_clean":"2024-03-14T09:01:20.119641+0000","last_became_active":"2024-03-14T09:01:19.117955+0000","last_became_peered":"2024-03-14T09:01:19.117955+0000","last_unstale":"2024-03-14T09:01:20.119641+0000","last_undegraded":"2024-03-14T09:01:20.119641+0000","last_fullsized":"2024-03-14T09:01:20.119641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123167+0000","last_change":"2024-03-14T09:01:19.126685+0000","last_active":"2024-03-14T09:01:20.123167+0000","last_peered":"2024-03-14T09:01:20.123167+0000","last_clean":"2024-03-14T09:01:20.123167+0000","last_became_active":"2024-03-14T09:01:19.126196+0000","last_became_peered":"2024-03-14T09:01:19.126196+0000","last_unstale":"2024-03-14T09:01:20.123167+0000","last_undegraded":"2024-03-14T09:01:20.123167+0000","last_fullsized":"2024-03-14T09:01:20.123167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744458+0000","last_change":"2024-03-14T09:01:19.119001+0000","last_active":"2024-03-14T09:01:20.744458+0000","last_peered":"2024-03-14T09:01:20.744458+0000","last_clean":"2024-03-14T09:01:20.744458+0000","last_became_active":"2024-03-14T09:01:19.118755+0000","last_became_peered":"2024-03-14T09:01:19.118755+0000","last_unstale":"2024-03-14T09:01:20.744458+0000","last_undegraded":"2024-03-14T09:01:20.744458+0000","last_fullsized":"2024-03-14T09:01:20.744458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745084+0000","last_change":"2024-03-14T09:01:19.136652+0000","last_active":"2024-03-14T09:01:20.745084+0000","last_peered":"2024-03-14T09:01:20.745084+0000","last_clean":"2024-03-14T09:01:20.745084+0000","last_became_active":"2024-03-14T09:01:19.136252+0000","last_became_peered":"2024-03-14T09:01:19.136252+0000","last_unstale":"2024-03-14T09:01:20.745084+0000","last_undegraded":"2024-03-14T09:01:20.745084+0000","last_fullsized":"2024-03-14T09:01:20.745084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123129+0000","last_change":"2024-03-14T09:01:19.130297+0000","last_active":"2024-03-14T09:01:20.123129+0000","last_peered":"2024-03-14T09:01:20.123129+0000","last_clean":"2024-03-14T09:01:20.123129+0000","last_became_active":"2024-03-14T09:01:19.130024+0000","last_became_peered":"2024-03-14T09:01:19.130024+0000","last_unstale":"2024-03-14T09:01:20.123129+0000","last_undegraded":"2024-03-14T09:01:20.123129+0000","last_fullsized":"2024-03-14T09:01:20.123129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744787+0000","last_change":"2024-03-14T09:01:19.137747+0000","last_active":"2024-03-14T09:01:20.744787+0000","last_peered":"2024-03-14T09:01:20.744787+0000","last_clean":"2024-03-14T09:01:20.744787+0000","last_became_active":"2024-03-14T09:01:19.137471+0000","last_became_peered":"2024-03-14T09:01:19.137471+0000","last_unstale":"2024-03-14T09:01:20.744787+0000","last_undegraded":"2024-03-14T09:01:20.744787+0000","last_fullsized":"2024-03-14T09:01:20.744787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122916+0000","last_change":"2024-03-14T09:01:19.128038+0000","last_active":"2024-03-14T09:01:20.122916+0000","last_peered":"2024-03-14T09:01:20.122916+0000","last_clean":"2024-03-14T09:01:20.122916+0000","last_became_active":"2024-03-14T09:01:19.127847+0000","last_became_peered":"2024-03-14T09:01:19.127847+0000","last_unstale":"2024-03-14T09:01:20.122916+0000","last_undegraded":"2024-03-14T09:01:20.122916+0000","last_fullsized":"2024-03-14T09:01:20.122916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745074+0000","last_change":"2024-03-14T09:01:19.128566+0000","last_active":"2024-03-14T09:01:20.745074+0000","last_peered":"2024-03-14T09:01:20.745074+0000","last_clean":"2024-03-14T09:01:20.745074+0000","last_became_active":"2024-03-14T09:01:19.127893+0000","last_became_peered":"2024-03-14T09:01:19.127893+0000","last_unstale":"2024-03-14T09:01:20.745074+0000","last_undegraded":"2024-03-14T09:01:20.745074+0000","last_fullsized":"2024-03-14T09:01:20.745074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119666+0000","last_change":"2024-03-14T09:01:19.134739+0000","last_active":"2024-03-14T09:01:20.119666+0000","last_peered":"2024-03-14T09:01:20.119666+0000","last_clean":"2024-03-14T09:01:20.119666+0000","last_became_active":"2024-03-14T09:01:19.134442+0000","last_became_peered":"2024-03-14T09:01:19.134442+0000","last_unstale":"2024-03-14T09:01:20.119666+0000","last_undegraded":"2024-03-14T09:01:20.119666+0000","last_fullsized":"2024-03-14T09:01:20.119666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746583+0000","last_change":"2024-03-14T09:01:19.132199+0000","last_active":"2024-03-14T09:01:20.746583+0000","last_peered":"2024-03-14T09:01:20.746583+0000","last_clean":"2024-03-14T09:01:20.746583+0000","last_became_active":"2024-03-14T09:01:19.131887+0000","last_became_peered":"2024-03-14T09:01:19.131887+0000","last_unstale":"2024-03-14T09:01:20.746583+0000","last_undegraded":"2024-03-14T09:01:20.746583+0000","last_fullsized":"2024-03-14T09:01:20.746583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:01:28.055 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:01:28.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.212+0000 7f8f0c2e2700 1 Processor -- start 2024-03-14T09:01:28.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.212+0000 7f8f0c2e2700 1 -- start start 2024-03-14T09:01:28.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0c2e2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8f040c1e90 0x7f8f0413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0c2e2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0413ade0 con 0x7f8f040c3250 2024-03-14T09:01:28.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0c2e2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0406fb30 con 0x7f8f040a8540 2024-03-14T09:01:28.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0c2e2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0406ec30 con 0x7f8f040c1e90 2024-03-14T09:01:28.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0987d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8f040c3250 0x7f8f040c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34616/0 (socket says 172.21.15.154:34616) 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.213+0000 7f8f0987d700 1 -- 172.21.15.154:0/3905406102 learned_addr learned my addr 172.21.15.154:0/3905406102 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/3905406102 >> v2:172.21.15.191:3300/0 conn(0x7f8f040c1e90 0x7f8f0413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802940149 0 0) 0x7f8f00001110 con 0x7f8f040c3250 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef00035b0 con 0x7f8f040c3250 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496621719 0 0) 0x7f8ef00035b0 con 0x7f8f040c3250 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 >> v2:172.21.15.191:3300/0 conn(0x7f8f040c1e90 msgr2=0x7f8f0413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 --2- 172.21.15.154:0/3905406102 >> v2:172.21.15.191:3300/0 conn(0x7f8f040c1e90 0x7f8f0413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 >> v1:172.21.15.191:6789/0 conn(0x7f8f040a8540 legacy=0x7f8f04154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0406ef10 con 0x7f8f040c3250 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8f00001800 con 0x7f8f040c3250 2024-03-14T09:01:28.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 >> v1:172.21.15.154:6789/0 conn(0x7f8f040c3250 legacy=0x7f8f040c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ef0003a00 0x7f8ef0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0004850 0x7f8ef0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 0x7f8ef000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0406ec30 con 0x7f8ef0005330 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef000c200 con 0x7f8ef0004850 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef000c3c0 con 0x7f8ef0003a00 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 0x7f8ef000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a07e700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ef0003a00 0x7f8ef0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0987d700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0004850 0x7f8ef0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 -- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ef0003a00 msgr2=0x7f8ef0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ef0003a00 0x7f8ef0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 -- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0004850 msgr2=0x7f8ef0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0004850 0x7f8ef0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 -- 172.21.15.154:0/3905406102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef00035b0 con 0x7f8ef0005330 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 -- 172.21.15.154:0/3905406102 reap_dead start 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.214+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 0x7f8ef000ba10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f90 tx=0x7f8ef0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3466e7bbce8dd76 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003a20 con 0x7f8ef0005330 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00f040 con 0x7f8ef0005330 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0907c700 1 -- 172.21.15.154:0/3905406102 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003dc0 con 0x7f8ef0005330 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 msgr2=0x7f8ef000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 0x7f8ef000ba10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f90 tx=0x7f8ef0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/3905406102 shutdown_connections 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/3905406102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ef0005330 0x7f8ef000ba10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/3905406102 >> 172.21.15.154:0/3905406102 conn(0x7f8f040c3880 msgr2=0x7f8f040c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/3905406102 shutdown_connections 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.215+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/3905406102 wait complete. 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.216+0000 7f8f0c2e2700 1 Processor -- start 2024-03-14T09:01:28.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.216+0000 7f8f0c2e2700 1 -- start start 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0005330 0x7f8f0412afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0412a620 0x7f8f04128630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040abee0 con 0x7f8f040aab50 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040ac810 con 0x7f8ef0005330 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0c2e2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04129cb0 con 0x7f8f0412a620 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49012/0 (socket says 172.21.15.154:49012) 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 -- 172.21.15.154:0/1963817212 learned_addr learned my addr 172.21.15.154:0/1963817212 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a07e700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0005330 0x7f8f0412afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0987d700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0412a620 0x7f8f04128630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 -- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0412a620 msgr2=0x7f8f04128630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0412a620 0x7f8f04128630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 -- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0005330 msgr2=0x7f8f0412afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0005330 0x7f8f0412afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.217+0000 7f8f0a87f700 1 -- 172.21.15.154:0/1963817212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040abc00 con 0x7f8f040aab50 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8f0a87f700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f040728f0 tx=0x7f8efc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003e10 con 0x7f8f040aab50 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc004230 con 0x7f8f040aab50 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f04128280 con 0x7f8f040aab50 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0045e0 con 0x7f8f040aab50 2024-03-14T09:01:28.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.218+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f04155310 con 0x7f8f040aab50 2024-03-14T09:01:28.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.219+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f040a92f0 con 0x7f8f040aab50 2024-03-14T09:01:28.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.219+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8efc004740 con 0x7f8f040aab50 2024-03-14T09:01:28.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.219+0000 7f8efaffd700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 0x7f8ee806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.220+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8efc089ec0 con 0x7f8f040aab50 2024-03-14T09:01:28.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.220+0000 7f8f0a07e700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 0x7f8ee806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.220+0000 7f8f0a07e700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 0x7f8ee806ccf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000c00 tx=0x7f8ef4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.224+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8efc0552f0 con 0x7f8f040aab50 2024-03-14T09:01:28.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.416+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f04155900 con 0x7f8ee806a830 2024-03-14T09:01:28.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.424+0000 7f8efaffd700 1 -- 172.21.15.154:0/1963817212 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264587 (secure 0 0 0) 0x7f8ef4001ca0 con 0x7f8ee806a830 2024-03-14T09:01:28.427 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.426+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 msgr2=0x7f8ee806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.426+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 0x7f8ee806ccf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4000c00 tx=0x7f8ef4009040 comp rx=0 tx=0).stop 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.426+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 msgr2=0x7f8f040aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.426+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f040728f0 tx=0x7f8efc004b40 comp rx=0 tx=0).stop 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 shutdown_connections 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ee806a830 0x7f8ee806ccf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f040aab50 0x7f8f040aafd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ef0005330 0x7f8f0412afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 --2- 172.21.15.154:0/1963817212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0412a620 0x7f8f04128630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 >> 172.21.15.154:0/1963817212 conn(0x7f8f040c3880 msgr2=0x7f8f04139b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 shutdown_connections 2024-03-14T09:01:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.427+0000 7f8f0c2e2700 1 -- 172.21.15.154:0/1963817212 wait complete. 2024-03-14T09:01:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:01:28.439 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-14T09:01:27.519287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1752644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.409468"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745804+0000","last_change":"2024-03-14T09:01:19.140304+0000","last_active":"2024-03-14T09:01:20.745804+0000","last_peered":"2024-03-14T09:01:20.745804+0000","last_clean":"2024-03-14T09:01:20.745804+0000","last_became_active":"2024-03-14T09:01:19.140051+0000","last_became_peered":"2024-03-14T09:01:19.140051+0000","last_unstale":"2024-03-14T09:01:20.745804+0000","last_undegraded":"2024-03-14T09:01:20.745804+0000","last_fullsized":"2024-03-14T09:01:20.745804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744549+0000","last_change":"2024-03-14T09:01:19.119652+0000","last_active":"2024-03-14T09:01:20.744549+0000","last_peered":"2024-03-14T09:01:20.744549+0000","last_clean":"2024-03-14T09:01:20.744549+0000","last_became_active":"2024-03-14T09:01:19.119408+0000","last_became_peered":"2024-03-14T09:01:19.119408+0000","last_unstale":"2024-03-14T09:01:20.744549+0000","last_undegraded":"2024-03-14T09:01:20.744549+0000","last_fullsized":"2024-03-14T09:01:20.744549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119387+0000","last_change":"2024-03-14T09:01:19.122167+0000","last_active":"2024-03-14T09:01:20.119387+0000","last_peered":"2024-03-14T09:01:20.119387+0000","last_clean":"2024-03-14T09:01:20.119387+0000","last_became_active":"2024-03-14T09:01:19.121201+0000","last_became_peered":"2024-03-14T09:01:19.121201+0000","last_unstale":"2024-03-14T09:01:20.119387+0000","last_undegraded":"2024-03-14T09:01:20.119387+0000","last_fullsized":"2024-03-14T09:01:20.119387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746011+0000","last_change":"2024-03-14T09:01:19.129747+0000","last_active":"2024-03-14T09:01:20.746011+0000","last_peered":"2024-03-14T09:01:20.746011+0000","last_clean":"2024-03-14T09:01:20.746011+0000","last_became_active":"2024-03-14T09:01:19.129254+0000","last_became_peered":"2024-03-14T09:01:19.129254+0000","last_unstale":"2024-03-14T09:01:20.746011+0000","last_undegraded":"2024-03-14T09:01:20.746011+0000","last_fullsized":"2024-03-14T09:01:20.746011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745535+0000","last_change":"2024-03-14T09:01:19.138324+0000","last_active":"2024-03-14T09:01:20.745535+0000","last_peered":"2024-03-14T09:01:20.745535+0000","last_clean":"2024-03-14T09:01:20.745535+0000","last_became_active":"2024-03-14T09:01:19.138018+0000","last_became_peered":"2024-03-14T09:01:19.138018+0000","last_unstale":"2024-03-14T09:01:20.745535+0000","last_undegraded":"2024-03-14T09:01:20.745535+0000","last_fullsized":"2024-03-14T09:01:20.745535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:51.127253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744951+0000","last_change":"2024-03-14T09:01:19.129902+0000","last_active":"2024-03-14T09:01:20.744951+0000","last_peered":"2024-03-14T09:01:20.744951+0000","last_clean":"2024-03-14T09:01:20.744951+0000","last_became_active":"2024-03-14T09:01:19.128311+0000","last_became_peered":"2024-03-14T09:01:19.128311+0000","last_unstale":"2024-03-14T09:01:20.744951+0000","last_undegraded":"2024-03-14T09:01:20.744951+0000","last_fullsized":"2024-03-14T09:01:20.744951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745141+0000","last_change":"2024-03-14T09:01:19.136657+0000","last_active":"2024-03-14T09:01:20.745141+0000","last_peered":"2024-03-14T09:01:20.745141+0000","last_clean":"2024-03-14T09:01:20.745141+0000","last_became_active":"2024-03-14T09:01:19.136253+0000","last_became_peered":"2024-03-14T09:01:19.136253+0000","last_unstale":"2024-03-14T09:01:20.745141+0000","last_undegraded":"2024-03-14T09:01:20.745141+0000","last_fullsized":"2024-03-14T09:01:20.745141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:49.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122779+0000","last_change":"2024-03-14T09:01:19.123809+0000","last_active":"2024-03-14T09:01:20.122779+0000","last_peered":"2024-03-14T09:01:20.122779+0000","last_clean":"2024-03-14T09:01:20.122779+0000","last_became_active":"2024-03-14T09:01:19.122471+0000","last_became_peered":"2024-03-14T09:01:19.122471+0000","last_unstale":"2024-03-14T09:01:20.122779+0000","last_undegraded":"2024-03-14T09:01:20.122779+0000","last_fullsized":"2024-03-14T09:01:20.122779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745747+0000","last_change":"2024-03-14T09:01:19.142452+0000","last_active":"2024-03-14T09:01:20.745747+0000","last_peered":"2024-03-14T09:01:20.745747+0000","last_clean":"2024-03-14T09:01:20.745747+0000","last_became_active":"2024-03-14T09:01:19.142164+0000","last_became_peered":"2024-03-14T09:01:19.142164+0000","last_unstale":"2024-03-14T09:01:20.745747+0000","last_undegraded":"2024-03-14T09:01:20.745747+0000","last_fullsized":"2024-03-14T09:01:20.745747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745182+0000","last_change":"2024-03-14T09:01:19.131516+0000","last_active":"2024-03-14T09:01:20.745182+0000","last_peered":"2024-03-14T09:01:20.745182+0000","last_clean":"2024-03-14T09:01:20.745182+0000","last_became_active":"2024-03-14T09:01:19.131190+0000","last_became_peered":"2024-03-14T09:01:19.131190+0000","last_unstale":"2024-03-14T09:01:20.745182+0000","last_undegraded":"2024-03-14T09:01:20.745182+0000","last_fullsized":"2024-03-14T09:01:20.745182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745021+0000","last_change":"2024-03-14T09:01:19.137722+0000","last_active":"2024-03-14T09:01:20.745021+0000","last_peered":"2024-03-14T09:01:20.745021+0000","last_clean":"2024-03-14T09:01:20.745021+0000","last_became_active":"2024-03-14T09:01:19.137447+0000","last_became_peered":"2024-03-14T09:01:19.137447+0000","last_unstale":"2024-03-14T09:01:20.745021+0000","last_undegraded":"2024-03-14T09:01:20.745021+0000","last_fullsized":"2024-03-14T09:01:20.745021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123173+0000","last_change":"2024-03-14T09:01:19.127732+0000","last_active":"2024-03-14T09:01:20.123173+0000","last_peered":"2024-03-14T09:01:20.123173+0000","last_clean":"2024-03-14T09:01:20.123173+0000","last_became_active":"2024-03-14T09:01:19.126961+0000","last_became_peered":"2024-03-14T09:01:19.126961+0000","last_unstale":"2024-03-14T09:01:20.123173+0000","last_undegraded":"2024-03-14T09:01:20.123173+0000","last_fullsized":"2024-03-14T09:01:20.123173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122936+0000","last_change":"2024-03-14T09:01:19.131805+0000","last_active":"2024-03-14T09:01:20.122936+0000","last_peered":"2024-03-14T09:01:20.122936+0000","last_clean":"2024-03-14T09:01:20.122936+0000","last_became_active":"2024-03-14T09:01:19.131545+0000","last_became_peered":"2024-03-14T09:01:19.131545+0000","last_unstale":"2024-03-14T09:01:20.122936+0000","last_undegraded":"2024-03-14T09:01:20.122936+0000","last_fullsized":"2024-03-14T09:01:20.122936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119342+0000","last_change":"2024-03-14T09:01:19.121888+0000","last_active":"2024-03-14T09:01:20.119342+0000","last_peered":"2024-03-14T09:01:20.119342+0000","last_clean":"2024-03-14T09:01:20.119342+0000","last_became_active":"2024-03-14T09:01:19.121604+0000","last_became_peered":"2024-03-14T09:01:19.121604+0000","last_unstale":"2024-03-14T09:01:20.119342+0000","last_undegraded":"2024-03-14T09:01:20.119342+0000","last_fullsized":"2024-03-14T09:01:20.119342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743969+0000","last_change":"2024-03-14T09:01:19.125119+0000","last_active":"2024-03-14T09:01:20.743969+0000","last_peered":"2024-03-14T09:01:20.743969+0000","last_clean":"2024-03-14T09:01:20.743969+0000","last_became_active":"2024-03-14T09:01:19.124829+0000","last_became_peered":"2024-03-14T09:01:19.124829+0000","last_unstale":"2024-03-14T09:01:20.743969+0000","last_undegraded":"2024-03-14T09:01:20.743969+0000","last_fullsized":"2024-03-14T09:01:20.743969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744669+0000","last_change":"2024-03-14T09:01:19.128411+0000","last_active":"2024-03-14T09:01:20.744669+0000","last_peered":"2024-03-14T09:01:20.744669+0000","last_clean":"2024-03-14T09:01:20.744669+0000","last_became_active":"2024-03-14T09:01:19.127935+0000","last_became_peered":"2024-03-14T09:01:19.127935+0000","last_unstale":"2024-03-14T09:01:20.744669+0000","last_undegraded":"2024-03-14T09:01:20.744669+0000","last_fullsized":"2024-03-14T09:01:20.744669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745529+0000","last_change":"2024-03-14T09:01:19.139316+0000","last_active":"2024-03-14T09:01:20.745529+0000","last_peered":"2024-03-14T09:01:20.745529+0000","last_clean":"2024-03-14T09:01:20.745529+0000","last_became_active":"2024-03-14T09:01:19.139045+0000","last_became_peered":"2024-03-14T09:01:19.139045+0000","last_unstale":"2024-03-14T09:01:20.745529+0000","last_undegraded":"2024-03-14T09:01:20.745529+0000","last_fullsized":"2024-03-14T09:01:20.745529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744233+0000","last_change":"2024-03-14T09:01:19.120202+0000","last_active":"2024-03-14T09:01:20.744233+0000","last_peered":"2024-03-14T09:01:20.744233+0000","last_clean":"2024-03-14T09:01:20.744233+0000","last_became_active":"2024-03-14T09:01:19.119935+0000","last_became_peered":"2024-03-14T09:01:19.119935+0000","last_unstale":"2024-03-14T09:01:20.744233+0000","last_undegraded":"2024-03-14T09:01:20.744233+0000","last_fullsized":"2024-03-14T09:01:20.744233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:55.449848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746144+0000","last_change":"2024-03-14T09:01:19.139436+0000","last_active":"2024-03-14T09:01:20.746144+0000","last_peered":"2024-03-14T09:01:20.746144+0000","last_clean":"2024-03-14T09:01:20.746144+0000","last_became_active":"2024-03-14T09:01:19.139165+0000","last_became_peered":"2024-03-14T09:01:19.139165+0000","last_unstale":"2024-03-14T09:01:20.746144+0000","last_undegraded":"2024-03-14T09:01:20.746144+0000","last_fullsized":"2024-03-14T09:01:20.746144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:44:30.654286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119464+0000","last_change":"2024-03-14T09:01:18.720548+0000","last_active":"2024-03-14T09:01:20.119464+0000","last_peered":"2024-03-14T09:01:20.119464+0000","last_clean":"2024-03-14T09:01:20.119464+0000","last_became_active":"2024-03-14T09:01:18.720258+0000","last_became_peered":"2024-03-14T09:01:18.720258+0000","last_unstale":"2024-03-14T09:01:20.119464+0000","last_undegraded":"2024-03-14T09:01:20.119464+0000","last_fullsized":"2024-03-14T09:01:20.119464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123199+0000","last_change":"2024-03-14T09:01:19.131625+0000","last_active":"2024-03-14T09:01:20.123199+0000","last_peered":"2024-03-14T09:01:20.123199+0000","last_clean":"2024-03-14T09:01:20.123199+0000","last_became_active":"2024-03-14T09:01:19.131308+0000","last_became_peered":"2024-03-14T09:01:19.131308+0000","last_unstale":"2024-03-14T09:01:20.123199+0000","last_undegraded":"2024-03-14T09:01:20.123199+0000","last_fullsized":"2024-03-14T09:01:20.123199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123216+0000","last_change":"2024-03-14T09:01:18.722396+0000","last_active":"2024-03-14T09:01:20.123216+0000","last_peered":"2024-03-14T09:01:20.123216+0000","last_clean":"2024-03-14T09:01:20.123216+0000","last_became_active":"2024-03-14T09:01:18.721852+0000","last_became_peered":"2024-03-14T09:01:18.721852+0000","last_unstale":"2024-03-14T09:01:20.123216+0000","last_undegraded":"2024-03-14T09:01:20.123216+0000","last_fullsized":"2024-03-14T09:01:20.123216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119416+0000","last_change":"2024-03-14T09:01:19.127918+0000","last_active":"2024-03-14T09:01:20.119416+0000","last_peered":"2024-03-14T09:01:20.119416+0000","last_clean":"2024-03-14T09:01:20.119416+0000","last_became_active":"2024-03-14T09:01:19.127663+0000","last_became_peered":"2024-03-14T09:01:19.127663+0000","last_unstale":"2024-03-14T09:01:20.119416+0000","last_undegraded":"2024-03-14T09:01:20.119416+0000","last_fullsized":"2024-03-14T09:01:20.119416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746324+0000","last_change":"2024-03-14T09:01:18.727557+0000","last_active":"2024-03-14T09:01:20.746324+0000","last_peered":"2024-03-14T09:01:20.746324+0000","last_clean":"2024-03-14T09:01:20.746324+0000","last_became_active":"2024-03-14T09:01:18.727245+0000","last_became_peered":"2024-03-14T09:01:18.727245+0000","last_unstale":"2024-03-14T09:01:20.746324+0000","last_undegraded":"2024-03-14T09:01:20.746324+0000","last_fullsized":"2024-03-14T09:01:20.746324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745761+0000","last_change":"2024-03-14T09:01:19.138918+0000","last_active":"2024-03-14T09:01:20.745761+0000","last_peered":"2024-03-14T09:01:20.745761+0000","last_clean":"2024-03-14T09:01:20.745761+0000","last_became_active":"2024-03-14T09:01:19.137549+0000","last_became_peered":"2024-03-14T09:01:19.137549+0000","last_unstale":"2024-03-14T09:01:20.745761+0000","last_undegraded":"2024-03-14T09:01:20.745761+0000","last_fullsized":"2024-03-14T09:01:20.745761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745273+0000","last_change":"2024-03-14T09:01:18.724542+0000","last_active":"2024-03-14T09:01:20.745273+0000","last_peered":"2024-03-14T09:01:20.745273+0000","last_clean":"2024-03-14T09:01:20.745273+0000","last_became_active":"2024-03-14T09:01:18.724257+0000","last_became_peered":"2024-03-14T09:01:18.724257+0000","last_unstale":"2024-03-14T09:01:20.745273+0000","last_undegraded":"2024-03-14T09:01:20.745273+0000","last_fullsized":"2024-03-14T09:01:20.745273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745073+0000","last_change":"2024-03-14T09:01:19.125917+0000","last_active":"2024-03-14T09:01:20.745073+0000","last_peered":"2024-03-14T09:01:20.745073+0000","last_clean":"2024-03-14T09:01:20.745073+0000","last_became_active":"2024-03-14T09:01:19.125654+0000","last_became_peered":"2024-03-14T09:01:19.125654+0000","last_unstale":"2024-03-14T09:01:20.745073+0000","last_undegraded":"2024-03-14T09:01:20.745073+0000","last_fullsized":"2024-03-14T09:01:20.745073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123032+0000","last_change":"2024-03-14T09:01:18.725364+0000","last_active":"2024-03-14T09:01:20.123032+0000","last_peered":"2024-03-14T09:01:20.123032+0000","last_clean":"2024-03-14T09:01:20.123032+0000","last_became_active":"2024-03-14T09:01:18.725141+0000","last_became_peered":"2024-03-14T09:01:18.725141+0000","last_unstale":"2024-03-14T09:01:20.123032+0000","last_undegraded":"2024-03-14T09:01:20.123032+0000","last_fullsized":"2024-03-14T09:01:20.123032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:06:40.925950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122963+0000","last_change":"2024-03-14T09:01:19.128226+0000","last_active":"2024-03-14T09:01:20.122963+0000","last_peered":"2024-03-14T09:01:20.122963+0000","last_clean":"2024-03-14T09:01:20.122963+0000","last_became_active":"2024-03-14T09:01:19.127601+0000","last_became_peered":"2024-03-14T09:01:19.127601+0000","last_unstale":"2024-03-14T09:01:20.122963+0000","last_undegraded":"2024-03-14T09:01:20.122963+0000","last_fullsized":"2024-03-14T09:01:20.122963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:05.796691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745089+0000","last_change":"2024-03-14T09:01:18.722204+0000","last_active":"2024-03-14T09:01:20.745089+0000","last_peered":"2024-03-14T09:01:20.745089+0000","last_clean":"2024-03-14T09:01:20.745089+0000","last_became_active":"2024-03-14T09:01:18.721901+0000","last_became_peered":"2024-03-14T09:01:18.721901+0000","last_unstale":"2024-03-14T09:01:20.745089+0000","last_undegraded":"2024-03-14T09:01:20.745089+0000","last_fullsized":"2024-03-14T09:01:20.745089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119757+0000","last_change":"2024-03-14T09:01:19.127938+0000","last_active":"2024-03-14T09:01:20.119757+0000","last_peered":"2024-03-14T09:01:20.119757+0000","last_clean":"2024-03-14T09:01:20.119757+0000","last_became_active":"2024-03-14T09:01:19.127718+0000","last_became_peered":"2024-03-14T09:01:19.127718+0000","last_unstale":"2024-03-14T09:01:20.119757+0000","last_undegraded":"2024-03-14T09:01:20.119757+0000","last_fullsized":"2024-03-14T09:01:20.119757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122920+0000","last_change":"2024-03-14T09:01:18.726440+0000","last_active":"2024-03-14T09:01:20.122920+0000","last_peered":"2024-03-14T09:01:20.122920+0000","last_clean":"2024-03-14T09:01:20.122920+0000","last_became_active":"2024-03-14T09:01:18.726143+0000","last_became_peered":"2024-03-14T09:01:18.726143+0000","last_unstale":"2024-03-14T09:01:20.122920+0000","last_undegraded":"2024-03-14T09:01:20.122920+0000","last_fullsized":"2024-03-14T09:01:20.122920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122876+0000","last_change":"2024-03-14T09:01:19.117249+0000","last_active":"2024-03-14T09:01:20.122876+0000","last_peered":"2024-03-14T09:01:20.122876+0000","last_clean":"2024-03-14T09:01:20.122876+0000","last_became_active":"2024-03-14T09:01:19.116999+0000","last_became_peered":"2024-03-14T09:01:19.116999+0000","last_unstale":"2024-03-14T09:01:20.122876+0000","last_undegraded":"2024-03-14T09:01:20.122876+0000","last_fullsized":"2024-03-14T09:01:20.122876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:56:30.566828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744666+0000","last_change":"2024-03-14T09:01:18.721268+0000","last_active":"2024-03-14T09:01:20.744666+0000","last_peered":"2024-03-14T09:01:20.744666+0000","last_clean":"2024-03-14T09:01:20.744666+0000","last_became_active":"2024-03-14T09:01:18.720984+0000","last_became_peered":"2024-03-14T09:01:18.720984+0000","last_unstale":"2024-03-14T09:01:20.744666+0000","last_undegraded":"2024-03-14T09:01:20.744666+0000","last_fullsized":"2024-03-14T09:01:20.744666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744322+0000","last_change":"2024-03-14T09:01:19.120543+0000","last_active":"2024-03-14T09:01:20.744322+0000","last_peered":"2024-03-14T09:01:20.744322+0000","last_clean":"2024-03-14T09:01:20.744322+0000","last_became_active":"2024-03-14T09:01:19.119215+0000","last_became_peered":"2024-03-14T09:01:19.119215+0000","last_unstale":"2024-03-14T09:01:20.744322+0000","last_undegraded":"2024-03-14T09:01:20.744322+0000","last_fullsized":"2024-03-14T09:01:20.744322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745938+0000","last_change":"2024-03-14T09:01:18.724696+0000","last_active":"2024-03-14T09:01:20.745938+0000","last_peered":"2024-03-14T09:01:20.745938+0000","last_clean":"2024-03-14T09:01:20.745938+0000","last_became_active":"2024-03-14T09:01:18.724393+0000","last_became_peered":"2024-03-14T09:01:18.724393+0000","last_unstale":"2024-03-14T09:01:20.745938+0000","last_undegraded":"2024-03-14T09:01:20.745938+0000","last_fullsized":"2024-03-14T09:01:20.745938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745050+0000","last_change":"2024-03-14T09:01:19.129590+0000","last_active":"2024-03-14T09:01:20.745050+0000","last_peered":"2024-03-14T09:01:20.745050+0000","last_clean":"2024-03-14T09:01:20.745050+0000","last_became_active":"2024-03-14T09:01:19.129324+0000","last_became_peered":"2024-03-14T09:01:19.129324+0000","last_unstale":"2024-03-14T09:01:20.745050+0000","last_undegraded":"2024-03-14T09:01:20.745050+0000","last_fullsized":"2024-03-14T09:01:20.745050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744921+0000","last_change":"2024-03-14T09:01:18.720316+0000","last_active":"2024-03-14T09:01:20.744921+0000","last_peered":"2024-03-14T09:01:20.744921+0000","last_clean":"2024-03-14T09:01:20.744921+0000","last_became_active":"2024-03-14T09:01:18.720030+0000","last_became_peered":"2024-03-14T09:01:18.720030+0000","last_unstale":"2024-03-14T09:01:20.744921+0000","last_undegraded":"2024-03-14T09:01:20.744921+0000","last_fullsized":"2024-03-14T09:01:20.744921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119562+0000","last_change":"2024-03-14T09:01:19.126442+0000","last_active":"2024-03-14T09:01:20.119562+0000","last_peered":"2024-03-14T09:01:20.119562+0000","last_clean":"2024-03-14T09:01:20.119562+0000","last_became_active":"2024-03-14T09:01:19.126192+0000","last_became_peered":"2024-03-14T09:01:19.126192+0000","last_unstale":"2024-03-14T09:01:20.119562+0000","last_undegraded":"2024-03-14T09:01:20.119562+0000","last_fullsized":"2024-03-14T09:01:20.119562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:57:19.311712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744536+0000","last_change":"2024-03-14T09:01:18.714712+0000","last_active":"2024-03-14T09:01:20.744536+0000","last_peered":"2024-03-14T09:01:20.744536+0000","last_clean":"2024-03-14T09:01:20.744536+0000","last_became_active":"2024-03-14T09:01:18.714304+0000","last_became_peered":"2024-03-14T09:01:18.714304+0000","last_unstale":"2024-03-14T09:01:20.744536+0000","last_undegraded":"2024-03-14T09:01:20.744536+0000","last_fullsized":"2024-03-14T09:01:20.744536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123108+0000","last_change":"2024-03-14T09:01:19.129609+0000","last_active":"2024-03-14T09:01:20.123108+0000","last_peered":"2024-03-14T09:01:20.123108+0000","last_clean":"2024-03-14T09:01:20.123108+0000","last_became_active":"2024-03-14T09:01:19.129345+0000","last_became_peered":"2024-03-14T09:01:19.129345+0000","last_unstale":"2024-03-14T09:01:20.123108+0000","last_undegraded":"2024-03-14T09:01:20.123108+0000","last_fullsized":"2024-03-14T09:01:20.123108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:57.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746528+0000","last_change":"2024-03-14T09:01:18.721291+0000","last_active":"2024-03-14T09:01:20.746528+0000","last_peered":"2024-03-14T09:01:20.746528+0000","last_clean":"2024-03-14T09:01:20.746528+0000","last_became_active":"2024-03-14T09:01:18.719243+0000","last_became_peered":"2024-03-14T09:01:18.719243+0000","last_unstale":"2024-03-14T09:01:20.746528+0000","last_undegraded":"2024-03-14T09:01:20.746528+0000","last_fullsized":"2024-03-14T09:01:20.746528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746209+0000","last_change":"2024-03-14T09:01:19.139742+0000","last_active":"2024-03-14T09:01:20.746209+0000","last_peered":"2024-03-14T09:01:20.746209+0000","last_clean":"2024-03-14T09:01:20.746209+0000","last_became_active":"2024-03-14T09:01:19.139476+0000","last_became_peered":"2024-03-14T09:01:19.139476+0000","last_unstale":"2024-03-14T09:01:20.746209+0000","last_undegraded":"2024-03-14T09:01:20.746209+0000","last_fullsized":"2024-03-14T09:01:20.746209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:33:59.744353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119687+0000","last_change":"2024-03-14T09:01:18.717256+0000","last_active":"2024-03-14T09:01:20.119687+0000","last_peered":"2024-03-14T09:01:20.119687+0000","last_clean":"2024-03-14T09:01:20.119687+0000","last_became_active":"2024-03-14T09:01:18.716989+0000","last_became_peered":"2024-03-14T09:01:18.716989+0000","last_unstale":"2024-03-14T09:01:20.119687+0000","last_undegraded":"2024-03-14T09:01:20.119687+0000","last_fullsized":"2024-03-14T09:01:20.119687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:10:03.073834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119309+0000","last_change":"2024-03-14T09:01:19.129023+0000","last_active":"2024-03-14T09:01:20.119309+0000","last_peered":"2024-03-14T09:01:20.119309+0000","last_clean":"2024-03-14T09:01:20.119309+0000","last_became_active":"2024-03-14T09:01:19.128683+0000","last_became_peered":"2024-03-14T09:01:19.128683+0000","last_unstale":"2024-03-14T09:01:20.119309+0000","last_undegraded":"2024-03-14T09:01:20.119309+0000","last_fullsized":"2024-03-14T09:01:20.119309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745348+0000","last_change":"2024-03-14T09:01:18.722021+0000","last_active":"2024-03-14T09:01:20.745348+0000","last_peered":"2024-03-14T09:01:20.745348+0000","last_clean":"2024-03-14T09:01:20.745348+0000","last_became_active":"2024-03-14T09:01:18.721794+0000","last_became_peered":"2024-03-14T09:01:18.721794+0000","last_unstale":"2024-03-14T09:01:20.745348+0000","last_undegraded":"2024-03-14T09:01:20.745348+0000","last_fullsized":"2024-03-14T09:01:20.745348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745561+0000","last_change":"2024-03-14T09:01:19.141735+0000","last_active":"2024-03-14T09:01:20.745561+0000","last_peered":"2024-03-14T09:01:20.745561+0000","last_clean":"2024-03-14T09:01:20.745561+0000","last_became_active":"2024-03-14T09:01:19.141460+0000","last_became_peered":"2024-03-14T09:01:19.141460+0000","last_unstale":"2024-03-14T09:01:20.745561+0000","last_undegraded":"2024-03-14T09:01:20.745561+0000","last_fullsized":"2024-03-14T09:01:20.745561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:13:05.273860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744488+0000","last_change":"2024-03-14T09:01:18.717974+0000","last_active":"2024-03-14T09:01:20.744488+0000","last_peered":"2024-03-14T09:01:20.744488+0000","last_clean":"2024-03-14T09:01:20.744488+0000","last_became_active":"2024-03-14T09:01:18.717677+0000","last_became_peered":"2024-03-14T09:01:18.717677+0000","last_unstale":"2024-03-14T09:01:20.744488+0000","last_undegraded":"2024-03-14T09:01:20.744488+0000","last_fullsized":"2024-03-14T09:01:20.744488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.127971+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.127683+0000","last_became_peered":"2024-03-14T09:01:19.127683+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:32:16.291489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743925+0000","last_change":"2024-03-14T09:01:18.727064+0000","last_active":"2024-03-14T09:01:20.743925+0000","last_peered":"2024-03-14T09:01:20.743925+0000","last_clean":"2024-03-14T09:01:20.743925+0000","last_became_active":"2024-03-14T09:01:18.726766+0000","last_became_peered":"2024-03-14T09:01:18.726766+0000","last_unstale":"2024-03-14T09:01:20.743925+0000","last_undegraded":"2024-03-14T09:01:20.743925+0000","last_fullsized":"2024-03-14T09:01:20.743925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745322+0000","last_change":"2024-03-14T09:01:19.136107+0000","last_active":"2024-03-14T09:01:20.745322+0000","last_peered":"2024-03-14T09:01:20.745322+0000","last_clean":"2024-03-14T09:01:20.745322+0000","last_became_active":"2024-03-14T09:01:19.135855+0000","last_became_peered":"2024-03-14T09:01:19.135855+0000","last_unstale":"2024-03-14T09:01:20.745322+0000","last_undegraded":"2024-03-14T09:01:20.745322+0000","last_fullsized":"2024-03-14T09:01:20.745322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123060+0000","last_change":"2024-03-14T09:01:18.717651+0000","last_active":"2024-03-14T09:01:20.123060+0000","last_peered":"2024-03-14T09:01:20.123060+0000","last_clean":"2024-03-14T09:01:20.123060+0000","last_became_active":"2024-03-14T09:01:18.717323+0000","last_became_peered":"2024-03-14T09:01:18.717323+0000","last_unstale":"2024-03-14T09:01:20.123060+0000","last_undegraded":"2024-03-14T09:01:20.123060+0000","last_fullsized":"2024-03-14T09:01:20.123060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:18:35.107355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745513+0000","last_change":"2024-03-14T09:01:19.141404+0000","last_active":"2024-03-14T09:01:20.745513+0000","last_peered":"2024-03-14T09:01:20.745513+0000","last_clean":"2024-03-14T09:01:20.745513+0000","last_became_active":"2024-03-14T09:01:19.141138+0000","last_became_peered":"2024-03-14T09:01:19.141138+0000","last_unstale":"2024-03-14T09:01:20.745513+0000","last_undegraded":"2024-03-14T09:01:20.745513+0000","last_fullsized":"2024-03-14T09:01:20.745513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:50.654277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744508+0000","last_change":"2024-03-14T09:01:18.714145+0000","last_active":"2024-03-14T09:01:20.744508+0000","last_peered":"2024-03-14T09:01:20.744508+0000","last_clean":"2024-03-14T09:01:20.744508+0000","last_became_active":"2024-03-14T09:01:18.713880+0000","last_became_peered":"2024-03-14T09:01:18.713880+0000","last_unstale":"2024-03-14T09:01:20.744508+0000","last_undegraded":"2024-03-14T09:01:20.744508+0000","last_fullsized":"2024-03-14T09:01:20.744508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745392+0000","last_change":"2024-03-14T09:01:19.120923+0000","last_active":"2024-03-14T09:01:20.745392+0000","last_peered":"2024-03-14T09:01:20.745392+0000","last_clean":"2024-03-14T09:01:20.745392+0000","last_became_active":"2024-03-14T09:01:19.120028+0000","last_became_peered":"2024-03-14T09:01:19.120028+0000","last_unstale":"2024-03-14T09:01:20.745392+0000","last_undegraded":"2024-03-14T09:01:20.745392+0000","last_fullsized":"2024-03-14T09:01:20.745392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745276+0000","last_change":"2024-03-14T09:01:18.715127+0000","last_active":"2024-03-14T09:01:20.745276+0000","last_peered":"2024-03-14T09:01:20.745276+0000","last_clean":"2024-03-14T09:01:20.745276+0000","last_became_active":"2024-03-14T09:01:18.714778+0000","last_became_peered":"2024-03-14T09:01:18.714778+0000","last_unstale":"2024-03-14T09:01:20.745276+0000","last_undegraded":"2024-03-14T09:01:20.745276+0000","last_fullsized":"2024-03-14T09:01:20.745276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122941+0000","last_change":"2024-03-14T09:01:19.130468+0000","last_active":"2024-03-14T09:01:20.122941+0000","last_peered":"2024-03-14T09:01:20.122941+0000","last_clean":"2024-03-14T09:01:20.122941+0000","last_became_active":"2024-03-14T09:01:19.130155+0000","last_became_peered":"2024-03-14T09:01:19.130155+0000","last_unstale":"2024-03-14T09:01:20.122941+0000","last_undegraded":"2024-03-14T09:01:20.122941+0000","last_fullsized":"2024-03-14T09:01:20.122941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:37.408944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744060+0000","last_change":"2024-03-14T09:01:19.118942+0000","last_active":"2024-03-14T09:01:20.744060+0000","last_peered":"2024-03-14T09:01:20.744060+0000","last_clean":"2024-03-14T09:01:20.744060+0000","last_became_active":"2024-03-14T09:01:19.118681+0000","last_became_peered":"2024-03-14T09:01:19.118681+0000","last_unstale":"2024-03-14T09:01:20.744060+0000","last_undegraded":"2024-03-14T09:01:20.744060+0000","last_fullsized":"2024-03-14T09:01:20.744060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743928+0000","last_change":"2024-03-14T09:01:18.722916+0000","last_active":"2024-03-14T09:01:20.743928+0000","last_peered":"2024-03-14T09:01:20.743928+0000","last_clean":"2024-03-14T09:01:20.743928+0000","last_became_active":"2024-03-14T09:01:18.722610+0000","last_became_peered":"2024-03-14T09:01:18.722610+0000","last_unstale":"2024-03-14T09:01:20.743928+0000","last_undegraded":"2024-03-14T09:01:20.743928+0000","last_fullsized":"2024-03-14T09:01:20.743928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:33:13.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123315+0000","last_change":"2024-03-14T09:01:19.124237+0000","last_active":"2024-03-14T09:01:20.123315+0000","last_peered":"2024-03-14T09:01:20.123315+0000","last_clean":"2024-03-14T09:01:20.123315+0000","last_became_active":"2024-03-14T09:01:19.123958+0000","last_became_peered":"2024-03-14T09:01:19.123958+0000","last_unstale":"2024-03-14T09:01:20.123315+0000","last_undegraded":"2024-03-14T09:01:20.123315+0000","last_fullsized":"2024-03-14T09:01:20.123315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:17:27.116098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744772+0000","last_change":"2024-03-14T09:01:18.716867+0000","last_active":"2024-03-14T09:01:20.744772+0000","last_peered":"2024-03-14T09:01:20.744772+0000","last_clean":"2024-03-14T09:01:20.744772+0000","last_became_active":"2024-03-14T09:01:18.716352+0000","last_became_peered":"2024-03-14T09:01:18.716352+0000","last_unstale":"2024-03-14T09:01:20.744772+0000","last_undegraded":"2024-03-14T09:01:20.744772+0000","last_fullsized":"2024-03-14T09:01:20.744772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-14T09:01:19.129296+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:19.129296+0000","last_peered":"2024-03-14T09:01:19.129296+0000","last_clean":"2024-03-14T09:01:19.129296+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:19.129296+0000","last_undegraded":"2024-03-14T09:01:19.129296+0000","last_fullsized":"2024-03-14T09:01:19.129296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119237+0000","last_change":"2024-03-14T09:01:19.125813+0000","last_active":"2024-03-14T09:01:20.119237+0000","last_peered":"2024-03-14T09:01:20.119237+0000","last_clean":"2024-03-14T09:01:20.119237+0000","last_became_active":"2024-03-14T09:01:19.125536+0000","last_became_peered":"2024-03-14T09:01:19.125536+0000","last_unstale":"2024-03-14T09:01:20.119237+0000","last_undegraded":"2024-03-14T09:01:20.119237+0000","last_fullsized":"2024-03-14T09:01:20.119237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745288+0000","last_change":"2024-03-14T09:01:18.728259+0000","last_active":"2024-03-14T09:01:20.745288+0000","last_peered":"2024-03-14T09:01:20.745288+0000","last_clean":"2024-03-14T09:01:20.745288+0000","last_became_active":"2024-03-14T09:01:18.728000+0000","last_became_peered":"2024-03-14T09:01:18.728000+0000","last_unstale":"2024-03-14T09:01:20.745288+0000","last_undegraded":"2024-03-14T09:01:20.745288+0000","last_fullsized":"2024-03-14T09:01:20.745288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746270+0000","last_change":"2024-03-14T09:01:18.722659+0000","last_active":"2024-03-14T09:01:20.746270+0000","last_peered":"2024-03-14T09:01:20.746270+0000","last_clean":"2024-03-14T09:01:20.746270+0000","last_became_active":"2024-03-14T09:01:18.722309+0000","last_became_peered":"2024-03-14T09:01:18.722309+0000","last_unstale":"2024-03-14T09:01:20.746270+0000","last_undegraded":"2024-03-14T09:01:20.746270+0000","last_fullsized":"2024-03-14T09:01:20.746270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:56.181811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744400+0000","last_change":"2024-03-14T09:01:19.119183+0000","last_active":"2024-03-14T09:01:20.744400+0000","last_peered":"2024-03-14T09:01:20.744400+0000","last_clean":"2024-03-14T09:01:20.744400+0000","last_became_active":"2024-03-14T09:01:19.118862+0000","last_became_peered":"2024-03-14T09:01:19.118862+0000","last_unstale":"2024-03-14T09:01:20.744400+0000","last_undegraded":"2024-03-14T09:01:20.744400+0000","last_fullsized":"2024-03-14T09:01:20.744400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:22.610280+0000","last_change":"2024-03-14T09:01:18.719534+0000","last_active":"2024-03-14T09:01:22.610280+0000","last_peered":"2024-03-14T09:01:22.610280+0000","last_clean":"2024-03-14T09:01:22.610280+0000","last_became_active":"2024-03-14T09:01:18.719220+0000","last_became_peered":"2024-03-14T09:01:18.719220+0000","last_unstale":"2024-03-14T09:01:22.610280+0000","last_undegraded":"2024-03-14T09:01:22.610280+0000","last_fullsized":"2024-03-14T09:01:22.610280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:04.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123213+0000","last_change":"2024-03-14T09:01:19.135946+0000","last_active":"2024-03-14T09:01:20.123213+0000","last_peered":"2024-03-14T09:01:20.123213+0000","last_clean":"2024-03-14T09:01:20.123213+0000","last_became_active":"2024-03-14T09:01:19.135763+0000","last_became_peered":"2024-03-14T09:01:19.135763+0000","last_unstale":"2024-03-14T09:01:20.123213+0000","last_undegraded":"2024-03-14T09:01:20.123213+0000","last_fullsized":"2024-03-14T09:01:20.123213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745836+0000","last_change":"2024-03-14T09:01:18.723947+0000","last_active":"2024-03-14T09:01:20.745836+0000","last_peered":"2024-03-14T09:01:20.745836+0000","last_clean":"2024-03-14T09:01:20.745836+0000","last_became_active":"2024-03-14T09:01:18.723622+0000","last_became_peered":"2024-03-14T09:01:18.723622+0000","last_unstale":"2024-03-14T09:01:20.745836+0000","last_undegraded":"2024-03-14T09:01:20.745836+0000","last_fullsized":"2024-03-14T09:01:20.745836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119439+0000","last_change":"2024-03-14T09:01:19.121831+0000","last_active":"2024-03-14T09:01:20.119439+0000","last_peered":"2024-03-14T09:01:20.119439+0000","last_clean":"2024-03-14T09:01:20.119439+0000","last_became_active":"2024-03-14T09:01:19.121672+0000","last_became_peered":"2024-03-14T09:01:19.121672+0000","last_unstale":"2024-03-14T09:01:20.119439+0000","last_undegraded":"2024-03-14T09:01:20.119439+0000","last_fullsized":"2024-03-14T09:01:20.119439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744636+0000","last_change":"2024-03-14T09:01:18.725255+0000","last_active":"2024-03-14T09:01:20.744636+0000","last_peered":"2024-03-14T09:01:20.744636+0000","last_clean":"2024-03-14T09:01:20.744636+0000","last_became_active":"2024-03-14T09:01:18.724977+0000","last_became_peered":"2024-03-14T09:01:18.724977+0000","last_unstale":"2024-03-14T09:01:20.744636+0000","last_undegraded":"2024-03-14T09:01:20.744636+0000","last_fullsized":"2024-03-14T09:01:20.744636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:54.443401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122789+0000","last_change":"2024-03-14T09:01:19.127280+0000","last_active":"2024-03-14T09:01:20.122789+0000","last_peered":"2024-03-14T09:01:20.122789+0000","last_clean":"2024-03-14T09:01:20.122789+0000","last_became_active":"2024-03-14T09:01:19.126973+0000","last_became_peered":"2024-03-14T09:01:19.126973+0000","last_unstale":"2024-03-14T09:01:20.122789+0000","last_undegraded":"2024-03-14T09:01:20.122789+0000","last_fullsized":"2024-03-14T09:01:20.122789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744026+0000","last_change":"2024-03-14T09:01:18.719084+0000","last_active":"2024-03-14T09:01:20.744026+0000","last_peered":"2024-03-14T09:01:20.744026+0000","last_clean":"2024-03-14T09:01:20.744026+0000","last_became_active":"2024-03-14T09:01:18.718785+0000","last_became_peered":"2024-03-14T09:01:18.718785+0000","last_unstale":"2024-03-14T09:01:20.744026+0000","last_undegraded":"2024-03-14T09:01:20.744026+0000","last_fullsized":"2024-03-14T09:01:20.744026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:09.672504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122827+0000","last_change":"2024-03-14T09:01:19.123558+0000","last_active":"2024-03-14T09:01:20.122827+0000","last_peered":"2024-03-14T09:01:20.122827+0000","last_clean":"2024-03-14T09:01:20.122827+0000","last_became_active":"2024-03-14T09:01:19.122663+0000","last_became_peered":"2024-03-14T09:01:19.122663+0000","last_unstale":"2024-03-14T09:01:20.122827+0000","last_undegraded":"2024-03-14T09:01:20.122827+0000","last_fullsized":"2024-03-14T09:01:20.122827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744437+0000","last_change":"2024-03-14T09:01:18.717632+0000","last_active":"2024-03-14T09:01:20.744437+0000","last_peered":"2024-03-14T09:01:20.744437+0000","last_clean":"2024-03-14T09:01:20.744437+0000","last_became_active":"2024-03-14T09:01:18.717322+0000","last_became_peered":"2024-03-14T09:01:18.717322+0000","last_unstale":"2024-03-14T09:01:20.744437+0000","last_undegraded":"2024-03-14T09:01:20.744437+0000","last_fullsized":"2024-03-14T09:01:20.744437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:57:36.402497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123124+0000","last_change":"2024-03-14T09:01:19.124078+0000","last_active":"2024-03-14T09:01:20.123124+0000","last_peered":"2024-03-14T09:01:20.123124+0000","last_clean":"2024-03-14T09:01:20.123124+0000","last_became_active":"2024-03-14T09:01:19.123793+0000","last_became_peered":"2024-03-14T09:01:19.123793+0000","last_unstale":"2024-03-14T09:01:20.123124+0000","last_undegraded":"2024-03-14T09:01:20.123124+0000","last_fullsized":"2024-03-14T09:01:20.123124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745643+0000","last_change":"2024-03-14T09:01:18.725304+0000","last_active":"2024-03-14T09:01:20.745643+0000","last_peered":"2024-03-14T09:01:20.745643+0000","last_clean":"2024-03-14T09:01:20.745643+0000","last_became_active":"2024-03-14T09:01:18.725064+0000","last_became_peered":"2024-03-14T09:01:18.725064+0000","last_unstale":"2024-03-14T09:01:20.745643+0000","last_undegraded":"2024-03-14T09:01:20.745643+0000","last_fullsized":"2024-03-14T09:01:20.745643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:53:28.235917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744474+0000","last_change":"2024-03-14T09:01:19.129552+0000","last_active":"2024-03-14T09:01:20.744474+0000","last_peered":"2024-03-14T09:01:20.744474+0000","last_clean":"2024-03-14T09:01:20.744474+0000","last_became_active":"2024-03-14T09:01:19.129287+0000","last_became_peered":"2024-03-14T09:01:19.129287+0000","last_unstale":"2024-03-14T09:01:20.744474+0000","last_undegraded":"2024-03-14T09:01:20.744474+0000","last_fullsized":"2024-03-14T09:01:20.744474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:39:31.143015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744170+0000","last_change":"2024-03-14T09:01:18.721721+0000","last_active":"2024-03-14T09:01:20.744170+0000","last_peered":"2024-03-14T09:01:20.744170+0000","last_clean":"2024-03-14T09:01:20.744170+0000","last_became_active":"2024-03-14T09:01:18.721450+0000","last_became_peered":"2024-03-14T09:01:18.721450+0000","last_unstale":"2024-03-14T09:01:20.744170+0000","last_undegraded":"2024-03-14T09:01:20.744170+0000","last_fullsized":"2024-03-14T09:01:20.744170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:01.528156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746277+0000","last_change":"2024-03-14T09:01:19.142624+0000","last_active":"2024-03-14T09:01:20.746277+0000","last_peered":"2024-03-14T09:01:20.746277+0000","last_clean":"2024-03-14T09:01:20.746277+0000","last_became_active":"2024-03-14T09:01:19.142294+0000","last_became_peered":"2024-03-14T09:01:19.142294+0000","last_unstale":"2024-03-14T09:01:20.746277+0000","last_undegraded":"2024-03-14T09:01:20.746277+0000","last_fullsized":"2024-03-14T09:01:20.746277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744505+0000","last_change":"2024-03-14T09:01:18.714548+0000","last_active":"2024-03-14T09:01:20.744505+0000","last_peered":"2024-03-14T09:01:20.744505+0000","last_clean":"2024-03-14T09:01:20.744505+0000","last_became_active":"2024-03-14T09:01:18.713927+0000","last_became_peered":"2024-03-14T09:01:18.713927+0000","last_unstale":"2024-03-14T09:01:20.744505+0000","last_undegraded":"2024-03-14T09:01:20.744505+0000","last_fullsized":"2024-03-14T09:01:20.744505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:25.554068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745857+0000","last_change":"2024-03-14T09:01:19.139093+0000","last_active":"2024-03-14T09:01:20.745857+0000","last_peered":"2024-03-14T09:01:20.745857+0000","last_clean":"2024-03-14T09:01:20.745857+0000","last_became_active":"2024-03-14T09:01:19.138560+0000","last_became_peered":"2024-03-14T09:01:19.138560+0000","last_unstale":"2024-03-14T09:01:20.745857+0000","last_undegraded":"2024-03-14T09:01:20.745857+0000","last_fullsized":"2024-03-14T09:01:20.745857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:07:11.861699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744326+0000","last_change":"2024-03-14T09:01:18.716783+0000","last_active":"2024-03-14T09:01:20.744326+0000","last_peered":"2024-03-14T09:01:20.744326+0000","last_clean":"2024-03-14T09:01:20.744326+0000","last_became_active":"2024-03-14T09:01:18.716538+0000","last_became_peered":"2024-03-14T09:01:18.716538+0000","last_unstale":"2024-03-14T09:01:20.744326+0000","last_undegraded":"2024-03-14T09:01:20.744326+0000","last_fullsized":"2024-03-14T09:01:20.744326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:17.693398+0000","last_clean_scrub_stamp":"2024-03-14T09:01:17.693398+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:53:06.817782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745724+0000","last_change":"2024-03-14T09:01:19.141337+0000","last_active":"2024-03-14T09:01:20.745724+0000","last_peered":"2024-03-14T09:01:20.745724+0000","last_clean":"2024-03-14T09:01:20.745724+0000","last_became_active":"2024-03-14T09:01:19.141093+0000","last_became_peered":"2024-03-14T09:01:19.141093+0000","last_unstale":"2024-03-14T09:01:20.745724+0000","last_undegraded":"2024-03-14T09:01:20.745724+0000","last_fullsized":"2024-03-14T09:01:20.745724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744540+0000","last_change":"2024-03-14T09:01:19.128035+0000","last_active":"2024-03-14T09:01:20.744540+0000","last_peered":"2024-03-14T09:01:20.744540+0000","last_clean":"2024-03-14T09:01:20.744540+0000","last_became_active":"2024-03-14T09:01:19.127787+0000","last_became_peered":"2024-03-14T09:01:19.127787+0000","last_unstale":"2024-03-14T09:01:20.744540+0000","last_undegraded":"2024-03-14T09:01:20.744540+0000","last_fullsized":"2024-03-14T09:01:20.744540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:35:30.339730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.743991+0000","last_change":"2024-03-14T09:01:19.121494+0000","last_active":"2024-03-14T09:01:20.743991+0000","last_peered":"2024-03-14T09:01:20.743991+0000","last_clean":"2024-03-14T09:01:20.743991+0000","last_became_active":"2024-03-14T09:01:19.120995+0000","last_became_peered":"2024-03-14T09:01:19.120995+0000","last_unstale":"2024-03-14T09:01:20.743991+0000","last_undegraded":"2024-03-14T09:01:20.743991+0000","last_fullsized":"2024-03-14T09:01:20.743991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:58:26.560514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745172+0000","last_change":"2024-03-14T09:01:19.136830+0000","last_active":"2024-03-14T09:01:20.745172+0000","last_peered":"2024-03-14T09:01:20.745172+0000","last_clean":"2024-03-14T09:01:20.745172+0000","last_became_active":"2024-03-14T09:01:19.136568+0000","last_became_peered":"2024-03-14T09:01:19.136568+0000","last_unstale":"2024-03-14T09:01:20.745172+0000","last_undegraded":"2024-03-14T09:01:20.745172+0000","last_fullsized":"2024-03-14T09:01:20.745172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:35:07.292515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119641+0000","last_change":"2024-03-14T09:01:19.118186+0000","last_active":"2024-03-14T09:01:20.119641+0000","last_peered":"2024-03-14T09:01:20.119641+0000","last_clean":"2024-03-14T09:01:20.119641+0000","last_became_active":"2024-03-14T09:01:19.117955+0000","last_became_peered":"2024-03-14T09:01:19.117955+0000","last_unstale":"2024-03-14T09:01:20.119641+0000","last_undegraded":"2024-03-14T09:01:20.119641+0000","last_fullsized":"2024-03-14T09:01:20.119641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:05:28.205081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123167+0000","last_change":"2024-03-14T09:01:19.126685+0000","last_active":"2024-03-14T09:01:20.123167+0000","last_peered":"2024-03-14T09:01:20.123167+0000","last_clean":"2024-03-14T09:01:20.123167+0000","last_became_active":"2024-03-14T09:01:19.126196+0000","last_became_peered":"2024-03-14T09:01:19.126196+0000","last_unstale":"2024-03-14T09:01:20.123167+0000","last_undegraded":"2024-03-14T09:01:20.123167+0000","last_fullsized":"2024-03-14T09:01:20.123167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:55:55.188049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744458+0000","last_change":"2024-03-14T09:01:19.119001+0000","last_active":"2024-03-14T09:01:20.744458+0000","last_peered":"2024-03-14T09:01:20.744458+0000","last_clean":"2024-03-14T09:01:20.744458+0000","last_became_active":"2024-03-14T09:01:19.118755+0000","last_became_peered":"2024-03-14T09:01:19.118755+0000","last_unstale":"2024-03-14T09:01:20.744458+0000","last_undegraded":"2024-03-14T09:01:20.744458+0000","last_fullsized":"2024-03-14T09:01:20.744458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:54.552510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745084+0000","last_change":"2024-03-14T09:01:19.136652+0000","last_active":"2024-03-14T09:01:20.745084+0000","last_peered":"2024-03-14T09:01:20.745084+0000","last_clean":"2024-03-14T09:01:20.745084+0000","last_became_active":"2024-03-14T09:01:19.136252+0000","last_became_peered":"2024-03-14T09:01:19.136252+0000","last_unstale":"2024-03-14T09:01:20.745084+0000","last_undegraded":"2024-03-14T09:01:20.745084+0000","last_fullsized":"2024-03-14T09:01:20.745084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:19:45.286968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.123129+0000","last_change":"2024-03-14T09:01:19.130297+0000","last_active":"2024-03-14T09:01:20.123129+0000","last_peered":"2024-03-14T09:01:20.123129+0000","last_clean":"2024-03-14T09:01:20.123129+0000","last_became_active":"2024-03-14T09:01:19.130024+0000","last_became_peered":"2024-03-14T09:01:19.130024+0000","last_unstale":"2024-03-14T09:01:20.123129+0000","last_undegraded":"2024-03-14T09:01:20.123129+0000","last_fullsized":"2024-03-14T09:01:20.123129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.744787+0000","last_change":"2024-03-14T09:01:19.137747+0000","last_active":"2024-03-14T09:01:20.744787+0000","last_peered":"2024-03-14T09:01:20.744787+0000","last_clean":"2024-03-14T09:01:20.744787+0000","last_became_active":"2024-03-14T09:01:19.137471+0000","last_became_peered":"2024-03-14T09:01:19.137471+0000","last_unstale":"2024-03-14T09:01:20.744787+0000","last_undegraded":"2024-03-14T09:01:20.744787+0000","last_fullsized":"2024-03-14T09:01:20.744787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:12:35.094251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.122916+0000","last_change":"2024-03-14T09:01:19.128038+0000","last_active":"2024-03-14T09:01:20.122916+0000","last_peered":"2024-03-14T09:01:20.122916+0000","last_clean":"2024-03-14T09:01:20.122916+0000","last_became_active":"2024-03-14T09:01:19.127847+0000","last_became_peered":"2024-03-14T09:01:19.127847+0000","last_unstale":"2024-03-14T09:01:20.122916+0000","last_undegraded":"2024-03-14T09:01:20.122916+0000","last_fullsized":"2024-03-14T09:01:20.122916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:39:25.157548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.745074+0000","last_change":"2024-03-14T09:01:19.128566+0000","last_active":"2024-03-14T09:01:20.745074+0000","last_peered":"2024-03-14T09:01:20.745074+0000","last_clean":"2024-03-14T09:01:20.745074+0000","last_became_active":"2024-03-14T09:01:19.127893+0000","last_became_peered":"2024-03-14T09:01:19.127893+0000","last_unstale":"2024-03-14T09:01:20.745074+0000","last_undegraded":"2024-03-14T09:01:20.745074+0000","last_fullsized":"2024-03-14T09:01:20.745074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:01:07.900847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.119666+0000","last_change":"2024-03-14T09:01:19.134739+0000","last_active":"2024-03-14T09:01:20.119666+0000","last_peered":"2024-03-14T09:01:20.119666+0000","last_clean":"2024-03-14T09:01:20.119666+0000","last_became_active":"2024-03-14T09:01:19.134442+0000","last_became_peered":"2024-03-14T09:01:19.134442+0000","last_unstale":"2024-03-14T09:01:20.119666+0000","last_undegraded":"2024-03-14T09:01:20.119666+0000","last_fullsized":"2024-03-14T09:01:20.119666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-14T09:01:20.746583+0000","last_change":"2024-03-14T09:01:19.132199+0000","last_active":"2024-03-14T09:01:20.746583+0000","last_peered":"2024-03-14T09:01:20.746583+0000","last_clean":"2024-03-14T09:01:20.746583+0000","last_became_active":"2024-03-14T09:01:19.131887+0000","last_became_peered":"2024-03-14T09:01:19.131887+0000","last_unstale":"2024-03-14T09:01:20.746583+0000","last_undegraded":"2024-03-14T09:01:20.746583+0000","last_fullsized":"2024-03-14T09:01:20.746583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:18.699844+0000","last_clean_scrub_stamp":"2024-03-14T09:01:18.699844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:33:29.023582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:01:28.443 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T09:01:28.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-14T09:01:28.443 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-14T09:01:28.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-14T09:01:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.602+0000 7fa66cd5a700 1 Processor -- start 2024-03-14T09:01:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.602+0000 7fa66cd5a700 1 -- start start 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66cd5a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa66814eaf0 0x7fa66806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66cd5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66806acb0 0x7fa668155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66cd5a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6680ac8e0 con 0x7fa668156330 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66cd5a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66806b310 con 0x7fa66814eaf0 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66cd5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66806de30 con 0x7fa66806acb0 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa66814eaf0 0x7fa66806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa66814eaf0 0x7fa66806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49026/0 (socket says 172.21.15.154:49026) 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 -- 172.21.15.154:0/960275660 learned_addr learned my addr 172.21.15.154:0/960275660 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa665d9b700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66806acb0 0x7fa668155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 -- 172.21.15.154:0/960275660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66806acb0 msgr2=0x7fa668155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66806acb0 0x7fa668155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 -- 172.21.15.154:0/960275660 >> v1:172.21.15.191:6789/0 conn(0x7fa668156330 legacy=0x7fa668157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 -- 172.21.15.154:0/960275660 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6680acb30 con 0x7fa66814eaf0 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66659c700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa66806a770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6580098f0 tx=0x7fa658009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6078c43baf54d6e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66559a700 1 -- 172.21.15.154:0/960275660 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800b8a0 con 0x7fa66814eaf0 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66559a700 1 -- 172.21.15.154:0/960275660 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65800ba00 con 0x7fa66814eaf0 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.603+0000 7fa66559a700 1 -- 172.21.15.154:0/960275660 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800bd90 con 0x7fa66814eaf0 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 -- 172.21.15.154:0/960275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 msgr2=0x7fa66806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa66806a770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6580098f0 tx=0x7fa658009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 -- 172.21.15.154:0/960275660 shutdown_connections 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa66806a770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/960275660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66806acb0 0x7fa668155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 -- 172.21.15.154:0/960275660 >> 172.21.15.154:0/960275660 conn(0x7fa6680c3880 msgr2=0x7fa6680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 -- 172.21.15.154:0/960275660 shutdown_connections 2024-03-14T09:01:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.604+0000 7fa66cd5a700 1 -- 172.21.15.154:0/960275660 wait complete. 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 Processor -- start 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 -- start start 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66806acb0 0x7fa668156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 0x7fa6680c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668152f70 con 0x7fa66814eaf0 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa668152b20 con 0x7fa66806acb0 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.606+0000 7fa66cd5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6680c17b0 con 0x7fa668156330 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49038/0 (socket says 172.21.15.154:49038) 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 -- 172.21.15.154:0/2440117706 learned_addr learned my addr 172.21.15.154:0/2440117706 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa66659c700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66806acb0 0x7fa668156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa666d9d700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 0x7fa6680c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 -- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 msgr2=0x7fa6680c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 0x7fa6680c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 -- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66806acb0 msgr2=0x7fa668156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66806acb0 0x7fa668156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 -- 172.21.15.154:0/2440117706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658009550 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa666d9d700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 0x7fa6680c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa665d9b700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00d830 tx=0x7fa65c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c0125f0 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c012750 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6680c1510 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c012b00 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.607+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6680c4d20 con 0x7fa66814eaf0 2024-03-14T09:01:28.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.608+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6680c4f10 con 0x7fa66814eaf0 2024-03-14T09:01:28.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.609+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa65c012c60 con 0x7fa66814eaf0 2024-03-14T09:01:28.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.609+0000 7fa656ffd700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 0x7fa64806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.609+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa65c089060 con 0x7fa66814eaf0 2024-03-14T09:01:28.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.609+0000 7fa66659c700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 0x7fa64806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.610+0000 7fa66659c700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 0x7fa64806cd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa658014040 tx=0x7fa658008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.613+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa65c054490 con 0x7fa66814eaf0 2024-03-14T09:01:28.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.849+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa6680697d0 con 0x7fa66814eaf0 2024-03-14T09:01:28.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.850+0000 7fa656ffd700 1 -- 172.21.15.154:0/2440117706 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa65c057ab0 con 0x7fa66814eaf0 2024-03-14T09:01:28.850 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:28.850 INFO:teuthology.orchestra.run.smithi154.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 msgr2=0x7fa64806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 0x7fa64806cd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa658014040 tx=0x7fa658008030 comp rx=0 tx=0).stop 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 msgr2=0x7fa6680c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00d830 tx=0x7fa65c00db40 comp rx=0 tx=0).stop 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 shutdown_connections 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa668156330 0x7fa6680c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66806acb0 0x7fa668156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa64806a860 0x7fa64806cd20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 --2- 172.21.15.154:0/2440117706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa66814eaf0 0x7fa6680c5240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 >> 172.21.15.154:0/2440117706 conn(0x7fa6680c3880 msgr2=0x7fa668148e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 shutdown_connections 2024-03-14T09:01:28.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.852+0000 7fa66cd5a700 1 -- 172.21.15.154:0/2440117706 wait complete. 2024-03-14T09:01:28.863 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-14T09:01:28.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:28.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.985+0000 7f4551e9e700 1 Processor -- start 2024-03-14T09:01:28.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.985+0000 7f4551e9e700 1 -- start start 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f4551e9e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f454c158840 0x7f454c0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f4551e9e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f454c0bf9c0 0x7f454c0bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f4551e9e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454c154f40 con 0x7f454c1553d0 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f4551e9e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c154930 con 0x7f454c0bf9c0 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f4551e9e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0c57e0 con 0x7f454c158840 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f454c0bf9c0 0x7f454c0bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f454c0bf9c0 0x7f454c0bfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49052/0 (socket says 172.21.15.154:49052) 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 -- 172.21.15.154:0/3224599810 learned_addr learned my addr 172.21.15.154:0/3224599810 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454affd700 1 --2- 172.21.15.154:0/3224599810 >> v2:172.21.15.191:3300/0 conn(0x7f454c158840 0x7f454c0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 -- 172.21.15.154:0/3224599810 >> v2:172.21.15.191:3300/0 conn(0x7f454c158840 msgr2=0x7f454c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 --2- 172.21.15.154:0/3224599810 >> v2:172.21.15.191:3300/0 conn(0x7f454c158840 0x7f454c0c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 -- 172.21.15.154:0/3224599810 >> v1:172.21.15.191:6789/0 conn(0x7f454c1553d0 legacy=0x7f454c0c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.986+0000 7f454bfff700 1 -- 172.21.15.154:0/3224599810 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0c5ac0 con 0x7f454c0bf9c0 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f454bfff700 1 --2- 172.21.15.154:0/3224599810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c0bf9c0 0x7f454c0bfe50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f454c062450 tx=0x7f454400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a061763c3d3f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f454a7fc700 1 -- 172.21.15.154:0/3224599810 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400e070 con 0x7f454c0bf9c0 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f454a7fc700 1 -- 172.21.15.154:0/3224599810 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400b910 con 0x7f454c0bf9c0 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f454a7fc700 1 -- 172.21.15.154:0/3224599810 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400bcc0 con 0x7f454c0bf9c0 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 -- 172.21.15.154:0/3224599810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c0bf9c0 msgr2=0x7f454c0bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 --2- 172.21.15.154:0/3224599810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c0bf9c0 0x7f454c0bfe50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f454c062450 tx=0x7f454400b330 comp rx=0 tx=0).stop 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 -- 172.21.15.154:0/3224599810 shutdown_connections 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 --2- 172.21.15.154:0/3224599810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c0bf9c0 0x7f454c0bfe50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 --2- 172.21.15.154:0/3224599810 >> v2:172.21.15.191:3300/0 conn(0x7f454c158840 0x7f454c0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 -- 172.21.15.154:0/3224599810 >> 172.21.15.154:0/3224599810 conn(0x7f454c06a770 msgr2=0x7f454c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 -- 172.21.15.154:0/3224599810 shutdown_connections 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.987+0000 7f4551e9e700 1 -- 172.21.15.154:0/3224599810 wait complete. 2024-03-14T09:01:28.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.988+0000 7f4551e9e700 1 Processor -- start 2024-03-14T09:01:28.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 -- start start 2024-03-14T09:01:28.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 0x7f454c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f454c158840 0x7f454c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c1574e0 con 0x7f454c1553d0 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c14ba50 con 0x7f454c0bf9c0 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f4551e9e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0abfc0 con 0x7f454c158840 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.989+0000 7f454affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49064/0 (socket says 172.21.15.154:49064) 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 -- 172.21.15.154:0/1732199870 learned_addr learned my addr 172.21.15.154:0/1732199870 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454bfff700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f454c158840 0x7f454c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454b7fe700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 0x7f454c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 -- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f454c158840 msgr2=0x7f454c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f454c158840 0x7f454c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 -- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 msgr2=0x7f454c156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 0x7f454c156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 -- 172.21.15.154:0/1732199870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544009b30 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454b7fe700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 0x7f454c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f454affd700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9b0 tx=0x7f453400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534009710 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400c850 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0ac2a0 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400cbc0 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.990+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c0c0590 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.991+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0697d0 con 0x7f454c1553d0 2024-03-14T09:01:28.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.991+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f453400cd20 con 0x7f454c1553d0 2024-03-14T09:01:28.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.992+0000 7f4543fff700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 0x7f452c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:28.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.992+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4534088e80 con 0x7f454c1553d0 2024-03-14T09:01:28.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.992+0000 7f454b7fe700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 0x7f452c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:28.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.992+0000 7f454b7fe700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 0x7f452c06ccf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f454c158020 tx=0x7f453c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:28.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:28.996+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4534054330 con 0x7f454c1553d0 2024-03-14T09:01:29.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.192+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f454c14e9d0 con 0x7f454c1553d0 2024-03-14T09:01:29.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.193+0000 7f4543fff700 1 -- 172.21.15.154:0/1732199870 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f4534057950 con 0x7f454c1553d0 2024-03-14T09:01:29.193 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:29.193 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:29.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.195+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 msgr2=0x7f452c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.195+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 0x7f452c06ccf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f454c158020 tx=0x7f453c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.195+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 msgr2=0x7f454c14ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.195+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9b0 tx=0x7f453400ed70 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 shutdown_connections 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f454c158840 0x7f454c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f454c0bf9c0 0x7f454c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f452c06a830 0x7f452c06ccf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 --2- 172.21.15.154:0/1732199870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c1553d0 0x7f454c14ac60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 >> 172.21.15.154:0/1732199870 conn(0x7f454c06a770 msgr2=0x7f454c1538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 shutdown_connections 2024-03-14T09:01:29.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.196+0000 7f4551e9e700 1 -- 172.21.15.154:0/1732199870 wait complete. 2024-03-14T09:01:29.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:29.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:29.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 Processor -- start 2024-03-14T09:01:29.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 -- start start 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f661c06d3c0 0x7f661c0c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c151860 con 0x7f661c0c1d90 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c150f30 con 0x7f661c0c1830 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6623ef8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c151210 con 0x7f661c06d3c0 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6621493700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f661c0c1d90 0x7f661c0c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52530/0 (socket says 172.21.15.154:52530) 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.333+0000 7f6621493700 1 -- 172.21.15.154:0/387414492 learned_addr learned my addr 172.21.15.154:0/387414492 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6621c94700 1 --2- 172.21.15.154:0/387414492 >> v2:172.21.15.191:3300/0 conn(0x7f661c06d3c0 0x7f661c0c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6620c92700 1 -- 172.21.15.154:0/387414492 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170223956 0 0) 0x7f661c151860 con 0x7f661c0c1d90 2024-03-14T09:01:29.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6620c92700 1 -- 172.21.15.154:0/387414492 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66080046c0 con 0x7f661c0c1d90 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6621c94700 1 -- 172.21.15.154:0/387414492 >> v1:172.21.15.191:6789/0 conn(0x7f661c0c1830 legacy=0x7f661c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6621c94700 1 -- 172.21.15.154:0/387414492 >> v1:172.21.15.154:6789/0 conn(0x7f661c0c1d90 legacy=0x7f661c0c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6621c94700 1 -- 172.21.15.154:0/387414492 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c1514f0 con 0x7f661c06d3c0 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6621c94700 1 --2- 172.21.15.154:0/387414492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c0c12f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6610009870 tx=0x7f6610009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d663aab5f92de702 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6620c92700 1 -- 172.21.15.154:0/387414492 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011830 con 0x7f661c06d3c0 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6620c92700 1 -- 172.21.15.154:0/387414492 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6610011990 con 0x7f661c06d3c0 2024-03-14T09:01:29.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.334+0000 7f6620c92700 1 -- 172.21.15.154:0/387414492 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011d20 con 0x7f661c06d3c0 2024-03-14T09:01:29.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 -- 172.21.15.154:0/387414492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 msgr2=0x7f661c0c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 --2- 172.21.15.154:0/387414492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c0c12f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6610009870 tx=0x7f6610009b80 comp rx=0 tx=0).stop 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 -- 172.21.15.154:0/387414492 shutdown_connections 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 --2- 172.21.15.154:0/387414492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c0c12f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 -- 172.21.15.154:0/387414492 >> 172.21.15.154:0/387414492 conn(0x7f661c06a770 msgr2=0x7f661c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 -- 172.21.15.154:0/387414492 shutdown_connections 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.335+0000 7f6623ef8700 1 -- 172.21.15.154:0/387414492 wait complete. 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.336+0000 7f6623ef8700 1 Processor -- start 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.336+0000 7f6623ef8700 1 -- start start 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f661c0c1d90 0x7f661c1ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c148780 con 0x7f661c0c1830 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c1534a0 con 0x7f661c06d3c0 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c153780 con 0x7f661c0c1d90 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33514/0 (socket says 172.21.15.154:33514) 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 -- 172.21.15.154:0/2636714654 learned_addr learned my addr 172.21.15.154:0/2636714654 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621c94700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6622495700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f661c0c1d90 0x7f661c1ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 -- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f661c0c1d90 msgr2=0x7f661c1ed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f661c0c1d90 0x7f661c1ed4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 -- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 msgr2=0x7f661c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 -- 172.21.15.154:0/2636714654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100094a0 con 0x7f661c0c1830 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6621493700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6618001eb0 tx=0x7f661800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618005d70 con 0x7f661c0c1830 2024-03-14T09:01:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66180070d0 con 0x7f661c0c1830 2024-03-14T09:01:29.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c153a60 con 0x7f661c0c1830 2024-03-14T09:01:29.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618007460 con 0x7f661c0c1830 2024-03-14T09:01:29.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.337+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c154020 con 0x7f661c0c1830 2024-03-14T09:01:29.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.338+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c072e10 con 0x7f661c0c1830 2024-03-14T09:01:29.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.339+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f66180075c0 con 0x7f661c0c1830 2024-03-14T09:01:29.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.339+0000 7f660e7fc700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 0x7f660006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.339+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6618088190 con 0x7f661c0c1830 2024-03-14T09:01:29.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.339+0000 7f6621c94700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 0x7f660006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.340+0000 7f6621c94700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 0x7f660006ccf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.344+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f66180535b0 con 0x7f661c0c1830 2024-03-14T09:01:29.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.534+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f661c150b70 con 0x7f661c0c1830 2024-03-14T09:01:29.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.534+0000 7f660e7fc700 1 -- 172.21.15.154:0/2636714654 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6618056bd0 con 0x7f661c0c1830 2024-03-14T09:01:29.535 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:29.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 msgr2=0x7f660006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 0x7f660006ccf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6610009840 tx=0x7f6610008030 comp rx=0 tx=0).stop 2024-03-14T09:01:29.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 msgr2=0x7f661c14a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6618001eb0 tx=0x7f661800c7b0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 shutdown_connections 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f661c0c1d90 0x7f661c1ed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f661c06d3c0 0x7f661c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f660006a830 0x7f660006ccf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 --2- 172.21.15.154:0/2636714654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0c1830 0x7f661c14a5c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.536+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 >> 172.21.15.154:0/2636714654 conn(0x7f661c06a770 msgr2=0x7f661c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.537+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 shutdown_connections 2024-03-14T09:01:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.537+0000 7f6623ef8700 1 -- 172.21.15.154:0/2636714654 wait complete. 2024-03-14T09:01:29.548 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.675+0000 7f9e570e3700 1 Processor -- start 2024-03-14T09:01:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.675+0000 7f9e570e3700 1 -- start start 2024-03-14T09:01:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e570e3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e500702f0 0x7f9e500c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e570e3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e500acd10 con 0x7f9e5014fbd0 2024-03-14T09:01:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e570e3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e50150050 con 0x7f9e500c5d60 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e570e3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e500ac7b0 con 0x7f9e500702f0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e54e7f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e500702f0 0x7f9e500c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e4ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9e5014fbd0 0x7f9e500c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52536/0 (socket says 172.21.15.154:52536) 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.676+0000 7f9e4ffff700 1 -- 172.21.15.154:0/3062064135 learned_addr learned my addr 172.21.15.154:0/3062064135 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e4f7fe700 1 -- 172.21.15.154:0/3062064135 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182428000 0 0) 0x7f9e500acd10 con 0x7f9e5014fbd0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e4f7fe700 1 -- 172.21.15.154:0/3062064135 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e400035b0 con 0x7f9e5014fbd0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e54e7f700 1 -- 172.21.15.154:0/3062064135 >> v1:172.21.15.191:6789/0 conn(0x7f9e500c5d60 legacy=0x7f9e5015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e54e7f700 1 -- 172.21.15.154:0/3062064135 >> v1:172.21.15.154:6789/0 conn(0x7f9e5014fbd0 legacy=0x7f9e500c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e54e7f700 1 -- 172.21.15.154:0/3062064135 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e500ac9c0 con 0x7f9e500702f0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e54e7f700 1 --2- 172.21.15.154:0/3062064135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e500c4380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009840 tx=0x7f9e44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf873f119293860 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e4f7fe700 1 -- 172.21.15.154:0/3062064135 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004230 con 0x7f9e500702f0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e4f7fe700 1 -- 172.21.15.154:0/3062064135 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e44004390 con 0x7f9e500702f0 2024-03-14T09:01:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e4f7fe700 1 -- 172.21.15.154:0/3062064135 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440036a0 con 0x7f9e500702f0 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e570e3700 1 -- 172.21.15.154:0/3062064135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 msgr2=0x7f9e500c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.677+0000 7f9e570e3700 1 --2- 172.21.15.154:0/3062064135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e500c4380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009840 tx=0x7f9e44009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.678+0000 7f9e570e3700 1 -- 172.21.15.154:0/3062064135 shutdown_connections 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.678+0000 7f9e570e3700 1 --2- 172.21.15.154:0/3062064135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e500c4380 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.678+0000 7f9e570e3700 1 -- 172.21.15.154:0/3062064135 >> 172.21.15.154:0/3062064135 conn(0x7f9e500a9bf0 msgr2=0x7f9e500a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.678+0000 7f9e570e3700 1 -- 172.21.15.154:0/3062064135 shutdown_connections 2024-03-14T09:01:29.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.678+0000 7f9e570e3700 1 -- 172.21.15.154:0/3062064135 wait complete. 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.679+0000 7f9e570e3700 1 Processor -- start 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.679+0000 7f9e570e3700 1 -- start start 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 0x7f9e5006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e5014fbd0 0x7f9e5006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501ebf30 con 0x7f9e500702f0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50142d60 con 0x7f9e500c5d60 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e570e3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5006df80 con 0x7f9e5014fbd0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33528/0 (socket says 172.21.15.154:33528) 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 -- 172.21.15.154:0/1615933129 learned_addr learned my addr 172.21.15.154:0/1615933129 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e4ffff700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 0x7f9e5006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e55680700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e5014fbd0 0x7f9e5006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 -- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e5014fbd0 msgr2=0x7f9e5006ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e5014fbd0 0x7f9e5006ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 -- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 msgr2=0x7f9e5006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 0x7f9e5006e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e54e7f700 1 -- 172.21.15.154:0/1615933129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440094a0 con 0x7f9e500702f0 2024-03-14T09:01:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.680+0000 7f9e4ffff700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 0x7f9e5006e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e54e7f700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9e5006b710 tx=0x7f9e44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011630 con 0x7f9e500702f0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4401e070 con 0x7f9e500702f0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e50150b70 con 0x7f9e500702f0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440119b0 con 0x7f9e500702f0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.681+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e50150e50 con 0x7f9e500702f0 2024-03-14T09:01:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.682+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e500697d0 con 0x7f9e500702f0 2024-03-14T09:01:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.682+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9e44011b10 con 0x7f9e500702f0 2024-03-14T09:01:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.682+0000 7f9e4d7fa700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 0x7f9e3406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.683+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e44089f00 con 0x7f9e500702f0 2024-03-14T09:01:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.683+0000 7f9e4ffff700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 0x7f9e3406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.683+0000 7f9e4ffff700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 0x7f9e3406ccf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e50151260 tx=0x7f9e3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.688+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9e440553b0 con 0x7f9e500702f0 2024-03-14T09:01:29.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.879+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e500c7160 con 0x7f9e500702f0 2024-03-14T09:01:29.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.879+0000 7f9e4d7fa700 1 -- 172.21.15.154:0/1615933129 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e44020020 con 0x7f9e500702f0 2024-03-14T09:01:29.879 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:29.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 msgr2=0x7f9e3406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 0x7f9e3406ccf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e50151260 tx=0x7f9e3c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 msgr2=0x7f9e501eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9e5006b710 tx=0x7f9e44004750 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 shutdown_connections 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e5014fbd0 0x7f9e5006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9e3406a830 0x7f9e3406ccf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e500702f0 0x7f9e501eb420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.881+0000 7f9e570e3700 1 --2- 172.21.15.154:0/1615933129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e500c5d60 0x7f9e5006e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.882+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 >> 172.21.15.154:0/1615933129 conn(0x7f9e500a9bf0 msgr2=0x7f9e500c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.882+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 shutdown_connections 2024-03-14T09:01:29.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:29.882+0000 7f9e570e3700 1 -- 172.21.15.154:0/1615933129 wait complete. 2024-03-14T09:01:29.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:30.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 Processor -- start 2024-03-14T09:01:30.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 -- start start 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4014eaf0 0x7f3c4006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c4006acb0 0x7f3c40155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400ac8e0 con 0x7f3c40156330 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4006b310 con 0x7f3c4006acb0 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.018+0000 7f3c46ae7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4006de30 con 0x7f3c4014eaf0 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c4006acb0 0x7f3c40155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3c4006acb0 0x7f3c40155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33532/0 (socket says 172.21.15.154:33532) 2024-03-14T09:01:30.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1887390774 learned_addr learned my addr 172.21.15.154:0/1887390774 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1887390774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4014eaf0 msgr2=0x7f3c4006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c44883700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4014eaf0 0x7f3c4006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4014eaf0 0x7f3c4006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1887390774 >> v1:172.21.15.191:6789/0 conn(0x7f3c40156330 legacy=0x7f3c40157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1887390774 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400acb30 con 0x7f3c4006acb0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3ffff700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4006acb0 0x7f3c40155df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098f0 tx=0x7f3c2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1059771001c9ed0c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3f7fe700 1 -- 172.21.15.154:0/1887390774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004230 con 0x7f3c4006acb0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3f7fe700 1 -- 172.21.15.154:0/1887390774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c004390 con 0x7f3c4006acb0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.019+0000 7f3c3f7fe700 1 -- 172.21.15.154:0/1887390774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0036a0 con 0x7f3c4006acb0 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1887390774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4006acb0 msgr2=0x7f3c40155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4006acb0 0x7f3c40155df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0098f0 tx=0x7f3c2c009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1887390774 shutdown_connections 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4014eaf0 0x7f3c4006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1887390774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4006acb0 0x7f3c40155df0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1887390774 >> 172.21.15.154:0/1887390774 conn(0x7f3c400c3880 msgr2=0x7f3c400c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1887390774 shutdown_connections 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.020+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1887390774 wait complete. 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.021+0000 7f3c46ae7700 1 Processor -- start 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.021+0000 7f3c46ae7700 1 -- start start 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4006acb0 0x7f3c400c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c40156330 0x7f3c401ee090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c3630 con 0x7f3c4014eaf0 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c3070 con 0x7f3c40156330 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c3350 con 0x7f3c4006acb0 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33540/0 (socket says 172.21.15.154:33540) 2024-03-14T09:01:30.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1303449318 learned_addr learned my addr 172.21.15.154:0/1303449318 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4006acb0 msgr2=0x7f3c400c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4006acb0 0x7f3c400c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c40156330 msgr2=0x7f3c401ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c45084700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c40156330 0x7f3c401ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c40156330 0x7f3c401ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 -- 172.21.15.154:0/1303449318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009550 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3ffff700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009130 tx=0x7f3c2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00b860 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00b9c0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c401ed890 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00bd70 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.022+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4014a7c0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.023+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c401eddb0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.024+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3c2c01b440 con 0x7f3c4014eaf0 2024-03-14T09:01:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.024+0000 7f3c3d7fa700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 0x7f3c2406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.024+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c2c08a6c0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.024+0000 7f3c44883700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 0x7f3c2406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.025+0000 7f3c44883700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 0x7f3c2406ccf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400a8d0 tx=0x7f3c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.030+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3c401eddb0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.222+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c401eddb0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.222+0000 7f3c3d7fa700 1 -- 172.21.15.154:0/1303449318 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c401eddb0 con 0x7f3c4014eaf0 2024-03-14T09:01:30.221 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 msgr2=0x7f3c2406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 0x7f3c2406ccf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400a8d0 tx=0x7f3c34008040 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 msgr2=0x7f3c4014a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009130 tx=0x7f3c2c004750 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 shutdown_connections 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c40156330 0x7f3c401ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c4006acb0 0x7f3c400c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3c2406a830 0x7f3c2406ccf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 --2- 172.21.15.154:0/1303449318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c4014eaf0 0x7f3c4014a280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 >> 172.21.15.154:0/1303449318 conn(0x7f3c400c3880 msgr2=0x7f3c40155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 shutdown_connections 2024-03-14T09:01:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.224+0000 7f3c46ae7700 1 -- 172.21.15.154:0/1303449318 wait complete. 2024-03-14T09:01:30.235 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.356+0000 7fe73010c700 1 Processor -- start 2024-03-14T09:01:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.357+0000 7fe73010c700 1 -- start start 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.357+0000 7fe73010c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7280c4a80 0x7fe7280c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.357+0000 7fe73010c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72814ed60 con 0x7fe728158370 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.357+0000 7fe73010c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280c6cf0 con 0x7fe72814e980 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.357+0000 7fe73010c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7280c6b10 con 0x7fe7280c4a80 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72d6a7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7280c4a80 0x7fe7280c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72e6a9700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe728158370 0x7fe728158780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52544/0 (socket says 172.21.15.154:52544) 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72e6a9700 1 -- 172.21.15.154:0/2352691371 learned_addr learned my addr 172.21.15.154:0/2352691371 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72d6a7700 1 -- 172.21.15.154:0/2352691371 >> v1:172.21.15.191:6789/0 conn(0x7fe72814e980 legacy=0x7fe7281552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72d6a7700 1 -- 172.21.15.154:0/2352691371 >> v1:172.21.15.154:6789/0 conn(0x7fe728158370 legacy=0x7fe728158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72d6a7700 1 -- 172.21.15.154:0/2352691371 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280c6fd0 con 0x7fe7280c4a80 2024-03-14T09:01:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72cea6700 1 -- 172.21.15.154:0/2352691371 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258523849 0 0) 0x7fe72814ed60 con 0x7fe728158370 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72d6a7700 1 --2- 172.21.15.154:0/2352691371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7280c4f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe724009840 tx=0x7fe724009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46d7774cbc87f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72cea6700 1 -- 172.21.15.154:0/2352691371 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724011830 con 0x7fe7280c4a80 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72cea6700 1 -- 172.21.15.154:0/2352691371 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724011990 con 0x7fe7280c4a80 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.358+0000 7fe72cea6700 1 -- 172.21.15.154:0/2352691371 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724011d20 con 0x7fe7280c4a80 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 -- 172.21.15.154:0/2352691371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 msgr2=0x7fe7280c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 --2- 172.21.15.154:0/2352691371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7280c4f10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe724009840 tx=0x7fe724009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 -- 172.21.15.154:0/2352691371 shutdown_connections 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 --2- 172.21.15.154:0/2352691371 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7280c4f10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 -- 172.21.15.154:0/2352691371 >> 172.21.15.154:0/2352691371 conn(0x7fe7280abcf0 msgr2=0x7fe7280ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 -- 172.21.15.154:0/2352691371 shutdown_connections 2024-03-14T09:01:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.359+0000 7fe73010c700 1 -- 172.21.15.154:0/2352691371 wait complete. 2024-03-14T09:01:30.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.360+0000 7fe73010c700 1 Processor -- start 2024-03-14T09:01:30.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.360+0000 7fe73010c700 1 -- start start 2024-03-14T09:01:30.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe72814e980 0x7fe7281eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe728158370 0x7fe728153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281eb310 con 0x7fe7280c4a80 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280c07f0 con 0x7fe728158370 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280c0ad0 con 0x7fe72814e980 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33560/0 (socket says 172.21.15.154:33560) 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 -- 172.21.15.154:0/784018095 learned_addr learned my addr 172.21.15.154:0/784018095 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72e6a9700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe728158370 0x7fe728153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72d6a7700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe72814e980 0x7fe7281eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 -- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe72814e980 msgr2=0x7fe7281eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe72814e980 0x7fe7281eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 -- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe728158370 msgr2=0x7fe728153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe728158370 0x7fe728153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 -- 172.21.15.154:0/784018095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240094a0 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe72dea8700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c4e0 tx=0x7fe71c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0052d0 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c005430 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0038c0 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7280bdca0 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.361+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7280be260 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.362+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7280697d0 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.363+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe71c003a20 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.363+0000 7fe71a7fc700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 0x7fe70c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.363+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe71c088100 con 0x7fe7280c4a80 2024-03-14T09:01:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.363+0000 7fe72d6a7700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 0x7fe70c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.364+0000 7fe72d6a7700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 0x7fe70c06ccf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe7240090b0 tx=0x7fe72401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.368+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe71c053520 con 0x7fe7280c4a80 2024-03-14T09:01:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.603+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe72814cb80 con 0x7fe7280c4a80 2024-03-14T09:01:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.604+0000 7fe71a7fc700 1 -- 172.21.15.154:0/784018095 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fe71c056b40 con 0x7fe7280c4a80 2024-03-14T09:01:30.603 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:30.603 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 msgr2=0x7fe70c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 0x7fe70c06ccf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe7240090b0 tx=0x7fe72401c040 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 msgr2=0x7fe7281ddec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00c4e0 tx=0x7fe71c00c7f0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 shutdown_connections 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe728158370 0x7fe728153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe70c06a830 0x7fe70c06ccf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7280c4a80 0x7fe7281ddec0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 --2- 172.21.15.154:0/784018095 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe72814e980 0x7fe7281eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 >> 172.21.15.154:0/784018095 conn(0x7fe7280abcf0 msgr2=0x7fe7281550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 shutdown_connections 2024-03-14T09:01:30.606 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.606+0000 7fe73010c700 1 -- 172.21.15.154:0/784018095 wait complete. 2024-03-14T09:01:30.606 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:30.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:30.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.740+0000 7fddbd8e7700 1 Processor -- start 2024-03-14T09:01:30.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.740+0000 7fddbd8e7700 1 -- start start 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddbd8e7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fddb80c4af0 0x7fddb80c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddbd8e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fddb815d370 0x7fddb815f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddbd8e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb80c55e0 con 0x7fddb806b300 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddbd8e7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb80c5ac0 con 0x7fddb80c4af0 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddbd8e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb80c8290 con 0x7fddb815d370 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fddb80c4af0 0x7fddb80c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fddb80c4af0 0x7fddb80c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33576/0 (socket says 172.21.15.154:33576) 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 -- 172.21.15.154:0/1883132408 learned_addr learned my addr 172.21.15.154:0/1883132408 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb77fe700 1 --2- 172.21.15.154:0/1883132408 >> v2:172.21.15.191:3300/0 conn(0x7fddb815d370 0x7fddb815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 -- 172.21.15.154:0/1883132408 >> v2:172.21.15.191:3300/0 conn(0x7fddb815d370 msgr2=0x7fddb815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 --2- 172.21.15.154:0/1883132408 >> v2:172.21.15.191:3300/0 conn(0x7fddb815d370 0x7fddb815f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 -- 172.21.15.154:0/1883132408 >> v1:172.21.15.191:6789/0 conn(0x7fddb806b300 legacy=0x7fddb806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 -- 172.21.15.154:0/1883132408 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8154bf0 con 0x7fddb80c4af0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb6ffd700 1 --2- 172.21.15.154:0/1883132408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb80c4af0 0x7fddb80c4ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddb80c6ba0 tx=0x7fddac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec895374a56a81f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb5ffb700 1 -- 172.21.15.154:0/1883132408 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac004230 con 0x7fddb80c4af0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb5ffb700 1 -- 172.21.15.154:0/1883132408 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac004390 con 0x7fddb80c4af0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.741+0000 7fddb5ffb700 1 -- 172.21.15.154:0/1883132408 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac0036a0 con 0x7fddb80c4af0 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 -- 172.21.15.154:0/1883132408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb80c4af0 msgr2=0x7fddb80c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/1883132408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb80c4af0 0x7fddb80c4ed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddb80c6ba0 tx=0x7fddac009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 -- 172.21.15.154:0/1883132408 shutdown_connections 2024-03-14T09:01:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/1883132408 >> v2:172.21.15.191:3300/0 conn(0x7fddb815d370 0x7fddb815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/1883132408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb80c4af0 0x7fddb80c4ed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 -- 172.21.15.154:0/1883132408 >> 172.21.15.154:0/1883132408 conn(0x7fddb80abc60 msgr2=0x7fddb80ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 -- 172.21.15.154:0/1883132408 shutdown_connections 2024-03-14T09:01:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.742+0000 7fddbd8e7700 1 -- 172.21.15.154:0/1883132408 wait complete. 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.743+0000 7fddbd8e7700 1 Processor -- start 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.743+0000 7fddbd8e7700 1 -- start start 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb806b300 0x7fddb80bec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80c4af0 0x7fddb80bf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80be770 con 0x7fddb815d370 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80bde40 con 0x7fddb80c4af0 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddbd8e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80be120 con 0x7fddb806b300 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33588/0 (socket says 172.21.15.154:33588) 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 -- 172.21.15.154:0/285944763 learned_addr learned my addr 172.21.15.154:0/285944763 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb67fc700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80c4af0 0x7fddb80bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb6ffd700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb806b300 0x7fddb80bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 -- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb806b300 msgr2=0x7fddb80bec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb806b300 0x7fddb80bec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 -- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80c4af0 msgr2=0x7fddb80bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80c4af0 0x7fddb80bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 -- 172.21.15.154:0/285944763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac009550 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.744+0000 7fddb77fe700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009fd0 tx=0x7fddb000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.745+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007ea0 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.745+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000ed30 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.745+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb80be400 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.745+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000eea0 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.745+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb80c1800 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.746+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb8067bd0 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.746+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fddb0014660 con 0x7fddb815d370 2024-03-14T09:01:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.746+0000 7fdd9ffff700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 0x7fdda406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.747+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb0089330 con 0x7fddb815d370 2024-03-14T09:01:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.747+0000 7fddb6ffd700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 0x7fdda406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.747+0000 7fddb6ffd700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 0x7fdda406ccf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddac009130 tx=0x7fddac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.751+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fddb0054740 con 0x7fddb815d370 2024-03-14T09:01:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.945+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddb80697d0 con 0x7fddb815d370 2024-03-14T09:01:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.946+0000 7fdd9ffff700 1 -- 172.21.15.154:0/285944763 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fddb0057d60 con 0x7fddb815d370 2024-03-14T09:01:30.945 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:30.945 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.948+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 msgr2=0x7fdda406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.948+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 0x7fdda406ccf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddac009130 tx=0x7fddac008030 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.948+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 msgr2=0x7fddb80c0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.948+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009fd0 tx=0x7fddb000e3c0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 shutdown_connections 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdda406a830 0x7fdda406ccf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fddb815d370 0x7fddb80c0120 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddb806b300 0x7fddb80bec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 --2- 172.21.15.154:0/285944763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddb80c4af0 0x7fddb80bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 >> 172.21.15.154:0/285944763 conn(0x7fddb80abc60 msgr2=0x7fddb8153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 shutdown_connections 2024-03-14T09:01:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:30.949+0000 7fddbd8e7700 1 -- 172.21.15.154:0/285944763 wait complete. 2024-03-14T09:01:30.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:30.960 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.087+0000 7f91d2030700 1 Processor -- start 2024-03-14T09:01:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.087+0000 7f91d2030700 1 -- start start 2024-03-14T09:01:31.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91d2030700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f91cc155df0 0x7f91cc156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91d2030700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91cc15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91d2030700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0ac8e0 con 0x7f91cc14eaf0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91d2030700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc06b310 con 0x7f91cc155df0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91d2030700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc06de30 con 0x7f91cc15d270 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91cb7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f91cc14eaf0 0x7f91cc06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52546/0 (socket says 172.21.15.154:52546) 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91cb7fe700 1 -- 172.21.15.154:0/980883763 learned_addr learned my addr 172.21.15.154:0/980883763 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91caffd700 1 --2- 172.21.15.154:0/980883763 >> v2:172.21.15.191:3300/0 conn(0x7f91cc155df0 0x7f91cc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91cbfff700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91cc15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921729260 0 0) 0x7f91cc0ac8e0 con 0x7f91cc14eaf0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b40035b0 con 0x7f91cc14eaf0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806387720 0 0) 0x7f91b40035b0 con 0x7f91cc14eaf0 2024-03-14T09:01:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 msgr2=0x7f91cc15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91cc15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 >> v2:172.21.15.191:3300/0 conn(0x7f91cc155df0 msgr2=0x7f91cc156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 --2- 172.21.15.154:0/980883763 >> v2:172.21.15.191:3300/0 conn(0x7f91cc155df0 0x7f91cc156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0acb30 con 0x7f91cc14eaf0 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.088+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f91c0001670 con 0x7f91cc14eaf0 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 >> v1:172.21.15.154:6789/0 conn(0x7f91cc14eaf0 legacy=0x7f91cc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91b4003a00 0x7f91b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b4004850 0x7f91b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc06b310 con 0x7f91b4003a00 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc06de30 con 0x7f91b4004850 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b400ba90 con 0x7f91cc15d270 2024-03-14T09:01:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91b4003a00 0x7f91b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91caffd700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b4004850 0x7f91b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cbfff700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 -- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 msgr2=0x7f91b4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc15d270 0x7f91b4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 -- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b4004850 msgr2=0x7f91b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b4004850 0x7f91b4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 -- 172.21.15.154:0/980883763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b40035b0 con 0x7f91b4003a00 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91b4003a00 0x7f91b4003e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c370 tx=0x7f91b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41fe8da457b6be server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0007c60 con 0x7f91b4003a00 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c0005960 con 0x7f91b4003a00 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.089+0000 7f91ca7fc700 1 -- 172.21.15.154:0/980883763 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0005d10 con 0x7f91b4003a00 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 -- 172.21.15.154:0/980883763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91b4003a00 msgr2=0x7f91b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 --2- 172.21.15.154:0/980883763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91b4003a00 0x7f91b4003e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91c000c370 tx=0x7f91b4005880 comp rx=0 tx=0).stop 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91cbfff700 1 -- 172.21.15.154:0/980883763 reap_dead start 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 -- 172.21.15.154:0/980883763 shutdown_connections 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 -- 172.21.15.154:0/980883763 >> 172.21.15.154:0/980883763 conn(0x7f91cc0c3880 msgr2=0x7f91cc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 -- 172.21.15.154:0/980883763 shutdown_connections 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.090+0000 7f91d2030700 1 -- 172.21.15.154:0/980883763 wait complete. 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.091+0000 7f91d2030700 1 Processor -- start 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.091+0000 7f91d2030700 1 -- start start 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b4003a00 0x7f91cc0c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 0x7f91cc152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0c1bb0 con 0x7f91cc153460 2024-03-14T09:01:31.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1ee0d0 con 0x7f91cc1ee4f0 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0c18d0 con 0x7f91b4003a00 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33612/0 (socket says 172.21.15.154:33612) 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 -- 172.21.15.154:0/4245669057 learned_addr learned my addr 172.21.15.154:0/4245669057 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91caffd700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 0x7f91cc152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b4003a00 0x7f91cc0c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 -- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b4003a00 msgr2=0x7f91cc0c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b4003a00 0x7f91cc0c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 -- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 msgr2=0x7f91cc152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 0x7f91cc152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 -- 172.21.15.154:0/4245669057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc1eda10 con 0x7f91cc153460 2024-03-14T09:01:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91caffd700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 0x7f91cc152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91cbfff700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0c12d0 tx=0x7f91c40114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c400e070 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4003cf0 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc1edcf0 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4003ea0 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.092+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc145120 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.093+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc1ed870 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.094+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f91c40119b0 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.094+0000 7f91b3fff700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 0x7f91b806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.094+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91c4089110 con 0x7f91cc153460 2024-03-14T09:01:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.094+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 0x7f91b806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.095+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 0x7f91b806cd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c0015040 tx=0x7f91c000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.098+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f91cc1ed870 con 0x7f91cc153460 2024-03-14T09:01:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.290+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91cc1ee210 con 0x7f91cc153460 2024-03-14T09:01:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.290+0000 7f91b3fff700 1 -- 172.21.15.154:0/4245669057 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91cc1ee210 con 0x7f91cc153460 2024-03-14T09:01:31.289 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 msgr2=0x7f91b806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 0x7f91b806cd20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c0015040 tx=0x7f91c000a000 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 msgr2=0x7f91cc13c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0c12d0 tx=0x7f91c40114b0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 shutdown_connections 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a860 0x7f91b806cd20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc153460 0x7f91cc13c550 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91b4003a00 0x7f91cc0c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 --2- 172.21.15.154:0/4245669057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc1ee4f0 0x7f91cc152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 >> 172.21.15.154:0/4245669057 conn(0x7f91cc0c3880 msgr2=0x7f91cc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 shutdown_connections 2024-03-14T09:01:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.293+0000 7f91d2030700 1 -- 172.21.15.154:0/4245669057 wait complete. 2024-03-14T09:01:31.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:31.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.430+0000 7f4e71899700 1 Processor -- start 2024-03-14T09:01:31.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 -- start start 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c6b00 0x7f4e6c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e6c155190 0x7f4e6c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0c7550 con 0x7f4e6c0c4a70 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c06c140 con 0x7f4e6c155190 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e71899700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c14b780 con 0x7f4e6c0c6b00 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e6a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e6c155190 0x7f4e6c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e6a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e6c155190 0x7f4e6c06c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33626/0 (socket says 172.21.15.154:33626) 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.431+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/135030609 learned_addr learned my addr 172.21.15.154:0/135030609 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:31.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6affd700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c6b00 0x7f4e6c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/135030609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c6b00 msgr2=0x7f4e6c0c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6a7fc700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c6b00 0x7f4e6c0c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/135030609 >> v1:172.21.15.191:6789/0 conn(0x7f4e6c0c4a70 legacy=0x7f4e6c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/135030609 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0c6f90 con 0x7f4e6c155190 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e6a7fc700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c06c5d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098f0 tx=0x7f4e5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5f11b6639123e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e69ffb700 1 -- 172.21.15.154:0/135030609 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004230 con 0x7f4e6c155190 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e69ffb700 1 -- 172.21.15.154:0/135030609 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c004390 con 0x7f4e6c155190 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.432+0000 7f4e69ffb700 1 -- 172.21.15.154:0/135030609 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0036a0 con 0x7f4e6c155190 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 -- 172.21.15.154:0/135030609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 msgr2=0x7f4e6c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c06c5d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0098f0 tx=0x7f4e5c009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 -- 172.21.15.154:0/135030609 shutdown_connections 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c6b00 0x7f4e6c0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 --2- 172.21.15.154:0/135030609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c06c5d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 -- 172.21.15.154:0/135030609 >> 172.21.15.154:0/135030609 conn(0x7f4e6c0abc80 msgr2=0x7f4e6c0ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 -- 172.21.15.154:0/135030609 shutdown_connections 2024-03-14T09:01:31.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.433+0000 7f4e71899700 1 -- 172.21.15.154:0/135030609 wait complete. 2024-03-14T09:01:31.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.434+0000 7f4e71899700 1 Processor -- start 2024-03-14T09:01:31.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.434+0000 7f4e71899700 1 -- start start 2024-03-14T09:01:31.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c4a70 0x7f4e6c0c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 0x7f4e6c1479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c1e9ec0 con 0x7f4e6c155190 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c1ea0d0 con 0x7f4e6c147500 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c1ea3b0 con 0x7f4e6c0c4a70 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33630/0 (socket says 172.21.15.154:33630) 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/1141396651 learned_addr learned my addr 172.21.15.154:0/1141396651 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6affd700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c4a70 0x7f4e6c0c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6b7fe700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 0x7f4e6c1479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c4a70 msgr2=0x7f4e6c0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c4a70 0x7f4e6c0c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 msgr2=0x7f4e6c1479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 0x7f4e6c1479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 -- 172.21.15.154:0/1141396651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c009550 con 0x7f4e6c155190 2024-03-14T09:01:31.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6b7fe700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 0x7f4e6c1479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e6a7fc700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009130 tx=0x7f4e5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003d80 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c01f070 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c1ea690 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00b8a0 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.435+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c1eac50 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.436+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c1eae10 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.437+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4e5c021030 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.437+0000 7f4e63fff700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 0x7f4e4c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.437+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e5c08a360 con 0x7f4e6c155190 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.437+0000 7f4e6affd700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 0x7f4e4c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.438+0000 7f4e6affd700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 0x7f4e4c06ccf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400ab60 tx=0x7f4e54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.442+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4e5c055790 con 0x7f4e6c155190 2024-03-14T09:01:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.640+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e6c1eae10 con 0x7f4e6c155190 2024-03-14T09:01:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.640+0000 7f4e63fff700 1 -- 172.21.15.154:0/1141396651 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e5c058db0 con 0x7f4e6c155190 2024-03-14T09:01:31.640 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 msgr2=0x7f4e4c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 0x7f4e4c06ccf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400ab60 tx=0x7f4e54008040 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 msgr2=0x7f4e6c0ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009130 tx=0x7f4e5c004750 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 shutdown_connections 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e6c147500 0x7f4e6c1479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e6c0c4a70 0x7f4e6c0c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4e4c06a830 0x7f4e4c06ccf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 --2- 172.21.15.154:0/1141396651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e6c155190 0x7f4e6c0ad420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 >> 172.21.15.154:0/1141396651 conn(0x7f4e6c0abc80 msgr2=0x7f4e6c0ab570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 shutdown_connections 2024-03-14T09:01:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.642+0000 7f4e71899700 1 -- 172.21.15.154:0/1141396651 wait complete. 2024-03-14T09:01:31.654 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:31.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.779+0000 7f010eb2b700 1 Processor -- start 2024-03-14T09:01:31.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.779+0000 7f010eb2b700 1 -- start start 2024-03-14T09:01:31.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010eb2b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f010814c5c0 0x7f0108152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010eb2b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0108070e10 con 0x7f0108072290 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010eb2b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010814c9a0 con 0x7f0108072710 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010eb2b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01080c1bb0 con 0x7f010814c5c0 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f0107fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0108072290 0x7f01080c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52558/0 (socket says 172.21.15.154:52558) 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f0107fff700 1 -- 172.21.15.154:0/1571320905 learned_addr learned my addr 172.21.15.154:0/1571320905 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010c8c7700 1 --2- 172.21.15.154:0/1571320905 >> v2:172.21.15.191:3300/0 conn(0x7f010814c5c0 0x7f0108152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108744883 0 0) 0x7f0108070e10 con 0x7f0108072290 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f80035b0 con 0x7f0108072290 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365518980 0 0) 0x7f00f80035b0 con 0x7f0108072290 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 >> v2:172.21.15.191:3300/0 conn(0x7f010814c5c0 msgr2=0x7f0108152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 --2- 172.21.15.154:0/1571320905 >> v2:172.21.15.191:3300/0 conn(0x7f010814c5c0 0x7f0108152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 >> v1:172.21.15.191:6789/0 conn(0x7f0108072710 legacy=0x7f01080c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080c1580 con 0x7f0108072290 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f00f4001670 con 0x7f0108072290 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 >> v1:172.21.15.154:6789/0 conn(0x7f0108072290 legacy=0x7f01080c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8003a00 0x7f00f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 0x7f00f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080c1bb0 con 0x7f00f8005210 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f800c160 con 0x7f00f8004850 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f800c320 con 0x7f00f8003a00 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010d0c8700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 0x7f00f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f0107fff700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.780+0000 7f010c8c7700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8003a00 0x7f00f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 -- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8003a00 msgr2=0x7f00f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8003a00 0x7f00f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 -- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 msgr2=0x7f00f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 -- 172.21.15.154:0/1571320905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80035b0 con 0x7f00f8005210 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 -- 172.21.15.154:0/1571320905 reap_dead start 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010d0c8700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 0x7f00f800ba10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f010000c290 tx=0x7f00f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ac5cee2d645315 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100001e70 con 0x7f00f8005210 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000ce70 con 0x7f00f8005210 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f01077fe700 1 -- 172.21.15.154:0/1571320905 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100003c20 con 0x7f00f8005210 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010eb2b700 1 -- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 msgr2=0x7f00f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.781+0000 7f010eb2b700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 0x7f00f800ba10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f010000c290 tx=0x7f00f8005880 comp rx=0 tx=0).stop 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.782+0000 7f010eb2b700 1 -- 172.21.15.154:0/1571320905 shutdown_connections 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.782+0000 7f010eb2b700 1 --2- 172.21.15.154:0/1571320905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8005210 0x7f00f800ba10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.782+0000 7f010eb2b700 1 -- 172.21.15.154:0/1571320905 >> 172.21.15.154:0/1571320905 conn(0x7f010806a770 msgr2=0x7f010806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.782+0000 7f010eb2b700 1 -- 172.21.15.154:0/1571320905 shutdown_connections 2024-03-14T09:01:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.782+0000 7f010eb2b700 1 -- 172.21.15.154:0/1571320905 wait complete. 2024-03-14T09:01:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.783+0000 7f010eb2b700 1 Processor -- start 2024-03-14T09:01:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.783+0000 7f010eb2b700 1 -- start start 2024-03-14T09:01:31.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8005210 0x7f01080aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01081e89e0 0x7f01081e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081e8160 con 0x7f010814b690 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081e7570 con 0x7f01081e89e0 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010eb2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01081e7830 con 0x7f00f8005210 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f0107fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f0107fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33644/0 (socket says 172.21.15.154:33644) 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f0107fff700 1 -- 172.21.15.154:0/2921085363 learned_addr learned my addr 172.21.15.154:0/2921085363 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010d0c8700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01081e89e0 0x7f01081e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.784+0000 7f010c8c7700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8005210 0x7f01080aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 -- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8005210 msgr2=0x7f01080aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8005210 0x7f01080aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 -- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01081e89e0 msgr2=0x7f01081e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01081e89e0 0x7f01081e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 -- 172.21.15.154:0/2921085363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01081e7b10 con 0x7f010814b690 2024-03-14T09:01:31.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f0107fff700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007290 tx=0x7f00f40075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4012560 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f40126c0 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01081e7df0 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4012a70 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.785+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010806bc50 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.786+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0108072d40 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.786+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f00f4012bd0 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.787+0000 7f01057fa700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 0x7f00ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.787+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00f4088cb0 con 0x7f010814b690 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.787+0000 7f010c8c7700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 0x7f00ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:31.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.787+0000 7f010c8c7700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 0x7f00ec06ccf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01081e53b0 tx=0x7f00fc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.791+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f00f40540c0 con 0x7f010814b690 2024-03-14T09:01:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.983+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01080697d0 con 0x7f010814b690 2024-03-14T09:01:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.984+0000 7f01057fa700 1 -- 172.21.15.154:0/2921085363 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00f40576e0 con 0x7f010814b690 2024-03-14T09:01:31.984 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 msgr2=0x7f00ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 0x7f00ec06ccf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01081e53b0 tx=0x7f00fc009210 comp rx=0 tx=0).stop 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 msgr2=0x7f010814bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00f4007290 tx=0x7f00f40075a0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 shutdown_connections 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01081e89e0 0x7f01081e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00f8005210 0x7f01080aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00ec06a830 0x7f00ec06ccf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 --2- 172.21.15.154:0/2921085363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f010814b690 0x7f010814bb10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 >> 172.21.15.154:0/2921085363 conn(0x7f010806a770 msgr2=0x7f01080a74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 shutdown_connections 2024-03-14T09:01:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:31.986+0000 7f010eb2b700 1 -- 172.21.15.154:0/2921085363 wait complete. 2024-03-14T09:01:31.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:32.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.120+0000 7fe4f56f5700 1 Processor -- start 2024-03-14T09:01:32.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.120+0000 7fe4f56f5700 1 -- start start 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4f56f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4f56f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f015d370 0x7fe4f015f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4f56f5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00c55e0 con 0x7fe4f006b300 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4f56f5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00c5ac0 con 0x7fe4f00c4af0 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4f56f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00c8290 con 0x7fe4f015d370 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33654/0 (socket says 172.21.15.154:33654) 2024-03-14T09:01:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4ee7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4f006b300 0x7fe4f006b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52570/0 (socket says 172.21.15.154:52570) 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 -- 172.21.15.154:0/2892342762 learned_addr learned my addr 172.21.15.154:0/2892342762 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4ef7fe700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f015d370 0x7fe4f015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 -- 172.21.15.154:0/2892342762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f015d370 msgr2=0x7fe4f015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f015d370 0x7fe4f015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 -- 172.21.15.154:0/2892342762 >> v1:172.21.15.154:6789/0 conn(0x7fe4f006b300 legacy=0x7fe4f006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4edffb700 1 -- 172.21.15.154:0/2892342762 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505224123 0 0) 0x7fe4f00c55e0 con 0x7fe4f006b300 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 -- 172.21.15.154:0/2892342762 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f0154bf0 con 0x7fe4f00c4af0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4eeffd700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40098f0 tx=0x7fe4e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21581419b5f391 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.121+0000 7fe4edffb700 1 -- 172.21.15.154:0/2892342762 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400b8a0 con 0x7fe4f00c4af0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4edffb700 1 -- 172.21.15.154:0/2892342762 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400ba00 con 0x7fe4f00c4af0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4edffb700 1 -- 172.21.15.154:0/2892342762 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400bd90 con 0x7fe4f00c4af0 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 -- 172.21.15.154:0/2892342762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f00c4af0 msgr2=0x7fe4f00c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40098f0 tx=0x7fe4e4009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 -- 172.21.15.154:0/2892342762 shutdown_connections 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f015d370 0x7fe4f015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/2892342762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f00c4af0 0x7fe4f00c4ed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 -- 172.21.15.154:0/2892342762 >> 172.21.15.154:0/2892342762 conn(0x7fe4f00abc60 msgr2=0x7fe4f00ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 -- 172.21.15.154:0/2892342762 shutdown_connections 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.122+0000 7fe4f56f5700 1 -- 172.21.15.154:0/2892342762 wait complete. 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.123+0000 7fe4f56f5700 1 Processor -- start 2024-03-14T09:01:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.123+0000 7fe4f56f5700 1 -- start start 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00c4af0 0x7fe4f00c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 0x7fe4f00c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f014d800 con 0x7fe4f006b300 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00c4760 con 0x7fe4f015d370 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4f56f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01584a0 con 0x7fe4f00c4af0 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33662/0 (socket says 172.21.15.154:33662) 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 -- 172.21.15.154:0/50313586 learned_addr learned my addr 172.21.15.154:0/50313586 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4ee7fc700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00c4af0 0x7fe4f00c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4ef7fe700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 0x7fe4f00c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 -- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00c4af0 msgr2=0x7fe4f00c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00c4af0 0x7fe4f00c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 -- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 msgr2=0x7fe4f00c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 0x7fe4f00c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4eeffd700 1 -- 172.21.15.154:0/50313586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4009550 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.124+0000 7fe4ef7fe700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 0x7fe4f00c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4eeffd700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0072e80 tx=0x7fe4e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4004300 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e401f070 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00c4430 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40036a0 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.125+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f0151960 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.126+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0067bd0 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.126+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe4e4021030 con 0x7fe4f006b300 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.127+0000 7fe4d77fe700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 0x7fe4dc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.127+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e408a160 con 0x7fe4f006b300 2024-03-14T09:01:32.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.127+0000 7fe4ee7fc700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 0x7fe4dc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.127+0000 7fe4ee7fc700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 0x7fe4dc06ccf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00c1e90 tx=0x7fe4e0007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.131+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe4e4055610 con 0x7fe4f006b300 2024-03-14T09:01:32.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.368+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4f00697d0 con 0x7fe4f006b300 2024-03-14T09:01:32.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.369+0000 7fe4d77fe700 1 -- 172.21.15.154:0/50313586 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fe4e4058c30 con 0x7fe4f006b300 2024-03-14T09:01:32.369 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:32.369 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 msgr2=0x7fe4dc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 0x7fe4dc06ccf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00c1e90 tx=0x7fe4e0007de0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 msgr2=0x7fe4f0151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0072e80 tx=0x7fe4e4004750 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 shutdown_connections 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f015d370 0x7fe4f00c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4dc06a830 0x7fe4dc06ccf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4f006b300 0x7fe4f0151420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 --2- 172.21.15.154:0/50313586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4f00c4af0 0x7fe4f00c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 >> 172.21.15.154:0/50313586 conn(0x7fe4f00abc60 msgr2=0x7fe4f00ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 shutdown_connections 2024-03-14T09:01:32.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.371+0000 7fe4f56f5700 1 -- 172.21.15.154:0/50313586 wait complete. 2024-03-14T09:01:32.372 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:32.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:32.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.505+0000 7f88b638a700 1 Processor -- start 2024-03-14T09:01:32.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.505+0000 7f88b638a700 1 -- start start 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b638a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f88b006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b638a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f88b0156500 0x7f88b015f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b638a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00ac8e0 con 0x7f88b006acb0 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b638a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b006b310 con 0x7f88b0156500 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b638a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006de30 con 0x7f88b014eaf0 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88af7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f88b006acb0 0x7f88b0155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52574/0 (socket says 172.21.15.154:52574) 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88af7fe700 1 -- 172.21.15.154:0/2971600968 learned_addr learned my addr 172.21.15.154:0/2971600968 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88b4927700 1 --2- 172.21.15.154:0/2971600968 >> v2:172.21.15.191:3300/0 conn(0x7f88b0156500 0x7f88b015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88affff700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f88b006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752500425 0 0) 0x7f88b00ac8e0 con 0x7f88b006acb0 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.506+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88980035b0 con 0x7f88b006acb0 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 281038047 0 0) 0x7f88980035b0 con 0x7f88b006acb0 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 msgr2=0x7f88b006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f88b006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 >> v2:172.21.15.191:3300/0 conn(0x7f88b0156500 msgr2=0x7f88b015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 --2- 172.21.15.154:0/2971600968 >> v2:172.21.15.191:3300/0 conn(0x7f88b0156500 0x7f88b015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0156950 con 0x7f88b006acb0 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f88a0001670 con 0x7f88b006acb0 2024-03-14T09:01:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 >> v1:172.21.15.154:6789/0 conn(0x7f88b006acb0 legacy=0x7f88b0155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f8898003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f8898004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8898004d90 0x7f8898005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006b310 con 0x7f8898003f80 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006de30 con 0x7f8898004d90 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800baf0 con 0x7f88b014eaf0 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f8898004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88affff700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f8898003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88b4927700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8898004d90 0x7f8898005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 -- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 msgr2=0x7f8898003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b014eaf0 0x7f8898003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 -- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8898004d90 msgr2=0x7f8898005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8898004d90 0x7f8898005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 -- 172.21.15.154:0/2971600968 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980035b0 con 0x7f8898003f80 2024-03-14T09:01:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88af7fe700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f8898004850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c240 tx=0x7f8898005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3486ca9428d9f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000cde0 con 0x7f8898003f80 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.507+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0007ba0 con 0x7f8898003f80 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88aeffd700 1 -- 172.21.15.154:0/2971600968 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0005b80 con 0x7f8898003f80 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 -- 172.21.15.154:0/2971600968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 msgr2=0x7f8898004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 --2- 172.21.15.154:0/2971600968 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f8898004850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c240 tx=0x7f8898005880 comp rx=0 tx=0).stop 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b4927700 1 -- 172.21.15.154:0/2971600968 reap_dead start 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 -- 172.21.15.154:0/2971600968 shutdown_connections 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 -- 172.21.15.154:0/2971600968 >> 172.21.15.154:0/2971600968 conn(0x7f88b00c3880 msgr2=0x7f88b00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 -- 172.21.15.154:0/2971600968 shutdown_connections 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.508+0000 7f88b638a700 1 -- 172.21.15.154:0/2971600968 wait complete. 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.509+0000 7f88b638a700 1 Processor -- start 2024-03-14T09:01:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.509+0000 7f88b638a700 1 -- start start 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88b014c310 0x7f88b0141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b0142260 0x7f88b01e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0143290 con 0x7f8898003f80 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00c2200 con 0x7f88b014c310 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b638a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00c1c40 con 0x7f88b0142260 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33676/0 (socket says 172.21.15.154:33676) 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 -- 172.21.15.154:0/1130935314 learned_addr learned my addr 172.21.15.154:0/1130935314 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88af7fe700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88b014c310 0x7f88b0141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88b4927700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b0142260 0x7f88b01e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 -- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b0142260 msgr2=0x7f88b01e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b0142260 0x7f88b01e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 -- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88b014c310 msgr2=0x7f88b0141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88b014c310 0x7f88b0141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.510+0000 7f88affff700 1 -- 172.21.15.154:0/1130935314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00c1f20 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88affff700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88a40119a0 tx=0x7f88a4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400f780 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400f8e0 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400fc70 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b0144190 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.511+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0144750 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.512+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b00697d0 con 0x7f8898003f80 2024-03-14T09:01:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.512+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f88a4014440 con 0x7f8898003f80 2024-03-14T09:01:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.512+0000 7f88acff9700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 0x7f889c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.513+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a4089190 con 0x7f8898003f80 2024-03-14T09:01:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.513+0000 7f88af7fe700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 0x7f889c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.513+0000 7f88af7fe700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 0x7f889c06ccf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88a0007d50 tx=0x7f88a000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.517+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f88a4054640 con 0x7f8898003f80 2024-03-14T09:01:32.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.714+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b006b470 con 0x7f8898003f80 2024-03-14T09:01:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.715+0000 7f88acff9700 1 -- 172.21.15.154:0/1130935314 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f88a4057c60 con 0x7f8898003f80 2024-03-14T09:01:32.715 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:32.715 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 msgr2=0x7f889c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 0x7f889c06ccf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88a0007d50 tx=0x7f88a000a000 comp rx=0 tx=0).stop 2024-03-14T09:01:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 msgr2=0x7f88b0142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f88a40119a0 tx=0x7f88a4011cb0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 shutdown_connections 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88b0142260 0x7f88b01e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f889c06a830 0x7f889c06ccf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8898003f80 0x7f88b0142d50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 --2- 172.21.15.154:0/1130935314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88b014c310 0x7f88b0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 >> 172.21.15.154:0/1130935314 conn(0x7f88b00c3880 msgr2=0x7f88b00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 shutdown_connections 2024-03-14T09:01:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.717+0000 7f88b638a700 1 -- 172.21.15.154:0/1130935314 wait complete. 2024-03-14T09:01:32.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:32.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.853+0000 7fe08010b700 1 Processor -- start 2024-03-14T09:01:32.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 -- start start 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07806d3c0 0x7fe0780c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe0780c1830 0x7fe0780c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe078151860 con 0x7fe0780c1d90 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780c50e0 con 0x7fe07806d3c0 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe08010b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe078150f30 con 0x7fe0780c1830 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.854+0000 7fe07dea7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07806d3c0 0x7fe0780c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07806d3c0 0x7fe0780c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33688/0 (socket says 172.21.15.154:33688) 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 -- 172.21.15.154:0/1392434283 learned_addr learned my addr 172.21.15.154:0/1392434283 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07e6a8700 1 --2- 172.21.15.154:0/1392434283 >> v2:172.21.15.191:3300/0 conn(0x7fe0780c1830 0x7fe0780c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 -- 172.21.15.154:0/1392434283 >> v2:172.21.15.191:3300/0 conn(0x7fe0780c1830 msgr2=0x7fe0780c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 --2- 172.21.15.154:0/1392434283 >> v2:172.21.15.191:3300/0 conn(0x7fe0780c1830 0x7fe0780c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 -- 172.21.15.154:0/1392434283 >> v1:172.21.15.191:6789/0 conn(0x7fe0780c1d90 legacy=0x7fe0780c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 -- 172.21.15.154:0/1392434283 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078151210 con 0x7fe07806d3c0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07dea7700 1 --2- 172.21.15.154:0/1392434283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07806d3c0 0x7fe0780c12f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bb90 tx=0x7fe06c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f804a09c0f8e6db server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07cea5700 1 -- 172.21.15.154:0/1392434283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00dcf0 con 0x7fe07806d3c0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07cea5700 1 -- 172.21.15.154:0/1392434283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c00de50 con 0x7fe07806d3c0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.855+0000 7fe07cea5700 1 -- 172.21.15.154:0/1392434283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c010690 con 0x7fe07806d3c0 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 -- 172.21.15.154:0/1392434283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07806d3c0 msgr2=0x7fe0780c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 --2- 172.21.15.154:0/1392434283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07806d3c0 0x7fe0780c12f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bb90 tx=0x7fe06c00bea0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 -- 172.21.15.154:0/1392434283 shutdown_connections 2024-03-14T09:01:32.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 --2- 172.21.15.154:0/1392434283 >> v2:172.21.15.191:3300/0 conn(0x7fe0780c1830 0x7fe0780c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 --2- 172.21.15.154:0/1392434283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07806d3c0 0x7fe0780c12f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 -- 172.21.15.154:0/1392434283 >> 172.21.15.154:0/1392434283 conn(0x7fe07806a770 msgr2=0x7fe07806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:32.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 -- 172.21.15.154:0/1392434283 shutdown_connections 2024-03-14T09:01:32.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.856+0000 7fe08010b700 1 -- 172.21.15.154:0/1392434283 wait complete. 2024-03-14T09:01:32.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.857+0000 7fe08010b700 1 Processor -- start 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.857+0000 7fe08010b700 1 -- start start 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe07806d3c0 0x7fe0781e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0780c1830 0x7fe07814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe078154150 con 0x7fe0780c1d90 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780aa3f0 con 0x7fe0780c1830 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe08010b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe078151f80 con 0x7fe07806d3c0 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe07e6a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe07e6a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33690/0 (socket says 172.21.15.154:33690) 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe07e6a8700 1 -- 172.21.15.154:0/3286539980 learned_addr learned my addr 172.21.15.154:0/3286539980 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe07d6a6700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0780c1830 0x7fe07814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.858+0000 7fe07dea7700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe07806d3c0 0x7fe0781e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 -- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe07806d3c0 msgr2=0x7fe0781e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe07806d3c0 0x7fe0781e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 -- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0780c1830 msgr2=0x7fe07814fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0780c1830 0x7fe07814fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 -- 172.21.15.154:0/3286539980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c00b7f0 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe07e6a8700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe07000efc0 tx=0x7fe07000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070009ea0 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07000f040 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0781e7120 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07000ced0 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.859+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe078153520 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.860+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0780697d0 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.860+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe070014480 con 0x7fe0780c1d90 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.861+0000 7fe06a7fc700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 0x7fe05c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:32.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.861+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe070089860 con 0x7fe0780c1d90 2024-03-14T09:01:32.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.861+0000 7fe07dea7700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 0x7fe05c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:32.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.861+0000 7fe07dea7700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 0x7fe05c06ccf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000c00 tx=0x7fe06c013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:32.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:32.865+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe070054c70 con 0x7fe0780c1d90 2024-03-14T09:01:33.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.104+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe07806bee0 con 0x7fe0780c1d90 2024-03-14T09:01:33.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.104+0000 7fe06a7fc700 1 -- 172.21.15.154:0/3286539980 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fe070058290 con 0x7fe0780c1d90 2024-03-14T09:01:33.104 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:33.104 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:33.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 msgr2=0x7fe05c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 0x7fe05c06ccf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000c00 tx=0x7fe06c013000 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 msgr2=0x7fe078149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe07000efc0 tx=0x7fe07000c3a0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 shutdown_connections 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05c06a830 0x7fe05c06ccf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0780c1d90 0x7fe078149aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe07806d3c0 0x7fe0781e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 --2- 172.21.15.154:0/3286539980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0780c1830 0x7fe07814fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 >> 172.21.15.154:0/3286539980 conn(0x7fe07806a770 msgr2=0x7fe07814dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 shutdown_connections 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.106+0000 7fe08010b700 1 -- 172.21.15.154:0/3286539980 wait complete. 2024-03-14T09:01:33.107 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:33.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T09:01:19.106939+0000', 'modified': '2024-03-14T09:01:22.301841+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4255': {'gid': 4255, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/1086778085', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1086778085}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1086778085}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T09:01:33.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T09:01:33.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-14T09:01:33.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.645+0000 7f1cfca95700 1 Processor -- start 2024-03-14T09:01:33.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.645+0000 7f1cfca95700 1 -- start start 2024-03-14T09:01:33.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cfca95700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1cf80c1e90 0x7f1cf813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cfca95700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf813ade0 con 0x7f1cf80c3250 2024-03-14T09:01:33.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cfca95700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf806fb30 con 0x7f1cf80a8540 2024-03-14T09:01:33.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cfca95700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf806ec30 con 0x7f1cf80c1e90 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cf5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1cf80c3250 0x7f1cf80c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52584/0 (socket says 172.21.15.154:52584) 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.646+0000 7f1cf5d9b700 1 -- 172.21.15.154:0/1715261236 learned_addr learned my addr 172.21.15.154:0/1715261236 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026259379 0 0) 0x7f1ce8001110 con 0x7f1cf80c3250 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/1715261236 >> v2:172.21.15.191:3300/0 conn(0x7f1cf80c1e90 0x7f1cf813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce00035b0 con 0x7f1cf80c3250 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651824061 0 0) 0x7f1ce00035b0 con 0x7f1cf80c3250 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 >> v2:172.21.15.191:3300/0 conn(0x7f1cf80c1e90 msgr2=0x7f1cf813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 --2- 172.21.15.154:0/1715261236 >> v2:172.21.15.191:3300/0 conn(0x7f1cf80c1e90 0x7f1cf813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 >> v1:172.21.15.191:6789/0 conn(0x7f1cf80a8540 legacy=0x7f1cf8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf806ef10 con 0x7f1cf80c3250 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f1ce8001800 con 0x7f1cf80c3250 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 >> v1:172.21.15.154:6789/0 conn(0x7f1cf80c3250 legacy=0x7f1cf80c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 0x7f1ce0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0004850 0x7f1ce0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0005330 0x7f1ce000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf806ec30 con 0x7f1ce0003a00 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000c200 con 0x7f1ce0004850 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce000c3c0 con 0x7f1ce0005330 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 0x7f1ce0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0004850 0x7f1ce0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0005330 0x7f1ce000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 -- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0005330 msgr2=0x7f1ce000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0005330 0x7f1ce000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 -- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0004850 msgr2=0x7f1ce0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0004850 0x7f1ce0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf659c700 1 -- 172.21.15.154:0/1715261236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00035b0 con 0x7f1ce0003a00 2024-03-14T09:01:33.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0004850 0x7f1ce0004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0005330 0x7f1ce000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.647+0000 7f1cf6d9d700 1 -- 172.21.15.154:0/1715261236 reap_dead start 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 0x7f1ce0003e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007fd0 tx=0x7f1ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ce5e5395efb75e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0052d0 con 0x7f1ce0003a00 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec005430 con 0x7f1ce0003a00 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cf559a700 1 -- 172.21.15.154:0/1715261236 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0038c0 con 0x7f1ce0003a00 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 -- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 msgr2=0x7f1ce0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 0x7f1ce0003e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007fd0 tx=0x7f1ce0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 -- 172.21.15.154:0/1715261236 shutdown_connections 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 --2- 172.21.15.154:0/1715261236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ce0003a00 0x7f1ce0003e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 -- 172.21.15.154:0/1715261236 >> 172.21.15.154:0/1715261236 conn(0x7f1cf80c3880 msgr2=0x7f1cf80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 -- 172.21.15.154:0/1715261236 shutdown_connections 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.648+0000 7f1cfca95700 1 -- 172.21.15.154:0/1715261236 wait complete. 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 Processor -- start 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 -- start start 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0003a00 0x7f1cf81eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cf80acdd0 0x7f1cf81394b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8130a70 con 0x7f1cf81e40c0 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8130520 con 0x7f1ce0003a00 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cfca95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8130790 con 0x7f1cf80acdd0 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cf6d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33708/0 (socket says 172.21.15.154:33708) 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cf6d9d700 1 -- 172.21.15.154:0/736155238 learned_addr learned my addr 172.21.15.154:0/736155238 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:33.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cf80acdd0 0x7f1cf81394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.650+0000 7f1cf659c700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0003a00 0x7f1cf81eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 -- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cf80acdd0 msgr2=0x7f1cf81394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cf80acdd0 0x7f1cf81394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 -- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0003a00 msgr2=0x7f1cf81eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0003a00 0x7f1cf81eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 -- 172.21.15.154:0/736155238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80a9ec0 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80728f0 tx=0x7f1cf00104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1ce6ffd700 1 -- 172.21.15.154:0/736155238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf001b070 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1ce6ffd700 1 -- 172.21.15.154:0/736155238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf000f040 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80aa1a0 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1ce6ffd700 1 -- 172.21.15.154:0/736155238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000e050 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.651+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf81399f0 con 0x7f1cf81e40c0 2024-03-14T09:01:33.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.652+0000 7f1ce6ffd700 1 -- 172.21.15.154:0/736155238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1cf0015440 con 0x7f1cf81e40c0 2024-03-14T09:01:33.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.653+0000 7f1ce6ffd700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 0x7f1cd806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.653+0000 7f1ce6ffd700 1 -- 172.21.15.154:0/736155238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cf0088d60 con 0x7f1cf81e40c0 2024-03-14T09:01:33.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.653+0000 7f1cf659c700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 0x7f1cd806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.653+0000 7f1cf659c700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 0x7f1cd806ccf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cec015040 tx=0x7f1cec009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.655+0000 7f1d08c4eb80 1 Processor -- start 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.655+0000 7f1d08c4eb80 1 -- start start 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.655+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.655+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cbf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4bdf940 con 0x55a3b4c863c0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4bdd300 con 0x55a3b4c85fe0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4bd8bf0 con 0x55a3b4cbf180 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33720/0 (socket says 172.21.15.154:33720) 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 -- 172.21.15.154:0/1549310568 learned_addr learned my addr 172.21.15.154:0/1549310568 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 -- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 msgr2=0x55a3b4cbf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cbf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 -- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c85fe0 msgr2=0x55a3b4ca53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 -- 172.21.15.154:0/1549310568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3b4a5a360 con 0x55a3b4c863c0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005a10 tx=0x7f1ce8001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77b5f9e4b9adac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cd7fff700 1 -- 172.21.15.154:0/1549310568 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8013720 con 0x55a3b4c863c0 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cd7fff700 1 -- 172.21.15.154:0/1549310568 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce8013880 con 0x55a3b4c863c0 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.656+0000 7f1cd7fff700 1 -- 172.21.15.154:0/1549310568 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8013c10 con 0x55a3b4c863c0 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1549310568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 msgr2=0x55a3b4ca5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005a10 tx=0x7f1ce8001720 comp rx=0 tx=0).stop 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1549310568 shutdown_connections 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1549310568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c863c0 0x55a3b4ca5920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1549310568 >> 172.21.15.154:0/1549310568 conn(0x55a3b4bfc1d0 msgr2=0x55a3b4c84e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1549310568 shutdown_connections 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.657+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1549310568 wait complete. 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.658+0000 7f1d08c4eb80 1 -- start start 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.658+0000 7f1d08c4eb80 1 Processor -- start 2024-03-14T09:01:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4c863c0 0x55a3b4ca8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4cbf180 0x55a3b4cd6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4b74d30 con 0x55a3b4c85fe0 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4b749d0 con 0x55a3b4cbf180 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4a5c650 con 0x55a3b4c863c0 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 10 client.0 _ll_register_callbacks cb 0x55a3b4b7e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33736/0 (socket says 172.21.15.154:33736) 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 -- 172.21.15.154:0/2450932260 learned_addr learned my addr 172.21.15.154:0/2450932260 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4cbf180 0x55a3b4cd6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4c863c0 0x55a3b4ca8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 -- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4c863c0 msgr2=0x55a3b4ca8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4c863c0 0x55a3b4ca8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 -- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4cbf180 msgr2=0x55a3b4cd6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4cbf180 0x55a3b4cd6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 -- 172.21.15.154:0/2450932260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce800f040 con 0x55a3b4c85fe0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c1d0 tx=0x7f1cec001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3710cf01868e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0052a0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cec033ae0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3b4c79450 con 0x55a3b4c85fe0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec033ca0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 10 client.4638 fetch_fsmap learned FSMap version 8 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.659+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a3b4a06b10 con 0x55a3b4c85fe0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3b4c79450 con 0x55a3b4c85fe0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1cec033020 con 0x55a3b4c85fe0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 20 client.4638 populate_metadata read hostname 'smithi154' 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4ca1cf0 0x55a3b4cc2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 4 client.4638 mds_command: new command op to 4255 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 --> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3b4bd16b0 con 0x55a3b4ca1cf0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4ca1cf0 0x55a3b4cc2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1cd57fa700 10 client.4638 ms_handle_connect on v2:172.21.15.191:6834/1086778085 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.660+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4ca1cf0 0x55a3b4cc2200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17a9fb10d2d85ff server_cookie=59e288e367895f0f in_seq=0 out_seq=0 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2450932260 <== mds.0 v2:172.21.15.191:6834/1086778085 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a3b4bd16b0 con 0x55a3b4ca1cf0 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1cd57fa700 10 client.4638 handle_command_reply: tid=0 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 client.4638 shutdown 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 msgr2=0x55a3b4ca55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c1d0 tx=0x7f1cec001190 comp rx=0 tx=0).stop 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 shutdown_connections 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4ca1cf0 0x55a3b4cc2200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4cbf180 0x55a3b4cd6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca55b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2450932260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4c863c0 0x55a3b4ca8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1cd57fa700 0 client.4638 ms_handle_reset on v2:172.21.15.191:6834/1086778085 2024-03-14T09:01:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 >> 172.21.15.154:0/2450932260 conn(0x55a3b4bfc1d0 msgr2=0x55a3b4cc07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 shutdown_connections 2024-03-14T09:01:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2450932260 wait complete. 2024-03-14T09:01:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.661+0000 7f1d08c4eb80 20 client.4638 trim_cache size 0 max 16384 2024-03-14T09:01:33.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.679+0000 7f1d08c4eb80 1 Processor -- start 2024-03-14T09:01:33.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.679+0000 7f1d08c4eb80 1 -- start start 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4c791f0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4cd63d0 con 0x55a3b4c863c0 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4c84730 con 0x55a3b4cbf180 2024-03-14T09:01:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33752/0 (socket says 172.21.15.154:33752) 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 -- 172.21.15.154:0/1387838691 learned_addr learned my addr 172.21.15.154:0/1387838691 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 -- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 msgr2=0x55a3b4cac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 -- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 msgr2=0x55a3b4cabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 -- 172.21.15.154:0/1387838691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3b4b9ee50 con 0x55a3b4c85fe0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1cf659c700 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0011d0 tx=0x7f1cec02b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c5214e93078abf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1ce4ff9700 1 -- 172.21.15.154:0/1387838691 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec003d90 con 0x55a3b4c85fe0 2024-03-14T09:01:33.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1ce4ff9700 1 -- 172.21.15.154:0/1387838691 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec02d070 con 0x55a3b4c85fe0 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.680+0000 7f1ce4ff9700 1 -- 172.21.15.154:0/1387838691 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec033b00 con 0x55a3b4c85fe0 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1387838691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 msgr2=0x55a3b4c800d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0011d0 tx=0x7f1cec02b070 comp rx=0 tx=0).stop 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1387838691 shutdown_connections 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4cac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4c800d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/1387838691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1387838691 >> 172.21.15.154:0/1387838691 conn(0x55a3b4bfc1d0 msgr2=0x55a3b4c867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1387838691 shutdown_connections 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/1387838691 wait complete. 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.681+0000 7f1d08c4eb80 1 -- start start 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 Processor -- start 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4d2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4c83d00 con 0x55a3b4c85fe0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4cabd40 con 0x55a3b4c863c0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3b4cca630 con 0x55a3b4cbf180 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1d08c4eb80 10 client.0 _ll_register_callbacks cb 0x55a3b4b7e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1cf659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33758/0 (socket says 172.21.15.154:33758) 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.682+0000 7f1cf659c700 1 -- 172.21.15.154:0/2255112932 learned_addr learned my addr 172.21.15.154:0/2255112932 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf6d9d700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4d2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 -- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 msgr2=0x55a3b4d2f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4d2f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 -- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 msgr2=0x55a3b4cca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 -- 172.21.15.154:0/2255112932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3b4a06b10 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cf659c700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007930 tx=0x7f1cec01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e2f05c9e9b2624 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec033a60 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cec033bc0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3b4cc38e0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cec033d80 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 10 client.4644 fetch_fsmap learned FSMap version 8 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a3b4a5ec80 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a3b4cc38e0 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1cec03c470 con 0x55a3b4c85fe0 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 10 client.4644 fetch_fsmap finished waiting for FSMap version 8 2024-03-14T09:01:33.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 10 client.4644 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 20 client.4644 populate_metadata read hostname 'smithi154' 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4d29850 0x55a3b4cd2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 4 client.4644 mds_command: new command op to 4255 tid=0[{"prefix": "status"}] 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.683+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 --> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f1ce8013880 con 0x55a3b4d29850 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4d29850 0x55a3b4cd2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cd57fa700 10 client.4644 ms_handle_connect on v2:172.21.15.191:6834/1086778085 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cf5d9b700 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4d29850 0x55a3b4cd2db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf4f0fe7621ad7c server_cookie=14f99e75cc3e8747 in_seq=0 out_seq=0 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cd57fa700 1 -- 172.21.15.154:0/2255112932 <== mds.0 v2:172.21.15.191:6834/1086778085 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x7f1ce8013880 con 0x55a3b4d29850 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cd57fa700 10 client.4644 handle_command_reply: tid=0 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 client.4644 shutdown 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 msgr2=0x55a3b4ca88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1cec007930 tx=0x7f1cec01f070 comp rx=0 tx=0).stop 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 shutdown_connections 2024-03-14T09:01:33.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a3b4cbf180 0x55a3b4d2f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a3b4c85fe0 0x55a3b4ca88e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:6834/1086778085,v1:172.21.15.191:6835/1086778085] conn(0x55a3b4d29850 0x55a3b4cd2db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 --2- 172.21.15.154:0/2255112932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3b4c863c0 0x55a3b4cca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1cd57fa700 0 client.4644 ms_handle_reset on v2:172.21.15.191:6834/1086778085 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 >> 172.21.15.154:0/2255112932 conn(0x55a3b4bfc1d0 msgr2=0x55a3b4c800d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 shutdown_connections 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.684+0000 7f1d08c4eb80 1 -- 172.21.15.154:0/2255112932 wait complete. 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1d08c4eb80 20 client.4644 trim_cache size 0 max 16384 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:01:33.691 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "661f6023-01e4-4d6e-b4b3-5b5701ce7cc5", 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4255, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 14.576131201999999, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 8, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 20, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 20, 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 21.003469320000001 2024-03-14T09:01:33.692 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:01:33.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 msgr2=0x7f1cd806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 0x7f1cd806ccf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cec015040 tx=0x7f1cec009030 comp rx=0 tx=0).stop 2024-03-14T09:01:33.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 msgr2=0x7f1cf81e4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:33.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80728f0 tx=0x7f1cf00104a0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 shutdown_connections 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cd806a830 0x7f1cd806ccf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cf81e40c0 0x7f1cf81e4570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0003a00 0x7f1cf81eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 --2- 172.21.15.154:0/736155238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cf80acdd0 0x7f1cf81394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 >> 172.21.15.154:0/736155238 conn(0x7f1cf80c3880 msgr2=0x7f1cf806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 shutdown_connections 2024-03-14T09:01:33.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:33.685+0000 7f1cfca95700 1 -- 172.21.15.154:0/736155238 wait complete. 2024-03-14T09:01:33.697 INFO:teuthology.run_tasks:Running task kclient... 2024-03-14T09:01:33.721 INFO:tasks.kclient:Mounting kernel clients... 2024-03-14T09:01:33.722 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-14T09:01:33.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:33.722 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:01:33.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:33.740 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-14T09:01:33.808 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2024-03-14T09:01:33.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:01:33.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:33.810 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:01:33.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:33.831 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-03-14T09:01:33.906 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2024-03-14T09:01:33.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:01:33.908 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-14T09:01:33.908 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T09:01:33.908 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:01:33.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:01:34.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.032+0000 7fbfd9201700 1 Processor -- start 2024-03-14T09:01:34.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.032+0000 7fbfd9201700 1 -- start start 2024-03-14T09:01:34.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c6b00 0x7fbfd40c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbfd4155190 0x7fbfd406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfd40c4a70 0x7fbfd40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40c7550 con 0x7fbfd40c4a70 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd406c140 con 0x7fbfd4155190 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd9201700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd414b780 con 0x7fbfd40c6b00 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfd40c4a70 0x7fbfd40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfd40c4a70 0x7fbfd40c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33774/0 (socket says 172.21.15.154:33774) 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 -- 172.21.15.154:0/1247669785 learned_addr learned my addr 172.21.15.154:0/1247669785 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd259c700 1 --2- 172.21.15.154:0/1247669785 >> v2:172.21.15.191:3300/0 conn(0x7fbfd4155190 0x7fbfd406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd2d9d700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c6b00 0x7fbfd40c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 -- 172.21.15.154:0/1247669785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c6b00 msgr2=0x7fbfd40c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c6b00 0x7fbfd40c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 -- 172.21.15.154:0/1247669785 >> v2:172.21.15.191:3300/0 conn(0x7fbfd4155190 msgr2=0x7fbfd406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 --2- 172.21.15.154:0/1247669785 >> v2:172.21.15.191:3300/0 conn(0x7fbfd4155190 0x7fbfd406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 -- 172.21.15.154:0/1247669785 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40c6f90 con 0x7fbfd40c4a70 2024-03-14T09:01:34.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.033+0000 7fbfd359e700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd40c4a70 0x7fbfd40c4f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfd408b4d0 tx=0x7fbfcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e73b6d86429e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd1d9b700 1 -- 172.21.15.154:0/1247669785 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00e070 con 0x7fbfd40c4a70 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd1d9b700 1 -- 172.21.15.154:0/1247669785 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc00b9c0 con 0x7fbfd40c4a70 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd1d9b700 1 -- 172.21.15.154:0/1247669785 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00bd70 con 0x7fbfd40c4a70 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 -- 172.21.15.154:0/1247669785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd40c4a70 msgr2=0x7fbfd40c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd40c4a70 0x7fbfd40c4f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfd408b4d0 tx=0x7fbfcc00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 -- 172.21.15.154:0/1247669785 shutdown_connections 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd40c4a70 0x7fbfd40c4f00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 --2- 172.21.15.154:0/1247669785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c6b00 0x7fbfd40c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 --2- 172.21.15.154:0/1247669785 >> v2:172.21.15.191:3300/0 conn(0x7fbfd4155190 0x7fbfd406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 -- 172.21.15.154:0/1247669785 >> 172.21.15.154:0/1247669785 conn(0x7fbfd40abc80 msgr2=0x7fbfd40ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 -- 172.21.15.154:0/1247669785 shutdown_connections 2024-03-14T09:01:34.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.034+0000 7fbfd9201700 1 -- 172.21.15.154:0/1247669785 wait complete. 2024-03-14T09:01:34.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 Processor -- start 2024-03-14T09:01:34.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 -- start start 2024-03-14T09:01:34.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c4a70 0x7fbfd41ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd4158ad0 0x7fbfd41eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40af740 con 0x7fbfd4155190 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41583d0 con 0x7fbfd4158ad0 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd9201700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40af990 con 0x7fbfd40c4a70 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33780/0 (socket says 172.21.15.154:33780) 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.036+0000 7fbfd259c700 1 -- 172.21.15.154:0/3433172620 learned_addr learned my addr 172.21.15.154:0/3433172620 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd359e700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd4158ad0 0x7fbfd41eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd2d9d700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c4a70 0x7fbfd41ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 -- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c4a70 msgr2=0x7fbfd41ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c4a70 0x7fbfd41ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 -- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd4158ad0 msgr2=0x7fbfd41eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd4158ad0 0x7fbfd41eb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 -- 172.21.15.154:0/3433172620 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009b30 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd259c700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400d800 tx=0x7fbfc400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40125f0 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc4012750 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40afc70 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4012b00 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.037+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41ed8e0 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.038+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40697d0 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.038+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbfc4012c60 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.039+0000 7fbfbb7fe700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 0x7fbfc006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.039+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfc4089180 con 0x7fbfd4155190 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.039+0000 7fbfd2d9d700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 0x7fbfc006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.039+0000 7fbfd2d9d700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 0x7fbfc006ccf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4152680 tx=0x7fbfc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:34.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.043+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbfc4054520 con 0x7fbfd4155190 2024-03-14T09:01:34.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.280+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbfd40c7730 con 0x7fbfd4155190 2024-03-14T09:01:34.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.280+0000 7fbfbb7fe700 1 -- 172.21.15.154:0/3433172620 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbfc4057b40 con 0x7fbfd4155190 2024-03-14T09:01:34.280 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 msgr2=0x7fbfc006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 0x7fbfc006ccf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4152680 tx=0x7fbfc8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 msgr2=0x7fbfd40c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400d800 tx=0x7fbfc400dbc0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 shutdown_connections 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd4158ad0 0x7fbfd41eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfd40c4a70 0x7fbfd41ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc006a830 0x7fbfc006ccf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 --2- 172.21.15.154:0/3433172620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfd4155190 0x7fbfd40c16b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:34.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 >> 172.21.15.154:0/3433172620 conn(0x7fbfd40abc80 msgr2=0x7fbfd40ab450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:34.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 shutdown_connections 2024-03-14T09:01:34.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:34.282+0000 7fbfd9201700 1 -- 172.21.15.154:0/3433172620 wait complete. 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:01:34.294 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:01:34.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.295 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:01:34.312 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:01:34.312 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:01:34.312 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:01:34.312 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:34.312 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-03-14T09:01:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:01:34.314 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:34.314 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-14T09:01:34.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.314 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:01:34.314 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:01:34.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2024-03-14T09:01:34.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2024-03-14T09:01:34.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2024-03-14T09:01:34.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-14T09:01:34.315 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:01:34.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.483 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:01:34.511 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:01:34.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.512 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:01:34.587 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:01:34.587 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:01:34.587 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:01:34.587 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:01:34.587 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:01:34.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:01:34.588 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:01:34.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.816 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:01:34.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.833 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2024-03-14T09:01:34.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.891 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:01:34.891 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:01:34.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:01:34.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-14T09:01:34.891 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:01:34.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-14T09:01:34.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:34.987 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-14T09:01:34.988 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:01:35.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.212 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:01:35.212 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:01:35.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:01:35.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2024-03-14T09:01:35.212 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:01:35.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:01:35.310 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:01:35.326 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:01:35.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:01:35.327 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:01:35.384 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T09:01:35.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:35.471 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:35.471 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:01:35.566 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:01:35.567 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:01:35.567 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:01:35.584 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:01:35.585 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:01:35.587 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:01:35.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:01:35.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.615 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:35.706 INFO:teuthology.orchestra.run.smithi154.stdout:client4653 2024-03-14T09:01:35.707 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:35.707 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:35.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.724 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:35.813 INFO:teuthology.orchestra.run.smithi154.stdout:client4653 2024-03-14T09:01:35.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.814 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:35.860 INFO:teuthology.orchestra.run.smithi154.stdout:client4653 2024-03-14T09:01:35.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:35.861 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.client4653/status') 2024-03-14T09:01:35.915 INFO:teuthology.orchestra.run.smithi154.stdout:instance: client.4653 (3)192.168.0.1:0/2012048998 2024-03-14T09:01:35.915 INFO:teuthology.orchestra.run.smithi154.stdout:blocklisted: false 2024-03-14T09:01:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records in 2024-03-14T09:01:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:0+1 records out 2024-03-14T09:01:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:69 bytes copied, 2.1461e-05 s, 3.2 MB/s 2024-03-14T09:01:35.917 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-14T09:01:35.917 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-14T09:01:35.917 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:01:35.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:01:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 Processor -- start 2024-03-14T09:01:36.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 -- start start 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3c0014c5c0 0x7f3c00152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3c000c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c00070e10 con 0x7f3c00072290 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c000c1bb0 con 0x7f3c0014c5c0 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3c05375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000c6170 con 0x7f3c00072710 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3bfe7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3c00072290 0x7f3c000c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52596/0 (socket says 172.21.15.154:52596) 2024-03-14T09:01:36.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.039+0000 7f3bfe7fc700 1 -- 172.21.15.154:0/57537069 learned_addr learned my addr 172.21.15.154:0/57537069 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419558883 0 0) 0x7f3c00070e10 con 0x7f3c00072290 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bff7fe700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3c000c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bec0035b0 con 0x7f3c00072290 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/57537069 >> v2:172.21.15.191:3300/0 conn(0x7f3c0014c5c0 0x7f3c00152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207675466 0 0) 0x7f3bec0035b0 con 0x7f3c00072290 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 msgr2=0x7f3c000c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3c000c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 >> v2:172.21.15.191:3300/0 conn(0x7f3c0014c5c0 msgr2=0x7f3c00152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 --2- 172.21.15.154:0/57537069 >> v2:172.21.15.191:3300/0 conn(0x7f3c0014c5c0 0x7f3c00152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0014c9a0 con 0x7f3c00072290 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f3bf0001670 con 0x7f3c00072290 2024-03-14T09:01:36.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 >> v1:172.21.15.154:6789/0 conn(0x7f3c00072290 legacy=0x7f3c000c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3bec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3bec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bec004d90 0x7f3bec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000c1bb0 con 0x7f3bec003a00 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c000c6170 con 0x7f3bec004d90 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec00ba40 con 0x7f3c00072710 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3bec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfe7fc700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3bec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bff7fe700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bec004d90 0x7f3bec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 -- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 msgr2=0x7f3bec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c00072710 0x7f3bec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 -- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bec004d90 msgr2=0x7f3bec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:36.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bec004d90 0x7f3bec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.040+0000 7f3bfeffd700 1 -- 172.21.15.154:0/57537069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0035b0 con 0x7f3bec003a00 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bff7fe700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bec004d90 0x7f3bec005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3bec003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009b60 tx=0x7f3bec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222f3a168c60f45e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800e6c0 con 0x7f3bec003a00 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be800e820 con 0x7f3bec003a00 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bfdffb700 1 -- 172.21.15.154:0/57537069 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800eb90 con 0x7f3bec003a00 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 -- 172.21.15.154:0/57537069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 msgr2=0x7f3bec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 --2- 172.21.15.154:0/57537069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3bec003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009b60 tx=0x7f3bec005880 comp rx=0 tx=0).stop 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3bff7fe700 1 -- 172.21.15.154:0/57537069 reap_dead start 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 -- 172.21.15.154:0/57537069 shutdown_connections 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 -- 172.21.15.154:0/57537069 >> 172.21.15.154:0/57537069 conn(0x7f3c0006a770 msgr2=0x7f3c0006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 -- 172.21.15.154:0/57537069 shutdown_connections 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.041+0000 7f3c05375700 1 -- 172.21.15.154:0/57537069 wait complete. 2024-03-14T09:01:36.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 Processor -- start 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 -- start start 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 0x7f3c00148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c001490f0 0x7f3c00155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00149570 con 0x7f3bec003a00 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c001557d0 con 0x7f3c001490f0 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.043+0000 7f3c05375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0014f230 con 0x7f3c000aba80 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33794/0 (socket says 172.21.15.154:33794) 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 -- 172.21.15.154:0/8267033 learned_addr learned my addr 172.21.15.154:0/8267033 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bff7fe700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c001490f0 0x7f3c00155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfe7fc700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 0x7f3c00148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 -- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 msgr2=0x7f3c00148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 0x7f3c00148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 -- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c001490f0 msgr2=0x7f3c00155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c001490f0 0x7f3c00155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 -- 172.21.15.154:0/8267033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c001540e0 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfe7fc700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 0x7f3c00148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bfeffd700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3bec004760 tx=0x7f3be80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be800e6c0 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8003ab0 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c001543c0 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8003e60 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.044+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0014fe70 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.045+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c000697d0 con 0x7f3bec003a00 2024-03-14T09:01:36.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.046+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3be8013470 con 0x7f3bec003a00 2024-03-14T09:01:36.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.046+0000 7f3bf77fe700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 0x7f3be006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:36.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.046+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be808a5a0 con 0x7f3bec003a00 2024-03-14T09:01:36.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.046+0000 7f3bfe7fc700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 0x7f3be006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:36.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.047+0000 7f3bfe7fc700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 0x7f3be006ccf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000fb0 tx=0x7f3bf0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:36.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.050+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3be8055a50 con 0x7f3bec003a00 2024-03-14T09:01:36.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.291+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c000ac670 con 0x7f3bec003a00 2024-03-14T09:01:36.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.291+0000 7f3bf77fe700 1 -- 172.21.15.154:0/8267033 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3be8059070 con 0x7f3bec003a00 2024-03-14T09:01:36.291 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:01:36.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 msgr2=0x7f3be006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 0x7f3be006ccf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000fb0 tx=0x7f3bf0009040 comp rx=0 tx=0).stop 2024-03-14T09:01:36.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 msgr2=0x7f3c0014f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:36.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3bec004760 tx=0x7f3be80047c0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 shutdown_connections 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c001490f0 0x7f3c00155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3be006a830 0x7f3be006ccf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bec003a00 0x7f3c0014f930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 --2- 172.21.15.154:0/8267033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c000aba80 0x7f3c00148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 >> 172.21.15.154:0/8267033 conn(0x7f3c0006a770 msgr2=0x7f3c000a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 shutdown_connections 2024-03-14T09:01:36.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:36.293+0000 7f3c05375700 1 -- 172.21.15.154:0/8267033 wait complete. 2024-03-14T09:01:36.305 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-14T09:01:36.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:01:36.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:01:36.306 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:01:36.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.306 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:01:36.326 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:01:36.326 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:01:36.327 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42368sec preferred_lft 42368sec 2024-03-14T09:01:36.328 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:01:36.328 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:01:36.328 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-14T09:01:36.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-14T09:01:36.328 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:01:36.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.514 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:01:36.542 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:01:36.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.544 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:01:36.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.615 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:01:36.616 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:01:36.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:01:36.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:01:36.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:01:36.616 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:01:36.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.892 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:01:36.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.910 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-03-14T09:01:36.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:36.967 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:01:36.967 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:01:36.967 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:01:36.967 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-14T09:01:36.967 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:01:37.062 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-14T09:01:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.062 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:01:37.062 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:01:37.062 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-14T09:01:37.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-14T09:01:37.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-14T09:01:37.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-14T09:01:37.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-14T09:01:37.063 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:01:37.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.283 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:01:37.283 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:01:37.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:01:37.283 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2024-03-14T09:01:37.283 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:01:37.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:01:37.381 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:01:37.398 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:01:37.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:01:37.398 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:01:37.454 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T09:01:37.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.455 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:37.541 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:37.541 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:37.630 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:01:37.631 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:01:37.649 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:01:37.650 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:01:37.650 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:01:37.650 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:01:37.650 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:01:37.650 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:01:37.652 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:01:37.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.652 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:01:37.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.679 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:37.752 INFO:teuthology.orchestra.run.smithi191.stdout:client4478 2024-03-14T09:01:37.753 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:37.753 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:37.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.811 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:37.883 INFO:teuthology.orchestra.run.smithi191.stdout:client4478 2024-03-14T09:01:37.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.883 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:01:37.956 INFO:teuthology.orchestra.run.smithi191.stdout:client4478 2024-03-14T09:01:37.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:37.956 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.client4478/status') 2024-03-14T09:01:38.037 INFO:teuthology.orchestra.run.smithi191.stdout:instance: client.4478 (3)192.168.0.1:0/636573391 2024-03-14T09:01:38.038 INFO:teuthology.orchestra.run.smithi191.stdout:blocklisted: false 2024-03-14T09:01:38.038 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records in 2024-03-14T09:01:38.038 INFO:teuthology.orchestra.run.smithi191.stderr:0+1 records out 2024-03-14T09:01:38.038 INFO:teuthology.orchestra.run.smithi191.stderr:68 bytes copied, 3.4338e-05 s, 2.0 MB/s 2024-03-14T09:01:38.039 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-14T09:01:38.076 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:38.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.212+0000 7f2673e66700 1 Processor -- start 2024-03-14T09:01:38.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.212+0000 7f2673e66700 1 -- start start 2024-03-14T09:01:38.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.213+0000 7f2673e66700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f266c158c10 0x7f266c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.213+0000 7f2673e66700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 0x7f266c06abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.213+0000 7f2673e66700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c0c1780 con 0x7f266c06d420 2024-03-14T09:01:38.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.213+0000 7f2673e66700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c06f660 con 0x7f266c158c10 2024-03-14T09:01:38.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.213+0000 7f2673e66700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c153180 con 0x7f266c06a770 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2672403700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f266c06d420 0x7f266c06d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52604/0 (socket says 172.21.15.154:52604) 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2672403700 1 -- 172.21.15.154:0/4163518601 learned_addr learned my addr 172.21.15.154:0/4163518601 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 --2- 172.21.15.154:0/4163518601 >> v2:172.21.15.191:3300/0 conn(0x7f266c158c10 0x7f266c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671401700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 0x7f266c06abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2670c00700 1 -- 172.21.15.154:0/4163518601 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070556998 0 0) 0x7f266c0c1780 con 0x7f266c06d420 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2670c00700 1 -- 172.21.15.154:0/4163518601 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26580035b0 con 0x7f266c06d420 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 -- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 msgr2=0x7f266c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 0x7f266c06abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 -- 172.21.15.154:0/4163518601 >> v1:172.21.15.154:6789/0 conn(0x7f266c06d420 legacy=0x7f266c06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 -- 172.21.15.154:0/4163518601 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0c25e0 con 0x7f266c158c10 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671401700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 0x7f266c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2671c02700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c158c10 0x7f266c152c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e216ef9d17462d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2670c00700 1 -- 172.21.15.154:0/4163518601 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c011830 con 0x7f266c158c10 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2670c00700 1 -- 172.21.15.154:0/4163518601 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c011990 con 0x7f266c158c10 2024-03-14T09:01:38.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.214+0000 7f2670c00700 1 -- 172.21.15.154:0/4163518601 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c011d00 con 0x7f266c158c10 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 -- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c158c10 msgr2=0x7f266c152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c158c10 0x7f266c152c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c009b80 comp rx=0 tx=0).stop 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 -- 172.21.15.154:0/4163518601 shutdown_connections 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c158c10 0x7f266c152c40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 --2- 172.21.15.154:0/4163518601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06a770 0x7f266c06abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 -- 172.21.15.154:0/4163518601 >> 172.21.15.154:0/4163518601 conn(0x7f266c0c3880 msgr2=0x7f266c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 -- 172.21.15.154:0/4163518601 shutdown_connections 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.215+0000 7f2673e66700 1 -- 172.21.15.154:0/4163518601 wait complete. 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.216+0000 7f2673e66700 1 Processor -- start 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 -- start start 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c06a770 0x7f266c14baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 0x7f266c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1533d0 con 0x7f266c158c10 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c156410 con 0x7f266c06a770 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2673e66700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c154c80 con 0x7f266c06d420 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2672403700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2672403700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33804/0 (socket says 172.21.15.154:33804) 2024-03-14T09:01:38.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.217+0000 7f2672403700 1 -- 172.21.15.154:0/4045717112 learned_addr learned my addr 172.21.15.154:0/4045717112 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2671c02700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c06a770 0x7f266c14baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2671401700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 0x7f266c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 -- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 msgr2=0x7f266c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 0x7f266c06e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 -- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c06a770 msgr2=0x7f266c14baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c06a770 0x7f266c14baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 -- 172.21.15.154:0/4045717112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26680094a0 con 0x7f266c158c10 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2671401700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 0x7f266c06e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2672403700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f266400cb40 tx=0x7f266400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400aaa0 con 0x7f266c158c10 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266400ac00 con 0x7f266c158c10 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0094a0 con 0x7f266c158c10 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664012400 con 0x7f266c158c10 2024-03-14T09:01:38.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.218+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c155240 con 0x7f266c158c10 2024-03-14T09:01:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.219+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0697d0 con 0x7f266c158c10 2024-03-14T09:01:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.220+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f266401b3b0 con 0x7f266c158c10 2024-03-14T09:01:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.220+0000 7f26627fc700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 0x7f265006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.220+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2664088a30 con 0x7f266c158c10 2024-03-14T09:01:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.220+0000 7f2671c02700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 0x7f265006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.222 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.221+0000 7f2671c02700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 0x7f265006ccf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f265c0090b0 tx=0x7f265c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.224+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2664053ee0 con 0x7f266c158c10 2024-03-14T09:01:38.421 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.421+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f266c0c1560 con 0x7f266c158c10 2024-03-14T09:01:38.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.422+0000 7f26627fc700 1 -- 172.21.15.154:0/4045717112 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f2664017090 con 0x7f266c158c10 2024-03-14T09:01:38.422 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:38.422 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:38.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 msgr2=0x7f265006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 0x7f265006ccf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f265c0090b0 tx=0x7f265c01c040 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 msgr2=0x7f266c1537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f266400cb40 tx=0x7f266400ce50 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 shutdown_connections 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f265006a830 0x7f265006ccf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f266c158c10 0x7f266c1537f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f266c06a770 0x7f266c14baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 --2- 172.21.15.154:0/4045717112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f266c06d420 0x7f266c06e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 >> 172.21.15.154:0/4045717112 conn(0x7f266c0c3880 msgr2=0x7f266c1583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 shutdown_connections 2024-03-14T09:01:38.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.424+0000 7f2673e66700 1 -- 172.21.15.154:0/4045717112 wait complete. 2024-03-14T09:01:38.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:38.437 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.562+0000 7f9f4a4f5700 1 Processor -- start 2024-03-14T09:01:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.562+0000 7f9f4a4f5700 1 -- start start 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f4a4f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f4a4f5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9f44158b00 0x7f9f4415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f4a4f5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440c2e30 con 0x7f9f44151e50 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f4a4f5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f44154c70 con 0x7f9f44158b00 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f4a4f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44154f50 con 0x7f9f44158160 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f43fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9f44151e50 0x7f9f440ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52616/0 (socket says 172.21.15.154:52616) 2024-03-14T09:01:38.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f43fff700 1 -- 172.21.15.154:0/3623337593 learned_addr learned my addr 172.21.15.154:0/3623337593 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f48a92700 1 --2- 172.21.15.154:0/3623337593 >> v2:172.21.15.191:3300/0 conn(0x7f9f44158b00 0x7f9f4415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f437fe700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f42ffd700 1 -- 172.21.15.154:0/3623337593 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180657389 0 0) 0x7f9f440c2e30 con 0x7f9f44151e50 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f42ffd700 1 -- 172.21.15.154:0/3623337593 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f300035b0 con 0x7f9f44151e50 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f48a92700 1 -- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 msgr2=0x7f9f441585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f48a92700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.563+0000 7f9f48a92700 1 -- 172.21.15.154:0/3623337593 >> v1:172.21.15.154:6789/0 conn(0x7f9f44151e50 legacy=0x7f9f440ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f48a92700 1 -- 172.21.15.154:0/3623337593 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44155230 con 0x7f9f44158b00 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f48a92700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44158b00 0x7f9f4415f210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b24db2dd4d19b72 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f42ffd700 1 -- 172.21.15.154:0/3623337593 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00e070 con 0x7f9f44158b00 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f42ffd700 1 -- 172.21.15.154:0/3623337593 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c00bb20 con 0x7f9f44158b00 2024-03-14T09:01:38.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f42ffd700 1 -- 172.21.15.154:0/3623337593 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c012430 con 0x7f9f44158b00 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44158b00 msgr2=0x7f9f4415f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44158b00 0x7f9f4415f210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/3623337593 shutdown_connections 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44158b00 0x7f9f4415f210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/3623337593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/3623337593 >> 172.21.15.154:0/3623337593 conn(0x7f9f440c3ad0 msgr2=0x7f9f440c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/3623337593 shutdown_connections 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.564+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/3623337593 wait complete. 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.565+0000 7f9f4a4f5700 1 Processor -- start 2024-03-14T09:01:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 -- start start 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44151e50 0x7f9f441483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4414c200 con 0x7f9f44158b00 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44157ec0 con 0x7f9f44151e50 2024-03-14T09:01:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f4a4f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c67c0 con 0x7f9f44158160 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f48a92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.566+0000 7f9f48a92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33806/0 (socket says 172.21.15.154:33806) 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 -- 172.21.15.154:0/622827103 learned_addr learned my addr 172.21.15.154:0/622827103 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f43fff700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44151e50 0x7f9f441483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f437fe700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 -- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 msgr2=0x7f9f441ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441ed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 -- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44151e50 msgr2=0x7f9f441483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44151e50 0x7f9f441483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 -- 172.21.15.154:0/622827103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009a70 con 0x7f9f44158b00 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f48a92700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c013000 tx=0x7f9f3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0129f0 con 0x7f9f44158b00 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c020070 con 0x7f9f44158b00 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4414bf20 con 0x7f9f44158b00 2024-03-14T09:01:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c012db0 con 0x7f9f44158b00 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.567+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441ea9b0 con 0x7f9f44158b00 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.568+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f441ea530 con 0x7f9f44158b00 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.569+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9f3c004040 con 0x7f9f44158b00 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.569+0000 7f9f40ff9700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 0x7f9f2406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.569+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f3c022070 con 0x7f9f44158b00 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.569+0000 7f9f43fff700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 0x7f9f2406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.570+0000 7f9f43fff700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 0x7f9f2406ccf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007fd0 tx=0x7f9f2c007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.574+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9f441ea530 con 0x7f9f44158b00 2024-03-14T09:01:38.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.764+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f440c6930 con 0x7f9f44158b00 2024-03-14T09:01:38.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.764+0000 7f9f40ff9700 1 -- 172.21.15.154:0/622827103 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f440c6930 con 0x7f9f44158b00 2024-03-14T09:01:38.764 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:38.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 msgr2=0x7f9f2406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 0x7f9f2406ccf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007fd0 tx=0x7f9f2c007dc0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 msgr2=0x7f9f441ed5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c013000 tx=0x7f9f3c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 shutdown_connections 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9f2406a830 0x7f9f2406ccf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f44158b00 0x7f9f441ed5c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f44151e50 0x7f9f441483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 --2- 172.21.15.154:0/622827103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f44158160 0x7f9f441ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 >> 172.21.15.154:0/622827103 conn(0x7f9f440c3ad0 msgr2=0x7f9f440ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.766+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 shutdown_connections 2024-03-14T09:01:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.767+0000 7f9f4a4f5700 1 -- 172.21.15.154:0/622827103 wait complete. 2024-03-14T09:01:38.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:38.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.904+0000 7f07fe6a6700 1 Processor -- start 2024-03-14T09:01:38.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.904+0000 7f07fe6a6700 1 -- start start 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fe6a6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f07f8153010 0x7f07f814b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fe6a6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f806cff0 con 0x7f07f8154c40 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fe6a6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f806d2d0 con 0x7f07f814b290 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fe6a6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f814bbb0 con 0x7f07f8153010 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fcc43700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f07f8154c40 0x7f07f81550d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52622/0 (socket says 172.21.15.154:52622) 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07fcc43700 1 -- 172.21.15.154:0/3495459212 learned_addr learned my addr 172.21.15.154:0/3495459212 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07f77fe700 1 --2- 172.21.15.154:0/3495459212 >> v2:172.21.15.191:3300/0 conn(0x7f07f8153010 0x7f07f814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751843927 0 0) 0x7f07f806cff0 con 0x7f07f8154c40 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.905+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e00035b0 con 0x7f07f8154c40 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2788223264 0 0) 0x7f07e00035b0 con 0x7f07f8154c40 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 >> v2:172.21.15.191:3300/0 conn(0x7f07f8153010 msgr2=0x7f07f814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:38.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 --2- 172.21.15.154:0/3495459212 >> v2:172.21.15.191:3300/0 conn(0x7f07f8153010 0x7f07f814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 >> v1:172.21.15.191:6789/0 conn(0x7f07f814b290 legacy=0x7f07f8152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f806d5b0 con 0x7f07f8154c40 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f07f00019f0 con 0x7f07f8154c40 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 >> v1:172.21.15.154:6789/0 conn(0x7f07f8154c40 legacy=0x7f07f81550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0003a00 0x7f07e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07e0004850 0x7f07e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 0x7f07e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e000c180 con 0x7f07e0005210 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e000c2f0 con 0x7f07e0004850 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e000c4b0 con 0x7f07e0003a00 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 0x7f07e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f7fff700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0003a00 0x7f07e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07f77fe700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07e0004850 0x7f07e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 -- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0003a00 msgr2=0x7f07e0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0003a00 0x7f07e0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 -- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07e0004850 msgr2=0x7f07e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07e0004850 0x7f07e0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 -- 172.21.15.154:0/3495459212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00035b0 con 0x7f07e0005210 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 -- 172.21.15.154:0/3495459212 reap_dead start 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.906+0000 7f07fcc43700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 0x7f07e000ba10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07f000a830 tx=0x7f07e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c329d9ead4a5bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000a2f0 con 0x7f07e0005210 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f0012620 con 0x7f07e0005210 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07f6ffd700 1 -- 172.21.15.154:0/3495459212 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00129d0 con 0x7f07e0005210 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 -- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 msgr2=0x7f07e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 0x7f07e000ba10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07f000a830 tx=0x7f07e0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 -- 172.21.15.154:0/3495459212 shutdown_connections 2024-03-14T09:01:38.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/3495459212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e0005210 0x7f07e000ba10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 -- 172.21.15.154:0/3495459212 >> 172.21.15.154:0/3495459212 conn(0x7f07f80c5e70 msgr2=0x7f07f80c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 -- 172.21.15.154:0/3495459212 shutdown_connections 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.907+0000 7f07fe6a6700 1 -- 172.21.15.154:0/3495459212 wait complete. 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.908+0000 7f07fe6a6700 1 Processor -- start 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.908+0000 7f07fe6a6700 1 -- start start 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 0x7f07f81506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80c3b10 0x7f07f8141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f80c4500 con 0x7f07f8143750 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8141fb0 con 0x7f07f80c3b10 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fe6a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f814f280 con 0x7f07e0005210 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fcc43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fcc43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33818/0 (socket says 172.21.15.154:33818) 2024-03-14T09:01:38.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07fcc43700 1 -- 172.21.15.154:0/1747235006 learned_addr learned my addr 172.21.15.154:0/1747235006 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07f77fe700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80c3b10 0x7f07f8141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.909+0000 7f07f7fff700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 0x7f07f81506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 -- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 msgr2=0x7f07f81506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 0x7f07f81506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 -- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80c3b10 msgr2=0x7f07f8141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80c3b10 0x7f07f8141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 -- 172.21.15.154:0/1747235006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f80c4220 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07f7fff700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 0x7f07f81506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fcc43700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07f8155020 tx=0x7f07f0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0020070 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000a210 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f8151b30 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00019d0 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.910+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f80ab090 con 0x7f07f8143750 2024-03-14T09:01:38.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.911+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f8067bd0 con 0x7f07f8143750 2024-03-14T09:01:38.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.911+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f07f001a4c0 con 0x7f07f8143750 2024-03-14T09:01:38.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.912+0000 7f07f4ff9700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 0x7f07e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:38.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.912+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07f008c440 con 0x7f07f8143750 2024-03-14T09:01:38.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.912+0000 7f07f7fff700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 0x7f07e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:38.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.912+0000 7f07f7fff700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 0x7f07e406ccf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07f81441f0 tx=0x7f07ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:38.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:38.916+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f07f0057a10 con 0x7f07f8143750 2024-03-14T09:01:39.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.111+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07f814f6d0 con 0x7f07f8143750 2024-03-14T09:01:39.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.111+0000 7f07f4ff9700 1 -- 172.21.15.154:0/1747235006 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07f814f6d0 con 0x7f07f8143750 2024-03-14T09:01:39.111 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:39.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 msgr2=0x7f07e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 0x7f07e406ccf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f07f81441f0 tx=0x7f07ec009040 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 msgr2=0x7f07f8143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07f8155020 tx=0x7f07f0001d80 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 shutdown_connections 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f07e406a830 0x7f07e406ccf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07f8143750 0x7f07f8143be0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07e0005210 0x7f07f81506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 --2- 172.21.15.154:0/1747235006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80c3b10 0x7f07f8141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.113+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 >> 172.21.15.154:0/1747235006 conn(0x7f07f80c5e70 msgr2=0x7f07f80c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.114+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 shutdown_connections 2024-03-14T09:01:39.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.114+0000 7f07fe6a6700 1 -- 172.21.15.154:0/1747235006 wait complete. 2024-03-14T09:01:39.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:39.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.249+0000 7f4fb0fd5700 1 Processor -- start 2024-03-14T09:01:39.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.249+0000 7f4fb0fd5700 1 -- start start 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 0x7f4fac06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fac06d510 0x7f4fac0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 0x7f4fac0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0c3e00 con 0x7f4fac06d510 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0c7bb0 con 0x7f4fac0c31c0 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fb0fd5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0c7e90 con 0x7f4fac06c670 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fa9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fac06d510 0x7f4fac0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fa9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fac06d510 0x7f4fac0c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33820/0 (socket says 172.21.15.154:33820) 2024-03-14T09:01:39.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.250+0000 7f4fa9d9b700 1 -- 172.21.15.154:0/22883177 learned_addr learned my addr 172.21.15.154:0/22883177 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4faa59c700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 0x7f4fac06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4faad9d700 1 --2- 172.21.15.154:0/22883177 >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 0x7f4fac0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 -- 172.21.15.154:0/22883177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 msgr2=0x7f4fac06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 0x7f4fac06cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 -- 172.21.15.154:0/22883177 >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 msgr2=0x7f4fac0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/22883177 >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 0x7f4fac0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 -- 172.21.15.154:0/22883177 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0c8170 con 0x7f4fac06d510 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4faad9d700 1 --2- 172.21.15.154:0/22883177 >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 0x7f4fac0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4faa59c700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 0x7f4fac06cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06d510 0x7f4fac0c6140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0a7b30 tx=0x7f4f9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad1435ad6198ef server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa959a700 1 -- 172.21.15.154:0/22883177 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00b870 con 0x7f4fac06d510 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa959a700 1 -- 172.21.15.154:0/22883177 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00b9d0 con 0x7f4fac06d510 2024-03-14T09:01:39.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.251+0000 7f4fa959a700 1 -- 172.21.15.154:0/22883177 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00bd60 con 0x7f4fac06d510 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/22883177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06d510 msgr2=0x7f4fac0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06d510 0x7f4fac0c6140 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0a7b30 tx=0x7f4f9c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/22883177 shutdown_connections 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/22883177 >> v2:172.21.15.191:3300/0 conn(0x7f4fac0c31c0 0x7f4fac0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06c670 0x7f4fac06cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/22883177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06d510 0x7f4fac0c6140 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/22883177 >> 172.21.15.154:0/22883177 conn(0x7f4fac0a9bf0 msgr2=0x7f4fac0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/22883177 shutdown_connections 2024-03-14T09:01:39.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.252+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/22883177 wait complete. 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.253+0000 7f4fb0fd5700 1 Processor -- start 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.253+0000 7f4fb0fd5700 1 -- start start 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06d510 0x7f4fac0ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fac0c31c0 0x7f4fac0ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1ea820 con 0x7f4fac06c670 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0ad2d0 con 0x7f4fac0c31c0 2024-03-14T09:01:39.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1ea1d0 con 0x7f4fac06d510 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33834/0 (socket says 172.21.15.154:33834) 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 -- 172.21.15.154:0/426487156 learned_addr learned my addr 172.21.15.154:0/426487156 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faad9d700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fac0c31c0 0x7f4fac0ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06d510 0x7f4fac0ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 -- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06d510 msgr2=0x7f4fac0ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06d510 0x7f4fac0ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 -- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fac0c31c0 msgr2=0x7f4fac0ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fac0c31c0 0x7f4fac0ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 -- 172.21.15.154:0/426487156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c009550 con 0x7f4fac06c670 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4faa59c700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c7f0 tx=0x7f4fa000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0011800 con 0x7f4fac06c670 2024-03-14T09:01:39.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0011960 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac1ea4b0 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0011d10 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.254+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac1e9e80 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.255+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0697d0 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.256+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4fa0018360 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.256+0000 7f4f9affd700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 0x7f4f8c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.256+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fa0088c90 con 0x7f4fac06c670 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.257+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 0x7f4f8c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.257+0000 7f4fa9d9b700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 0x7f4f8c06ccf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c014040 tx=0x7f4f9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.261+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4fa0054140 con 0x7f4fac06c670 2024-03-14T09:01:39.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.452+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fac0a71b0 con 0x7f4fac06c670 2024-03-14T09:01:39.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.453+0000 7f4f9affd700 1 -- 172.21.15.154:0/426487156 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fa0057760 con 0x7f4fac06c670 2024-03-14T09:01:39.453 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 msgr2=0x7f4f8c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 0x7f4f8c06ccf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c014040 tx=0x7f4f9c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 msgr2=0x7f4fac1dd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000c7f0 tx=0x7f4fa000cbb0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 shutdown_connections 2024-03-14T09:01:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fac0c31c0 0x7f4fac0ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f8c06a830 0x7f4f8c06ccf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fac06c670 0x7f4fac1dd9e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 --2- 172.21.15.154:0/426487156 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fac06d510 0x7f4fac0ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 >> 172.21.15.154:0/426487156 conn(0x7f4fac0a9bf0 msgr2=0x7f4fac0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 shutdown_connections 2024-03-14T09:01:39.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.455+0000 7f4fb0fd5700 1 -- 172.21.15.154:0/426487156 wait complete. 2024-03-14T09:01:39.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:39.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.589+0000 7f436ed15700 1 Processor -- start 2024-03-14T09:01:39.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.589+0000 7f436ed15700 1 -- start start 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436ed15700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368158160 0x7f43681585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436ed15700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4368158b00 0x7f436815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436ed15700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680c2e30 con 0x7f4368151e50 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436ed15700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4368154c70 con 0x7f4368158b00 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436ed15700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4368154f50 con 0x7f4368158160 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436cab1700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4368151e50 0x7f43680ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48418/0 (socket says 172.21.15.154:48418) 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436cab1700 1 -- 172.21.15.154:0/3723224253 learned_addr learned my addr 172.21.15.154:0/3723224253 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4368158b00 0x7f436815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f4367fff700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368158160 0x7f43681585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 -- 172.21.15.154:0/3723224253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368158160 msgr2=0x7f43681585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368158160 0x7f43681585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 -- 172.21.15.154:0/3723224253 >> v1:172.21.15.154:6789/0 conn(0x7f4368151e50 legacy=0x7f43680ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 -- 172.21.15.154:0/3723224253 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368155230 con 0x7f4368158b00 2024-03-14T09:01:39.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.590+0000 7f436d2b2700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158b00 0x7f436815f210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f436000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53176c3501a992 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f43677fe700 1 -- 172.21.15.154:0/3723224253 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000e070 con 0x7f4368158b00 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f43677fe700 1 -- 172.21.15.154:0/3723224253 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000bb20 con 0x7f4368158b00 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f43677fe700 1 -- 172.21.15.154:0/3723224253 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360012430 con 0x7f4368158b00 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 -- 172.21.15.154:0/3723224253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158b00 msgr2=0x7f436815f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158b00 0x7f436815f210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4360009d90 tx=0x7f436000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 -- 172.21.15.154:0/3723224253 shutdown_connections 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158b00 0x7f436815f210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 --2- 172.21.15.154:0/3723224253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368158160 0x7f43681585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 -- 172.21.15.154:0/3723224253 >> 172.21.15.154:0/3723224253 conn(0x7f43680c3ad0 msgr2=0x7f43680c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 -- 172.21.15.154:0/3723224253 shutdown_connections 2024-03-14T09:01:39.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.592+0000 7f436ed15700 1 -- 172.21.15.154:0/3723224253 wait complete. 2024-03-14T09:01:39.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.593+0000 7f436ed15700 1 Processor -- start 2024-03-14T09:01:39.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.593+0000 7f436ed15700 1 -- start start 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368151e50 0x7f4368148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4368158b00 0x7f43681ed790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680acb60 con 0x7f4368158160 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43681570f0 con 0x7f4368158b00 2024-03-14T09:01:39.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436ed15700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4368157390 con 0x7f4368151e50 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f4367fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f4367fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59040/0 (socket says 172.21.15.154:59040) 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f4367fff700 1 -- 172.21.15.154:0/1452467299 learned_addr learned my addr 172.21.15.154:0/1452467299 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436cab1700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368151e50 0x7f4368148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.594+0000 7f436d2b2700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4368158b00 0x7f43681ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 -- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368151e50 msgr2=0x7f4368148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368151e50 0x7f4368148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 -- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4368158b00 msgr2=0x7f43681ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4368158b00 0x7f43681ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 -- 172.21.15.154:0/1452467299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009a70 con 0x7f4368158160 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f4367fff700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9b0 tx=0x7f435400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354009710 con 0x7f4368158160 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435400c850 con 0x7f4368158160 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4368157670 con 0x7f4368158160 2024-03-14T09:01:39.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400cbc0 con 0x7f4368158160 2024-03-14T09:01:39.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.595+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43680c6d50 con 0x7f4368158160 2024-03-14T09:01:39.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.596+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368157cc0 con 0x7f4368158160 2024-03-14T09:01:39.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.597+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f435400cd20 con 0x7f4368158160 2024-03-14T09:01:39.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.597+0000 7f43657fa700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 0x7f434c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.597+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4354089540 con 0x7f4368158160 2024-03-14T09:01:39.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.598+0000 7f436cab1700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 0x7f434c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.598+0000 7f436cab1700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 0x7f434c06ccf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f435c005910 tx=0x7f435c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.601+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4354054ad0 con 0x7f4368158160 2024-03-14T09:01:39.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.798+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43680697d0 con 0x7f4368158160 2024-03-14T09:01:39.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.799+0000 7f43657fa700 1 -- 172.21.15.154:0/1452467299 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f43540580f0 con 0x7f4368158160 2024-03-14T09:01:39.798 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:39.799 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 msgr2=0x7f434c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 0x7f434c06ccf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f435c005910 tx=0x7f435c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 msgr2=0x7f43680c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9b0 tx=0x7f435400ed70 comp rx=0 tx=0).stop 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 shutdown_connections 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4368158b00 0x7f43681ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4368151e50 0x7f4368148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f434c06a830 0x7f434c06ccf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 --2- 172.21.15.154:0/1452467299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4368158160 0x7f43680c6810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 >> 172.21.15.154:0/1452467299 conn(0x7f43680c3ad0 msgr2=0x7f436814da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 shutdown_connections 2024-03-14T09:01:39.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.801+0000 7f436ed15700 1 -- 172.21.15.154:0/1452467299 wait complete. 2024-03-14T09:01:39.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:39.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:39.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.937+0000 7f2d33c24700 1 Processor -- start 2024-03-14T09:01:39.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.937+0000 7f2d33c24700 1 -- start start 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d33c24700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2d2c0c1e90 0x7f2d2c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d33c24700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c13ade0 con 0x7f2d2c0c3250 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d33c24700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c06fb30 con 0x7f2d2c0a8540 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d33c24700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c06ec30 con 0x7f2d2c0c1e90 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d311bf700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2d2c0c3250 0x7f2d2c0c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48424/0 (socket says 172.21.15.154:48424) 2024-03-14T09:01:39.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.938+0000 7f2d311bf700 1 -- 172.21.15.154:0/474503602 learned_addr learned my addr 172.21.15.154:0/474503602 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 --2- 172.21.15.154:0/474503602 >> v2:172.21.15.191:3300/0 conn(0x7f2d2c0c1e90 0x7f2d2c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105237189 0 0) 0x7f2d28001110 con 0x7f2d2c0c3250 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d180035b0 con 0x7f2d2c0c3250 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243953435 0 0) 0x7f2d180035b0 con 0x7f2d2c0c3250 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 >> v2:172.21.15.191:3300/0 conn(0x7f2d2c0c1e90 msgr2=0x7f2d2c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 --2- 172.21.15.154:0/474503602 >> v2:172.21.15.191:3300/0 conn(0x7f2d2c0c1e90 0x7f2d2c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 >> v1:172.21.15.191:6789/0 conn(0x7f2d2c0a8540 legacy=0x7f2d2c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c06ef10 con 0x7f2d2c0c3250 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2d28001800 con 0x7f2d2c0c3250 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 >> v1:172.21.15.154:6789/0 conn(0x7f2d2c0c3250 legacy=0x7f2d2c0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d18003a00 0x7f2d18003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d18004850 0x7f2d18004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06ec30 con 0x7f2d18005330 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800c200 con 0x7f2d18004850 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800c3c0 con 0x7f2d18003a00 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d311bf700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d18004850 0x7f2d18004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d319c0700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d18003a00 0x7f2d18003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 -- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d18003a00 msgr2=0x7f2d18003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d18003a00 0x7f2d18003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 -- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d18004850 msgr2=0x7f2d18004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d18004850 0x7f2d18004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.939+0000 7f2d321c1700 1 -- 172.21.15.154:0/474503602 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180035b0 con 0x7f2d18005330 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d321c1700 1 -- 172.21.15.154:0/474503602 reap_dead start 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d321c1700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d1800ba10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009bd0 tx=0x7f2d18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4597b379beb25b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24003c20 con 0x7f2d18005330 2024-03-14T09:01:39.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2400f040 con 0x7f2d18005330 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d309be700 1 -- 172.21.15.154:0/474503602 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400e9d0 con 0x7f2d18005330 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 -- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 msgr2=0x7f2d1800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d1800ba10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009bd0 tx=0x7f2d18005880 comp rx=0 tx=0).stop 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 -- 172.21.15.154:0/474503602 shutdown_connections 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 --2- 172.21.15.154:0/474503602 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d1800ba10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 -- 172.21.15.154:0/474503602 >> 172.21.15.154:0/474503602 conn(0x7f2d2c0c3880 msgr2=0x7f2d2c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 -- 172.21.15.154:0/474503602 shutdown_connections 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.940+0000 7f2d33c24700 1 -- 172.21.15.154:0/474503602 wait complete. 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 Processor -- start 2024-03-14T09:01:39.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 -- start start 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 0x7f2d2c1db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 0x7f2d2c14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1e6a20 con 0x7f2d18005330 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1e6f50 con 0x7f2d2c1dbd30 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d33c24700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1db360 con 0x7f2d2c1dd4d0 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d319c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d319c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59060/0 (socket says 172.21.15.154:59060) 2024-03-14T09:01:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.942+0000 7f2d319c0700 1 -- 172.21.15.154:0/3255214277 learned_addr learned my addr 172.21.15.154:0/3255214277 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d321c1700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 0x7f2d2c14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d311bf700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 0x7f2d2c1db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 -- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 msgr2=0x7f2d2c1db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 0x7f2d2c1db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 -- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 msgr2=0x7f2d2c14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 0x7f2d2c14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 -- 172.21.15.154:0/3255214277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c1e6c70 con 0x7f2d18005330 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d321c1700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 0x7f2d2c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d311bf700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 0x7f2d2c1db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d319c0700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00f850 tx=0x7f2d1c00fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0050d0 con 0x7f2d18005330 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c005230 con 0x7f2d18005330 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1dad10 con 0x7f2d18005330 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c003680 con 0x7f2d18005330 2024-03-14T09:01:39.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.943+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c139210 con 0x7f2d18005330 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.944+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c0697d0 con 0x7f2d18005330 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.945+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2d1c0037e0 con 0x7f2d18005330 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.945+0000 7f2d227fc700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 0x7f2d1006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.945+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d1c0893f0 con 0x7f2d18005330 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.945+0000 7f2d311bf700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 0x7f2d1006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.946+0000 7f2d311bf700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 0x7f2d1006ccf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d28007fd0 tx=0x7f2d28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:39.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:39.949+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2d1c054980 con 0x7f2d18005330 2024-03-14T09:01:40.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.139+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d2c06d420 con 0x7f2d18005330 2024-03-14T09:01:40.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.139+0000 7f2d227fc700 1 -- 172.21.15.154:0/3255214277 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d1c057fa0 con 0x7f2d18005330 2024-03-14T09:01:40.139 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 msgr2=0x7f2d1006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 0x7f2d1006ccf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d28007fd0 tx=0x7f2d28009040 comp rx=0 tx=0).stop 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 msgr2=0x7f2d2c1dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00f850 tx=0x7f2d1c00fc10 comp rx=0 tx=0).stop 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 shutdown_connections 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c1dbd30 0x7f2d2c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2d1006a830 0x7f2d1006ccf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d18005330 0x7f2d2c1dcf90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 --2- 172.21.15.154:0/3255214277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d2c1dd4d0 0x7f2d2c1db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 >> 172.21.15.154:0/3255214277 conn(0x7f2d2c0c3880 msgr2=0x7f2d2c0c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 shutdown_connections 2024-03-14T09:01:40.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.141+0000 7f2d33c24700 1 -- 172.21.15.154:0/3255214277 wait complete. 2024-03-14T09:01:40.153 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:40.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.278+0000 7f240684f700 1 Processor -- start 2024-03-14T09:01:40.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.278+0000 7f240684f700 1 -- start start 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f240014fbd0 0x7f24000c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f24000c78c0 0x7f24000c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24000ac7b0 con 0x7f240014fbd0 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24000acd10 con 0x7f24000c78c0 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f240684f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2400150050 con 0x7f24000702f0 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f23ff7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f240014fbd0 0x7f24000c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f23ff7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f240014fbd0 0x7f24000c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59070/0 (socket says 172.21.15.154:59070) 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.279+0000 7f23ff7fe700 1 -- 172.21.15.154:0/1744377489 learned_addr learned my addr 172.21.15.154:0/1744377489 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23fffff700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 -- 172.21.15.154:0/1744377489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 msgr2=0x7f24000c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 -- 172.21.15.154:0/1744377489 >> v2:172.21.15.191:3300/0 conn(0x7f24000c78c0 msgr2=0x7f24000c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f2404dec700 1 --2- 172.21.15.154:0/1744377489 >> v2:172.21.15.191:3300/0 conn(0x7f24000c78c0 0x7f24000c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 --2- 172.21.15.154:0/1744377489 >> v2:172.21.15.191:3300/0 conn(0x7f24000c78c0 0x7f24000c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 -- 172.21.15.154:0/1744377489 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000ac9c0 con 0x7f240014fbd0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23ff7fe700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f24000c7380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2400073fd0 tx=0x7f23f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09229e6b5b4c89e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23feffd700 1 -- 172.21.15.154:0/1744377489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000b870 con 0x7f240014fbd0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23feffd700 1 -- 172.21.15.154:0/1744377489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f000b9d0 con 0x7f240014fbd0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.280+0000 7f23feffd700 1 -- 172.21.15.154:0/1744377489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000bd60 con 0x7f240014fbd0 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 -- 172.21.15.154:0/1744377489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 msgr2=0x7f24000c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f24000c7380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2400073fd0 tx=0x7f23f0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 -- 172.21.15.154:0/1744377489 shutdown_connections 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 --2- 172.21.15.154:0/1744377489 >> v2:172.21.15.191:3300/0 conn(0x7f24000c78c0 0x7f24000c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 --2- 172.21.15.154:0/1744377489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f24000c7380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 -- 172.21.15.154:0/1744377489 >> 172.21.15.154:0/1744377489 conn(0x7f24000a9bf0 msgr2=0x7f24000a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 -- 172.21.15.154:0/1744377489 shutdown_connections 2024-03-14T09:01:40.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.281+0000 7f240684f700 1 -- 172.21.15.154:0/1744377489 wait complete. 2024-03-14T09:01:40.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.282+0000 7f240684f700 1 Processor -- start 2024-03-14T09:01:40.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.282+0000 7f240684f700 1 -- start start 2024-03-14T09:01:40.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 0x7f2400142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240014a2f0 con 0x7f240014fbd0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240006c0b0 con 0x7f24000c78c0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240006c5e0 con 0x7f24000702f0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59076/0 (socket says 172.21.15.154:59076) 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 -- 172.21.15.154:0/679401076 learned_addr learned my addr 172.21.15.154:0/679401076 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23ff7fe700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 0x7f2400142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23fffff700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 -- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 msgr2=0x7f24000c1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 -- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 msgr2=0x7f2400142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 0x7f2400142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 -- 172.21.15.154:0/679401076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f0009550 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23ff7fe700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 0x7f2400142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f2404dec700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23f800cc30 tx=0x7f23f800a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800ab70 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f800acd0 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240014ba30 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.283+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8003c40 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.284+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240006c300 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.284+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240014a430 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.285+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f23f800f3f0 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.285+0000 7f23fcff9700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 0x7f23e006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.285+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23f8088ec0 con 0x7f240014fbd0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.285+0000 7f23fffff700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 0x7f23e006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.286+0000 7f23fffff700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 0x7f23e006ccf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23e800a950 tx=0x7f23e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.290+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f240014a430 con 0x7f240014fbd0 2024-03-14T09:01:40.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.482+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f240014a430 con 0x7f240014fbd0 2024-03-14T09:01:40.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.482+0000 7f23fcff9700 1 -- 172.21.15.154:0/679401076 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f240014a430 con 0x7f240014fbd0 2024-03-14T09:01:40.482 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 msgr2=0x7f23e006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 0x7f23e006ccf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f23e800a950 tx=0x7f23e8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 msgr2=0x7f2400142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23f800cc30 tx=0x7f23f800a1f0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 shutdown_connections 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23e006a830 0x7f23e006ccf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f240014fbd0 0x7f2400142ff0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24000702f0 0x7f24000c1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 --2- 172.21.15.154:0/679401076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24000c78c0 0x7f2400142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 >> 172.21.15.154:0/679401076 conn(0x7f24000a9bf0 msgr2=0x7f24000c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 shutdown_connections 2024-03-14T09:01:40.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.484+0000 7f240684f700 1 -- 172.21.15.154:0/679401076 wait complete. 2024-03-14T09:01:40.496 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:40.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 Processor -- start 2024-03-14T09:01:40.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 -- start start 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe41414fbd0 0x7fe41415df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140acd10 con 0x7fe4140c4380 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140ac9c0 con 0x7fe4140702f0 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.622+0000 7fe41c52e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4140ac7b0 con 0x7fe41414fbd0 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe419ac9700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4140c4380 0x7fe4140c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48432/0 (socket says 172.21.15.154:48432) 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe419ac9700 1 -- 172.21.15.154:0/4249733552 learned_addr learned my addr 172.21.15.154:0/4249733552 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe41aacb700 1 --2- 172.21.15.154:0/4249733552 >> v2:172.21.15.191:3300/0 conn(0x7fe41414fbd0 0x7fe41415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355702759 0 0) 0x7fe4140acd10 con 0x7fe4140c4380 2024-03-14T09:01:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4000035b0 con 0x7fe4140c4380 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3723189730 0 0) 0x7fe4000035b0 con 0x7fe4140c4380 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe41aacb700 1 -- 172.21.15.154:0/4249733552 >> v1:172.21.15.191:6789/0 conn(0x7fe4140702f0 legacy=0x7fe4140c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe41aacb700 1 -- 172.21.15.154:0/4249733552 >> v1:172.21.15.154:6789/0 conn(0x7fe4140c4380 legacy=0x7fe4140c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe41aacb700 1 -- 172.21.15.154:0/4249733552 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140707f0 con 0x7fe41414fbd0 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe41aacb700 1 --2- 172.21.15.154:0/4249733552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe41415df60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd743b43be213b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.623+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00e070 con 0x7fe41414fbd0 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c003a20 con 0x7fe41414fbd0 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe4192c8700 1 -- 172.21.15.154:0/4249733552 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410001670 con 0x7fe41414fbd0 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 -- 172.21.15.154:0/4249733552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 msgr2=0x7fe41415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 --2- 172.21.15.154:0/4249733552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe41415df60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 -- 172.21.15.154:0/4249733552 shutdown_connections 2024-03-14T09:01:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 --2- 172.21.15.154:0/4249733552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe41415df60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 -- 172.21.15.154:0/4249733552 >> 172.21.15.154:0/4249733552 conn(0x7fe4140a9bf0 msgr2=0x7fe4140a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 -- 172.21.15.154:0/4249733552 shutdown_connections 2024-03-14T09:01:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.624+0000 7fe41c52e700 1 -- 172.21.15.154:0/4249733552 wait complete. 2024-03-14T09:01:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.625+0000 7fe41c52e700 1 Processor -- start 2024-03-14T09:01:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.625+0000 7fe41c52e700 1 -- start start 2024-03-14T09:01:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 0x7fe4141eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe414144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414150860 con 0x7fe4140c4380 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414150ab0 con 0x7fe41414fbd0 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41c52e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141ec370 con 0x7fe4140702f0 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59080/0 (socket says 172.21.15.154:59080) 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 -- 172.21.15.154:0/1101156040 learned_addr learned my addr 172.21.15.154:0/1101156040 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41aacb700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe414144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41a2ca700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 0x7fe4141eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 -- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 msgr2=0x7fe4141eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 0x7fe4141eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 -- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 msgr2=0x7fe414144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe414144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 -- 172.21.15.154:0/1101156040 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c009a70 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41aacb700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe414144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe41a2ca700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 0x7fe4141eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.626+0000 7fe419ac9700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe4100011e0 tx=0x7fe41000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.627+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410007560 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.627+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100076c0 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.627+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe414144cc0 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.627+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410004a90 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.627+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414145160 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.628+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4140697d0 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.628+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe410004bf0 con 0x7fe4140c4380 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.628+0000 7fe40affd700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 0x7fe3f806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.629+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe410088180 con 0x7fe4140c4380 2024-03-14T09:01:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.629+0000 7fe41a2ca700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 0x7fe3f806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.629+0000 7fe41a2ca700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 0x7fe3f806ccf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4040017c0 tx=0x7fe404009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.633+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe410053630 con 0x7fe4140c4380 2024-03-14T09:01:40.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.825+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4141ec530 con 0x7fe4140c4380 2024-03-14T09:01:40.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.825+0000 7fe40affd700 1 -- 172.21.15.154:0/1101156040 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe410018020 con 0x7fe4140c4380 2024-03-14T09:01:40.825 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:40.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 msgr2=0x7fe3f806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 0x7fe3f806ccf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4040017c0 tx=0x7fe404009040 comp rx=0 tx=0).stop 2024-03-14T09:01:40.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 msgr2=0x7fe414144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe4100011e0 tx=0x7fe41000cb00 comp rx=0 tx=0).stop 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 shutdown_connections 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe41414fbd0 0x7fe414144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4140702f0 0x7fe4141eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe3f806a830 0x7fe3f806ccf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 --2- 172.21.15.154:0/1101156040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4140c4380 0x7fe414144240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 >> 172.21.15.154:0/1101156040 conn(0x7fe4140a9bf0 msgr2=0x7fe4140aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 shutdown_connections 2024-03-14T09:01:40.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.827+0000 7fe41c52e700 1 -- 172.21.15.154:0/1101156040 wait complete. 2024-03-14T09:01:40.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:40.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.960+0000 7f4ee3cf9700 1 Processor -- start 2024-03-14T09:01:40.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 -- start start 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4edc06c6e0 0x7f4edc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4edc070a10 0x7f4edc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0c5f10 con 0x7f4edc06d600 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0c7d50 con 0x7f4edc06c6e0 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.961+0000 7f4ee3cf9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0c8030 con 0x7f4edc070a10 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4edc06c6e0 0x7f4edc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4edc06c6e0 0x7f4edc06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59092/0 (socket says 172.21.15.154:59092) 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 -- 172.21.15.154:0/3032017154 learned_addr learned my addr 172.21.15.154:0/3032017154 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1294700 1 --2- 172.21.15.154:0/3032017154 >> v2:172.21.15.191:3300/0 conn(0x7f4edc070a10 0x7f4edc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 -- 172.21.15.154:0/3032017154 >> v2:172.21.15.191:3300/0 conn(0x7f4edc070a10 msgr2=0x7f4edc0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 --2- 172.21.15.154:0/3032017154 >> v2:172.21.15.191:3300/0 conn(0x7f4edc070a10 0x7f4edc0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 -- 172.21.15.154:0/3032017154 >> v1:172.21.15.191:6789/0 conn(0x7f4edc06d600 legacy=0x7f4edc0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:40.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 -- 172.21.15.154:0/3032017154 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0c8310 con 0x7f4edc06c6e0 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee1a95700 1 --2- 172.21.15.154:0/3032017154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06c6e0 0x7f4edc06d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0c4620 tx=0x7f4ed0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331b9ec7e83cf589 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee0a93700 1 -- 172.21.15.154:0/3032017154 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0004230 con 0x7f4edc06c6e0 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee0a93700 1 -- 172.21.15.154:0/3032017154 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed0004390 con 0x7f4edc06c6e0 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.962+0000 7f4ee0a93700 1 -- 172.21.15.154:0/3032017154 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00036a0 con 0x7f4edc06c6e0 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/3032017154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06c6e0 msgr2=0x7f4edc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/3032017154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06c6e0 0x7f4edc06d0c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0c4620 tx=0x7f4ed0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/3032017154 shutdown_connections 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/3032017154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06c6e0 0x7f4edc06d0c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/3032017154 >> v2:172.21.15.191:3300/0 conn(0x7f4edc070a10 0x7f4edc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/3032017154 >> 172.21.15.154:0/3032017154 conn(0x7f4edc0a9bf0 msgr2=0x7f4edc0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/3032017154 shutdown_connections 2024-03-14T09:01:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.963+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/3032017154 wait complete. 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.964+0000 7f4ee3cf9700 1 Processor -- start 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 -- start start 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4edc06c6e0 0x7f4edc1edee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4edc070a10 0x7f4edc1e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0a7350 con 0x7f4edc06d600 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1eda50 con 0x7f4edc06c6e0 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee3cf9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1ed440 con 0x7f4edc070a10 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.965+0000 7f4ee1294700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59108/0 (socket says 172.21.15.154:59108) 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 -- 172.21.15.154:0/2753550169 learned_addr learned my addr 172.21.15.154:0/2753550169 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1a95700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4edc06c6e0 0x7f4edc1edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee2296700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4edc070a10 0x7f4edc1e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 -- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4edc070a10 msgr2=0x7f4edc1e9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4edc070a10 0x7f4edc1e9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 -- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4edc06c6e0 msgr2=0x7f4edc1edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4edc06c6e0 0x7f4edc1edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 -- 172.21.15.154:0/2753550169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed0009550 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee1294700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800d800 tx=0x7f4ed800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80126b0 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8012810 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc0acc10 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8012ba0 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.966+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1dcce0 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.967+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc1edc00 con 0x7f4edc06d600 2024-03-14T09:01:40.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.968+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4ed8012d00 con 0x7f4edc06d600 2024-03-14T09:01:40.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.968+0000 7f4ece7fc700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 0x7f4ec006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:40.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.968+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ed8088d70 con 0x7f4edc06d600 2024-03-14T09:01:40.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.968+0000 7f4ee1a95700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 0x7f4ec006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:40.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.969+0000 7f4ee1a95700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 0x7f4ec006ccf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009130 tx=0x7f4ed0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:40.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:40.972+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4edc1edc00 con 0x7f4edc06d600 2024-03-14T09:01:41.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.167+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4edc1edc00 con 0x7f4edc06d600 2024-03-14T09:01:41.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.169+0000 7f4ece7fc700 1 -- 172.21.15.154:0/2753550169 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f4edc1edc00 con 0x7f4edc06d600 2024-03-14T09:01:41.168 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:41.168 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 msgr2=0x7f4ec006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 0x7f4ec006ccf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009130 tx=0x7f4ed0008030 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 msgr2=0x7f4edc1dc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800d800 tx=0x7f4ed800db10 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 shutdown_connections 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4edc070a10 0x7f4edc1e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4edc06c6e0 0x7f4edc1edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4ec006a830 0x7f4ec006ccf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 --2- 172.21.15.154:0/2753550169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4edc06d600 0x7f4edc1dc7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 >> 172.21.15.154:0/2753550169 conn(0x7f4edc0a9bf0 msgr2=0x7f4edc0c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:41.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 shutdown_connections 2024-03-14T09:01:41.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.171+0000 7f4ee3cf9700 1 -- 172.21.15.154:0/2753550169 wait complete. 2024-03-14T09:01:41.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:41.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:41.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.308+0000 7fdf3d9b1700 1 Processor -- start 2024-03-14T09:01:41.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.308+0000 7fdf3d9b1700 1 -- start start 2024-03-14T09:01:41.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf3d9b1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf3814e980 0x7fdf381552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf3d9b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf380c4a80 0x7fdf380c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf3d9b1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3814ed60 con 0x7fdf38158370 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf3d9b1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380c6b10 con 0x7fdf3814e980 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf3d9b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf38155820 con 0x7fdf380c4a80 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf3814e980 0x7fdf381552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf3814e980 0x7fdf381552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59124/0 (socket says 172.21.15.154:59124) 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 -- 172.21.15.154:0/1895485539 learned_addr learned my addr 172.21.15.154:0/1895485539 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf377fe700 1 --1- 172.21.15.154:0/1895485539 >> v1:172.21.15.154:6789/0 conn(0x7fdf38158370 0x7fdf38158780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48434/0 (socket says 172.21.15.154:48434) 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf367fc700 1 --2- 172.21.15.154:0/1895485539 >> v2:172.21.15.191:3300/0 conn(0x7fdf380c4a80 0x7fdf380c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 -- 172.21.15.154:0/1895485539 >> v2:172.21.15.191:3300/0 conn(0x7fdf380c4a80 msgr2=0x7fdf380c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 --2- 172.21.15.154:0/1895485539 >> v2:172.21.15.191:3300/0 conn(0x7fdf380c4a80 0x7fdf380c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 -- 172.21.15.154:0/1895485539 >> v1:172.21.15.154:6789/0 conn(0x7fdf38158370 legacy=0x7fdf38158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 -- 172.21.15.154:0/1895485539 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380c6cf0 con 0x7fdf3814e980 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf35ffb700 1 -- 172.21.15.154:0/1895485539 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351651743 0 0) 0x7fdf3814ed60 con 0x7fdf38158370 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.309+0000 7fdf36ffd700 1 --2- 172.21.15.154:0/1895485539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381552e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf380c1140 tx=0x7fdf2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b88eab4c5b03e48 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf35ffb700 1 -- 172.21.15.154:0/1895485539 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00b870 con 0x7fdf3814e980 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf35ffb700 1 -- 172.21.15.154:0/1895485539 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00b9d0 con 0x7fdf3814e980 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf35ffb700 1 -- 172.21.15.154:0/1895485539 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00bd60 con 0x7fdf3814e980 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/1895485539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 msgr2=0x7fdf381552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/1895485539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381552e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf380c1140 tx=0x7fdf2c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/1895485539 shutdown_connections 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/1895485539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381552e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/1895485539 >> v2:172.21.15.191:3300/0 conn(0x7fdf380c4a80 0x7fdf380c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/1895485539 >> 172.21.15.154:0/1895485539 conn(0x7fdf380abcf0 msgr2=0x7fdf380ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/1895485539 shutdown_connections 2024-03-14T09:01:41.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.310+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/1895485539 wait complete. 2024-03-14T09:01:41.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.311+0000 7fdf3d9b1700 1 Processor -- start 2024-03-14T09:01:41.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 -- start start 2024-03-14T09:01:41.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf380c4a80 0x7fdf381542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf38158370 0x7fdf381481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38151890 con 0x7fdf3814e980 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38153e60 con 0x7fdf38158370 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf3d9b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38147fc0 con 0x7fdf380c4a80 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf367fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf367fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59138/0 (socket says 172.21.15.154:59138) 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf367fc700 1 -- 172.21.15.154:0/4082174390 learned_addr learned my addr 172.21.15.154:0/4082174390 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf36ffd700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf380c4a80 0x7fdf381542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.312+0000 7fdf377fe700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf38158370 0x7fdf381481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 -- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf380c4a80 msgr2=0x7fdf381542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf380c4a80 0x7fdf381542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 -- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf38158370 msgr2=0x7fdf381481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf38158370 0x7fdf381481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 -- 172.21.15.154:0/4082174390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c009550 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf367fc700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800d800 tx=0x7fdf2800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280125f0 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28012750 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf381edae0 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28012b00 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.313+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf38153850 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.314+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38153fd0 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.315+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdf28012c60 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.315+0000 7fdf1ffff700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 0x7fdf2406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.315+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf28088cb0 con 0x7fdf3814e980 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.315+0000 7fdf36ffd700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 0x7fdf2406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.315+0000 7fdf36ffd700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 0x7fdf2406ccf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009c10 tx=0x7fdf2c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.319+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdf28054160 con 0x7fdf3814e980 2024-03-14T09:01:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.512+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf380697d0 con 0x7fdf3814e980 2024-03-14T09:01:41.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.512+0000 7fdf1ffff700 1 -- 172.21.15.154:0/4082174390 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf28057780 con 0x7fdf3814e980 2024-03-14T09:01:41.512 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.514+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 msgr2=0x7fdf2406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.514+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 0x7fdf2406ccf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009c10 tx=0x7fdf2c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.514+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 msgr2=0x7fdf381ed5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.514+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800d800 tx=0x7fdf2800db10 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.514+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 shutdown_connections 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf38158370 0x7fdf381481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf380c4a80 0x7fdf381542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf2406a830 0x7fdf2406ccf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 --2- 172.21.15.154:0/4082174390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf3814e980 0x7fdf381ed5a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 >> 172.21.15.154:0/4082174390 conn(0x7fdf380abcf0 msgr2=0x7fdf3814cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 shutdown_connections 2024-03-14T09:01:41.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.515+0000 7fdf3d9b1700 1 -- 172.21.15.154:0/4082174390 wait complete. 2024-03-14T09:01:41.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:41.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.651+0000 7f326693f700 1 Processor -- start 2024-03-14T09:01:41.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f326693f700 1 -- start start 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f326693f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f32600c3250 0x7f326013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f326693f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326013ade0 con 0x7f32600a8540 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f326693f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326006fb30 con 0x7f32600c1780 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f326693f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326006ec30 con 0x7f32600c3250 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f325ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f32600a8540 0x7f3260154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48440/0 (socket says 172.21.15.154:48440) 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.652+0000 7f325ffff700 1 -- 172.21.15.154:0/580170561 learned_addr learned my addr 172.21.15.154:0/580170561 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:41.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382547600 0 0) 0x7f3254001110 con 0x7f32600a8540 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32480035b0 con 0x7f32600a8540 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325f7fe700 1 --2- 172.21.15.154:0/580170561 >> v2:172.21.15.191:3300/0 conn(0x7f32600c3250 0x7f326013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920766874 0 0) 0x7f32480035b0 con 0x7f32600a8540 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 >> v2:172.21.15.191:3300/0 conn(0x7f32600c3250 msgr2=0x7f326013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 --2- 172.21.15.154:0/580170561 >> v2:172.21.15.191:3300/0 conn(0x7f32600c3250 0x7f326013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 >> v1:172.21.15.191:6789/0 conn(0x7f32600c1780 legacy=0x7f32600c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326006ef10 con 0x7f32600a8540 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f3254001800 con 0x7f32600a8540 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 >> v1:172.21.15.154:6789/0 conn(0x7f32600a8540 legacy=0x7f3260154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 0x7f3248003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248004850 0x7f3248004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3248005330 0x7f324800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326006ec30 con 0x7f3248003a00 2024-03-14T09:01:41.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324800c200 con 0x7f3248005330 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324800c3c0 con 0x7f3248004850 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 0x7f3248003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f3264edc700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3248005330 0x7f324800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325f7fe700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248004850 0x7f3248004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 -- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248004850 msgr2=0x7f3248004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248004850 0x7f3248004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 -- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3248005330 msgr2=0x7f324800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3248005330 0x7f324800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325ffff700 1 -- 172.21.15.154:0/580170561 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480035b0 con 0x7f3248003a00 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f3264edc700 1 -- 172.21.15.154:0/580170561 reap_dead start 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.653+0000 7f325f7fe700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248004850 0x7f3248004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f325ffff700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 0x7f3248003e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f325400c210 tx=0x7f3248005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79326553d26d0ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254007d80 con 0x7f3248003a00 2024-03-14T09:01:41.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325400cd70 con 0x7f3248003a00 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f325effd700 1 -- 172.21.15.154:0/580170561 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254005ba0 con 0x7f3248003a00 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 -- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 msgr2=0x7f3248003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 0x7f3248003e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f325400c210 tx=0x7f3248005880 comp rx=0 tx=0).stop 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 -- 172.21.15.154:0/580170561 shutdown_connections 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 --2- 172.21.15.154:0/580170561 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3248003a00 0x7f3248003e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 -- 172.21.15.154:0/580170561 >> 172.21.15.154:0/580170561 conn(0x7f32600c3880 msgr2=0x7f32600c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 -- 172.21.15.154:0/580170561 shutdown_connections 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.654+0000 7f326693f700 1 -- 172.21.15.154:0/580170561 wait complete. 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.655+0000 7f326693f700 1 Processor -- start 2024-03-14T09:01:41.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 -- start start 2024-03-14T09:01:41.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 0x7f32600abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 0x7f32601dac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600ab3e0 con 0x7f32601db1b0 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600aa7d0 con 0x7f32600ac200 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f326693f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600aaab0 con 0x7f3248003a00 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f3264edc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f3264edc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59142/0 (socket says 172.21.15.154:59142) 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f3264edc700 1 -- 172.21.15.154:0/2493803047 learned_addr learned my addr 172.21.15.154:0/2493803047 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f325ffff700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 0x7f32600abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.656+0000 7f325f7fe700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 0x7f32601dac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 -- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 msgr2=0x7f32600abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 0x7f32600abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 -- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 msgr2=0x7f32601dac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 0x7f32601dac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 -- 172.21.15.154:0/2493803047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600aad90 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f325ffff700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 0x7f32600abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f325f7fe700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 0x7f32601dac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f3264edc700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f326006d8e0 tx=0x7f325800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258003c10 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258003d70 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800dbd0 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32600ab070 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.657+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326006b090 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.658+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32600697d0 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.658+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f325800dd30 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.659+0000 7f325cff9700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 0x7f324c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.659+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3258088ff0 con 0x7f32601db1b0 2024-03-14T09:01:41.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.659+0000 7f325ffff700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 0x7f324c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.660+0000 7f325ffff700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 0x7f324c06ccf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3254000c00 tx=0x7f325400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.663+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f32580544a0 con 0x7f32601db1b0 2024-03-14T09:01:41.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.857+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f326006d420 con 0x7f32601db1b0 2024-03-14T09:01:41.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.857+0000 7f325cff9700 1 -- 172.21.15.154:0/2493803047 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3258057ac0 con 0x7f32601db1b0 2024-03-14T09:01:41.857 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 msgr2=0x7f324c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 0x7f324c06ccf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3254000c00 tx=0x7f325400a000 comp rx=0 tx=0).stop 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 msgr2=0x7f3260135090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f326006d8e0 tx=0x7f325800d430 comp rx=0 tx=0).stop 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 shutdown_connections 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f324c06a830 0x7f324c06ccf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32601db1b0 0x7f3260135090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3248003a00 0x7f32600abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 --2- 172.21.15.154:0/2493803047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32600ac200 0x7f32601dac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 >> 172.21.15.154:0/2493803047 conn(0x7f32600c3880 msgr2=0x7f3260138bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:41.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 shutdown_connections 2024-03-14T09:01:41.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.860+0000 7f326693f700 1 -- 172.21.15.154:0/2493803047 wait complete. 2024-03-14T09:01:41.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:41.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.996+0000 7f2f9a967700 1 Processor -- start 2024-03-14T09:01:41.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.996+0000 7f2f9a967700 1 -- start start 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f9a967700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 0x7f2f94156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f9a967700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f9415d270 0x7f2f9415f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f9a967700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f940ac8e0 con 0x7f2f9414eaf0 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f9a967700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9406b310 con 0x7f2f9415d270 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f9a967700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9406de30 con 0x7f2f94155df0 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f98f04700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f9415d270 0x7f2f9415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f93fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2f9414eaf0 0x7f2f9406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48442/0 (socket says 172.21.15.154:48442) 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f93fff700 1 -- 172.21.15.154:0/2141634851 learned_addr learned my addr 172.21.15.154:0/2141634851 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f98f04700 1 -- 172.21.15.154:0/2141634851 >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 msgr2=0x7f2f94156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:41.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f937fe700 1 --2- 172.21.15.154:0/2141634851 >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 0x7f2f94156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f92ffd700 1 -- 172.21.15.154:0/2141634851 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838752039 0 0) 0x7f2f940ac8e0 con 0x7f2f9414eaf0 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f98f04700 1 --2- 172.21.15.154:0/2141634851 >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 0x7f2f94156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f98f04700 1 -- 172.21.15.154:0/2141634851 >> v1:172.21.15.154:6789/0 conn(0x7f2f9414eaf0 legacy=0x7f2f9406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f98f04700 1 -- 172.21.15.154:0/2141634851 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940acb30 con 0x7f2f9415d270 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.997+0000 7f2f937fe700 1 --2- 172.21.15.154:0/2141634851 >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 0x7f2f94156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f98f04700 1 --2- 172.21.15.154:0/2141634851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9415f660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f9414f1c0 tx=0x7f2f8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2020adfe827c2b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f92ffd700 1 -- 172.21.15.154:0/2141634851 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00e070 con 0x7f2f9415d270 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f92ffd700 1 -- 172.21.15.154:0/2141634851 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00bbd0 con 0x7f2f9415d270 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f92ffd700 1 -- 172.21.15.154:0/2141634851 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c012430 con 0x7f2f9415d270 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 -- 172.21.15.154:0/2141634851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 msgr2=0x7f2f9415f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2141634851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9415f660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f9414f1c0 tx=0x7f2f8c00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 -- 172.21.15.154:0/2141634851 shutdown_connections 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2141634851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9415f660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2141634851 >> v2:172.21.15.191:3300/0 conn(0x7f2f94155df0 0x7f2f94156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:41.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 -- 172.21.15.154:0/2141634851 >> 172.21.15.154:0/2141634851 conn(0x7f2f940c3880 msgr2=0x7f2f940c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 -- 172.21.15.154:0/2141634851 shutdown_connections 2024-03-14T09:01:42.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.998+0000 7f2f9a967700 1 -- 172.21.15.154:0/2141634851 wait complete. 2024-03-14T09:01:42.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:41.999+0000 7f2f9a967700 1 Processor -- start 2024-03-14T09:01:42.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 -- start start 2024-03-14T09:01:42.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9414eaf0 0x7f2f9413dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 0x7f2f9413c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9413e870 con 0x7f2f9415d270 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f94141b40 con 0x7f2f94155df0 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f9a967700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f941413d0 con 0x7f2f9414eaf0 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f98f04700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f98f04700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59158/0 (socket says 172.21.15.154:59158) 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.000+0000 7f2f98f04700 1 -- 172.21.15.154:0/2796686078 learned_addr learned my addr 172.21.15.154:0/2796686078 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f937fe700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 0x7f2f9413c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f93fff700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9414eaf0 0x7f2f9413dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 -- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9414eaf0 msgr2=0x7f2f9413dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9414eaf0 0x7f2f9413dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 -- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 msgr2=0x7f2f9413c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 0x7f2f9413c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 -- 172.21.15.154:0/2796686078 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c009a70 con 0x7f2f9415d270 2024-03-14T09:01:42.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f937fe700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 0x7f2f9413c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f98f04700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c014040 tx=0x7f2f8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c012430 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c020070 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f9413e5d0 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c004280 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.001+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f94141620 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.002+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9413ef80 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.002+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2f8c0043e0 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.003+0000 7f2f90ff9700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.003+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f8c022070 con 0x7f2f9415d270 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.003+0000 7f2f93fff700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.004+0000 7f2f93fff700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.007+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2f9413ef80 con 0x7f2f9415d270 2024-03-14T09:01:42.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.198+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f9413ef80 con 0x7f2f9415d270 2024-03-14T09:01:42.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.198+0000 7f2f90ff9700 1 -- 172.21.15.154:0/2796686078 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f9413ef80 con 0x7f2f9415d270 2024-03-14T09:01:42.198 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:42.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 msgr2=0x7f2f8006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2024-03-14T09:01:42.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 msgr2=0x7f2f9413ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c014040 tx=0x7f2f8c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 shutdown_connections 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f9415d270 0x7f2f9413ca00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f9414eaf0 0x7f2f9413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 --2- 172.21.15.154:0/2796686078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f94155df0 0x7f2f9413c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 >> 172.21.15.154:0/2796686078 conn(0x7f2f940c3880 msgr2=0x7f2f9406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 shutdown_connections 2024-03-14T09:01:42.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.200+0000 7f2f9a967700 1 -- 172.21.15.154:0/2796686078 wait complete. 2024-03-14T09:01:42.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:42.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 Processor -- start 2024-03-14T09:01:42.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 -- start start 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200702f0 0x7fd8200c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82014fbd0 0x7fd8200c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8200acd10 con 0x7fd8200c78c0 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200ac7b0 con 0x7fd82014fbd0 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd8251cf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd820150050 con 0x7fd8200702f0 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.334+0000 7fd81e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82014fbd0 0x7fd8200c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82014fbd0 0x7fd8200c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59160/0 (socket says 172.21.15.154:59160) 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 -- 172.21.15.154:0/2678568356 learned_addr learned my addr 172.21.15.154:0/2678568356 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81ed9d700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200702f0 0x7fd8200c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 -- 172.21.15.154:0/2678568356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200702f0 msgr2=0x7fd8200c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200702f0 0x7fd8200c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 -- 172.21.15.154:0/2678568356 >> v1:172.21.15.191:6789/0 conn(0x7fd8200c78c0 legacy=0x7fd8200c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 -- 172.21.15.154:0/2678568356 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200ac9c0 con 0x7fd82014fbd0 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81e59c700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd8200c7380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd994b273fc8c62f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81dd9b700 1 -- 172.21.15.154:0/2678568356 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011830 con 0x7fd82014fbd0 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81dd9b700 1 -- 172.21.15.154:0/2678568356 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810011990 con 0x7fd82014fbd0 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.335+0000 7fd81dd9b700 1 -- 172.21.15.154:0/2678568356 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011d20 con 0x7fd82014fbd0 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 -- 172.21.15.154:0/2678568356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 msgr2=0x7fd8200c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd8200c7380 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098f0 tx=0x7fd810009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 -- 172.21.15.154:0/2678568356 shutdown_connections 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200702f0 0x7fd8200c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 --2- 172.21.15.154:0/2678568356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd8200c7380 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 -- 172.21.15.154:0/2678568356 >> 172.21.15.154:0/2678568356 conn(0x7fd8200a9bf0 msgr2=0x7fd8200a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 -- 172.21.15.154:0/2678568356 shutdown_connections 2024-03-14T09:01:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.336+0000 7fd8251cf700 1 -- 172.21.15.154:0/2678568356 wait complete. 2024-03-14T09:01:42.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.337+0000 7fd8251cf700 1 Processor -- start 2024-03-14T09:01:42.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.337+0000 7fd8251cf700 1 -- start start 2024-03-14T09:01:42.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 0x7fd8201485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200c78c0 0x7fd820153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820149080 con 0x7fd82014fbd0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201565b0 con 0x7fd8200702f0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd8251cf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201558f0 con 0x7fd8200c78c0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 0x7fd8201485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59168/0 (socket says 172.21.15.154:59168) 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 -- 172.21.15.154:0/1690143685 learned_addr learned my addr 172.21.15.154:0/1690143685 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81ed9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 0x7fd8201485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:59906/0 (socket says 172.21.15.154:59906) 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81e59c700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200c78c0 0x7fd820153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 -- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200c78c0 msgr2=0x7fd820153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200c78c0 0x7fd820153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 -- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 msgr2=0x7fd8201485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 0x7fd8201485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.338+0000 7fd81f59e700 1 -- 172.21.15.154:0/1690143685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810009550 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd81f59e700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cb50 tx=0x7fd81800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818003c00 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818003d60 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd820152cb0 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81800aa20 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.339+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd820152f00 con 0x7fd82014fbd0 2024-03-14T09:01:42.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.340+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82014dd80 con 0x7fd82014fbd0 2024-03-14T09:01:42.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.340+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd81800ab80 con 0x7fd82014fbd0 2024-03-14T09:01:42.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.341+0000 7fd8077fe700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 0x7fd80c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.341+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd818088d70 con 0x7fd82014fbd0 2024-03-14T09:01:42.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.341+0000 7fd81ed9d700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 0x7fd80c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.341+0000 7fd81ed9d700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 0x7fd80c06ccf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd814009b40 tx=0x7fd814009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.345+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd818054180 con 0x7fd82014fbd0 2024-03-14T09:01:42.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.539+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd820074110 con 0x7fd82014fbd0 2024-03-14T09:01:42.540 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.540+0000 7fd8077fe700 1 -- 172.21.15.154:0/1690143685 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fd8180577a0 con 0x7fd82014fbd0 2024-03-14T09:01:42.540 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:42.540 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:42.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 msgr2=0x7fd80c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 0x7fd80c06ccf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd814009b40 tx=0x7fd814009210 comp rx=0 tx=0).stop 2024-03-14T09:01:42.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 msgr2=0x7fd820154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd81800cb50 tx=0x7fd81800ce60 comp rx=0 tx=0).stop 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 shutdown_connections 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd80c06a830 0x7fd80c06ccf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82014fbd0 0x7fd820154010 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8200702f0 0x7fd8201485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 --2- 172.21.15.154:0/1690143685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8200c78c0 0x7fd820153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.542+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 >> 172.21.15.154:0/1690143685 conn(0x7fd8200a9bf0 msgr2=0x7fd820149f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.543+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 shutdown_connections 2024-03-14T09:01:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.543+0000 7fd8251cf700 1 -- 172.21.15.154:0/1690143685 wait complete. 2024-03-14T09:01:42.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:42.555 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.681+0000 7f2f12b40700 1 Processor -- start 2024-03-14T09:01:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.681+0000 7f2f12b40700 1 -- start start 2024-03-14T09:01:42.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 0x7f2f0c0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 0x7f2f0c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c0ac7b0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c0acd10 con 0x7f2f0c0702f0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f12b40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c150050 con 0x7f2f0c0c78c0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59174/0 (socket says 172.21.15.154:59174) 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 -- 172.21.15.154:0/2644056105 learned_addr learned my addr 172.21.15.154:0/2644056105 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f108dc700 1 --2- 172.21.15.154:0/2644056105 >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 0x7f2f0c0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f110dd700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 0x7f2f0c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 -- 172.21.15.154:0/2644056105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 msgr2=0x7f2f0c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 0x7f2f0c0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 -- 172.21.15.154:0/2644056105 >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 msgr2=0x7f2f0c0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:42.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 --2- 172.21.15.154:0/2644056105 >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 0x7f2f0c0c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 -- 172.21.15.154:0/2644056105 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c0ac9c0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f108dc700 1 --2- 172.21.15.154:0/2644056105 >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 0x7f2f0c0c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f110dd700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 0x7f2f0c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.682+0000 7f2f0bfff700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0c8c00 tx=0x7f2ef8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3ba776c3f0ae10 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f0b7fe700 1 -- 172.21.15.154:0/2644056105 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800b870 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f0b7fe700 1 -- 172.21.15.154:0/2644056105 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef800b9d0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f0b7fe700 1 -- 172.21.15.154:0/2644056105 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800bd60 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 -- 172.21.15.154:0/2644056105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 msgr2=0x7f2f0c0c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0c8c00 tx=0x7f2ef8009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 -- 172.21.15.154:0/2644056105 shutdown_connections 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0c78c0 0x7f2f0c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 --2- 172.21.15.154:0/2644056105 >> v2:172.21.15.191:3300/0 conn(0x7f2f0c0702f0 0x7f2f0c0c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 --2- 172.21.15.154:0/2644056105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c0c7380 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 -- 172.21.15.154:0/2644056105 >> 172.21.15.154:0/2644056105 conn(0x7f2f0c0a9bf0 msgr2=0x7f2f0c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 -- 172.21.15.154:0/2644056105 shutdown_connections 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.683+0000 7f2f12b40700 1 -- 172.21.15.154:0/2644056105 wait complete. 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.685+0000 7f2f12b40700 1 Processor -- start 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.685+0000 7f2f12b40700 1 -- start start 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0702f0 0x7f2f0c1e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f0c0c78c0 0x7f2f0c152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c153200 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c156af0 con 0x7f2f0c0c78c0 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c1e8670 con 0x7f2f0c0702f0 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59180/0 (socket says 172.21.15.154:59180) 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 -- 172.21.15.154:0/3932112032 learned_addr learned my addr 172.21.15.154:0/3932112032 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f108dc700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0702f0 0x7f2f0c1e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f0bfff700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f0c0c78c0 0x7f2f0c152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 -- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0702f0 msgr2=0x7f2f0c1e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0702f0 0x7f2f0c1e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 -- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f0c0c78c0 msgr2=0x7f2f0c152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f0c0c78c0 0x7f2f0c152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 -- 172.21.15.154:0/3932112032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef8009550 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f110dd700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400cef0 tx=0x7f2f0400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400ad00 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0400ae60 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04015680 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.686+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c1e8950 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.687+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c1e8370 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.687+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0c150ca0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.688+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2f0400d430 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.688+0000 7f2f097fa700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 0x7f2ef006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:42.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.688+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f04089130 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.689+0000 7f2f108dc700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 0x7f2ef006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:42.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.689+0000 7f2f108dc700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 0x7f2ef006ccf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:42.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.693+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2f0c150ca0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.887+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f0c150ca0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.887+0000 7f2f097fa700 1 -- 172.21.15.154:0/3932112032 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f0c150ca0 con 0x7f2f0c14fbd0 2024-03-14T09:01:42.890 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.889+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 msgr2=0x7f2ef006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.889+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 0x7f2ef006ccf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.889+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 msgr2=0x7f2f0c152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.889+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400cef0 tx=0x7f2f0400cf20 comp rx=0 tx=0).stop 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 shutdown_connections 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2ef006a830 0x7f2ef006ccf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c14fbd0 0x7f2f0c152cc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f0c0702f0 0x7f2f0c1e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 --2- 172.21.15.154:0/3932112032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f0c0c78c0 0x7f2f0c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:42.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 >> 172.21.15.154:0/3932112032 conn(0x7f2f0c0a9bf0 msgr2=0x7f2f0c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:42.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 shutdown_connections 2024-03-14T09:01:42.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:42.890+0000 7f2f12b40700 1 -- 172.21.15.154:0/3932112032 wait complete. 2024-03-14T09:01:42.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.026+0000 7f8173a3f700 1 Processor -- start 2024-03-14T09:01:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.026+0000 7f8173a3f700 1 -- start start 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8173a3f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f816c155df0 0x7f816c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8173a3f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c0ac8e0 con 0x7f816c15d270 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8173a3f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c06de30 con 0x7f816c14eaf0 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8173a3f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c06b310 con 0x7f816c155df0 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8171fdc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f816c15d270 0x7f816c15f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48454/0 (socket says 172.21.15.154:48454) 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8171fdc700 1 -- 172.21.15.154:0/3761689894 learned_addr learned my addr 172.21.15.154:0/3761689894 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391840081 0 0) 0x7f816c0ac8e0 con 0x7f816c15d270 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81580035b0 con 0x7f816c15d270 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.027+0000 7f8170fda700 1 --2- 172.21.15.154:0/3761689894 >> v2:172.21.15.191:3300/0 conn(0x7f816c155df0 0x7f816c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935386183 0 0) 0x7f81580035b0 con 0x7f816c15d270 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 >> v2:172.21.15.191:3300/0 conn(0x7f816c155df0 msgr2=0x7f816c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 --2- 172.21.15.154:0/3761689894 >> v2:172.21.15.191:3300/0 conn(0x7f816c155df0 0x7f816c156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 >> v1:172.21.15.191:6789/0 conn(0x7f816c14eaf0 legacy=0x7f816c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0acb30 con 0x7f816c15d270 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f81640019f0 con 0x7f816c15d270 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 >> v1:172.21.15.154:6789/0 conn(0x7f816c15d270 legacy=0x7f816c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158003a00 0x7f8158003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8158004850 0x7f8158004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 0x7f815800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c06b310 con 0x7f8158005330 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815800c200 con 0x7f8158004850 2024-03-14T09:01:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815800c3c0 con 0x7f8158003a00 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 0x7f815800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8170fda700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8158004850 0x7f8158004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f81717db700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158003a00 0x7f8158003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 -- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158003a00 msgr2=0x7f8158003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158003a00 0x7f8158003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 -- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8158004850 msgr2=0x7f8158004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8158004850 0x7f8158004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 -- 172.21.15.154:0/3761689894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580035b0 con 0x7f8158005330 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 -- 172.21.15.154:0/3761689894 reap_dead start 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8171fdc700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 0x7f815800ba10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f816400a800 tx=0x7f8158005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9c515556d0b174 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400a2f0 con 0x7f8158005330 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164012620 con 0x7f8158005330 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.028+0000 7f8163fff700 1 -- 172.21.15.154:0/3761689894 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640129d0 con 0x7f8158005330 2024-03-14T09:01:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 -- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 msgr2=0x7f815800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 0x7f815800ba10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f816400a800 tx=0x7f8158005880 comp rx=0 tx=0).stop 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 -- 172.21.15.154:0/3761689894 shutdown_connections 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 --2- 172.21.15.154:0/3761689894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8158005330 0x7f815800ba10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 -- 172.21.15.154:0/3761689894 >> 172.21.15.154:0/3761689894 conn(0x7f816c0c3880 msgr2=0x7f816c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 -- 172.21.15.154:0/3761689894 shutdown_connections 2024-03-14T09:01:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.029+0000 7f8173a3f700 1 -- 172.21.15.154:0/3761689894 wait complete. 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.030+0000 7f8173a3f700 1 Processor -- start 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.030+0000 7f8173a3f700 1 -- start start 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158005330 0x7f816c0c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f816c149f50 0x7f816c1ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c148d90 con 0x7f816c1471b0 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c149070 con 0x7f816c149f50 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8173a3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c145de0 con 0x7f8158005330 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59190/0 (socket says 172.21.15.154:59190) 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 -- 172.21.15.154:0/138369100 learned_addr learned my addr 172.21.15.154:0/138369100 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f81717db700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158005330 0x7f816c0c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8171fdc700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f816c149f50 0x7f816c1ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 -- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158005330 msgr2=0x7f816c0c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158005330 0x7f816c0c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 -- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f816c149f50 msgr2=0x7f816c1ed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f816c149f50 0x7f816c1ed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 -- 172.21.15.154:0/138369100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c1460c0 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.031+0000 7f8170fda700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81680119d0 tx=0x7f8168011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.032+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168011e70 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.032+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816800f940 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.032+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c1463a0 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.032+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816800fcd0 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.032+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c1484a0 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.033+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c0697d0 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.033+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8168014440 con 0x7f816c1471b0 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.034+0000 7f8161ffb700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 0x7f815006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.034+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8168089490 con 0x7f816c1471b0 2024-03-14T09:01:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.034+0000 7f81717db700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 0x7f815006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.034+0000 7f81717db700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 0x7f815006ccf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.038+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8168054830 con 0x7f816c1471b0 2024-03-14T09:01:43.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.233+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f816c1491b0 con 0x7f816c1471b0 2024-03-14T09:01:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.233+0000 7f8161ffb700 1 -- 172.21.15.154:0/138369100 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8168057e50 con 0x7f816c1471b0 2024-03-14T09:01:43.233 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 msgr2=0x7f815006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 0x7f815006ccf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f815c000c00 tx=0x7f815c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 msgr2=0x7f816c149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81680119d0 tx=0x7f8168011ce0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 shutdown_connections 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f816c149f50 0x7f816c1ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8158005330 0x7f816c0c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f815006a830 0x7f815006ccf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 --2- 172.21.15.154:0/138369100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f816c1471b0 0x7f816c149500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 >> 172.21.15.154:0/138369100 conn(0x7f816c0c3880 msgr2=0x7f816c1561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 shutdown_connections 2024-03-14T09:01:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.235+0000 7f8173a3f700 1 -- 172.21.15.154:0/138369100 wait complete. 2024-03-14T09:01:43.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:43.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.372+0000 7fcce0c42700 1 Processor -- start 2024-03-14T09:01:43.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.372+0000 7fcce0c42700 1 -- start start 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fcce0c42700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 0x7fccdc0c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fcce0c42700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fccdc1564e0 0x7fccdc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fcce0c42700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccdc156050 con 0x7fccdc156960 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fcce0c42700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccdc0a86f0 con 0x7fccdc1564e0 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fcce0c42700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccdc0c34b0 con 0x7fccdc0a9650 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fccdc1564e0 0x7fccdc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccdad9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fccdc156960 0x7fccdc152690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48456/0 (socket says 172.21.15.154:48456) 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccdad9d700 1 -- 172.21.15.154:0/1914255100 learned_addr learned my addr 172.21.15.154:0/1914255100 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd9d9b700 1 -- 172.21.15.154:0/1914255100 >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 msgr2=0x7fccdc0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccda59c700 1 --2- 172.21.15.154:0/1914255100 >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 0x7fccdc0c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd9d9b700 1 --2- 172.21.15.154:0/1914255100 >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 0x7fccdc0c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd959a700 1 -- 172.21.15.154:0/1914255100 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506849120 0 0) 0x7fccdc156050 con 0x7fccdc156960 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd9d9b700 1 -- 172.21.15.154:0/1914255100 >> v1:172.21.15.154:6789/0 conn(0x7fccdc156960 legacy=0x7fccdc152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.373+0000 7fccd9d9b700 1 -- 172.21.15.154:0/1914255100 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0a8990 con 0x7fccdc1564e0 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fccda59c700 1 --2- 172.21.15.154:0/1914255100 >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 0x7fccdc0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fccd9d9b700 1 --2- 172.21.15.154:0/1914255100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc06a770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fccdc070da0 tx=0x7fcccc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e0cabf28bc143d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fccd959a700 1 -- 172.21.15.154:0/1914255100 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011830 con 0x7fccdc1564e0 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fccd959a700 1 -- 172.21.15.154:0/1914255100 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc011990 con 0x7fccdc1564e0 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fccd959a700 1 -- 172.21.15.154:0/1914255100 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011d20 con 0x7fccdc1564e0 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 -- 172.21.15.154:0/1914255100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 msgr2=0x7fccdc06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 --2- 172.21.15.154:0/1914255100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc06a770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fccdc070da0 tx=0x7fcccc009b20 comp rx=0 tx=0).stop 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 -- 172.21.15.154:0/1914255100 shutdown_connections 2024-03-14T09:01:43.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 --2- 172.21.15.154:0/1914255100 >> v2:172.21.15.191:3300/0 conn(0x7fccdc0a9650 0x7fccdc0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 --2- 172.21.15.154:0/1914255100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc06a770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.374+0000 7fcce0c42700 1 -- 172.21.15.154:0/1914255100 >> 172.21.15.154:0/1914255100 conn(0x7fccdc0c3880 msgr2=0x7fccdc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:43.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.375+0000 7fcce0c42700 1 -- 172.21.15.154:0/1914255100 shutdown_connections 2024-03-14T09:01:43.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.375+0000 7fcce0c42700 1 -- 172.21.15.154:0/1914255100 wait complete. 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.376+0000 7fcce0c42700 1 Processor -- start 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.376+0000 7fcce0c42700 1 -- start start 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.376+0000 7fcce0c42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccdc0a9650 0x7fccdc0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.376+0000 7fcce0c42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.376+0000 7fcce0c42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccdc156960 0x7fccdc1412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcce0c42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc144430 con 0x7fccdc1564e0 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcce0c42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc144710 con 0x7fccdc0a9650 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcce0c42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1449f0 con 0x7fccdc156960 2024-03-14T09:01:43.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59210/0 (socket says 172.21.15.154:59210) 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 -- 172.21.15.154:0/2860812403 learned_addr learned my addr 172.21.15.154:0/2860812403 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 -- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccdc156960 msgr2=0x7fccdc1412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccdad9d700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccdc156960 0x7fccdc1412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccda59c700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccdc0a9650 0x7fccdc0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccdc156960 0x7fccdc1412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 -- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccdc0a9650 msgr2=0x7fccdc0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccdc0a9650 0x7fccdc0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 -- 172.21.15.154:0/2860812403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0094a0 con 0x7fccdc1564e0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fccd9d9b700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcccc014040 tx=0x7fcccc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0038a0 con 0x7fccdc1564e0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc01e070 con 0x7fccdc1564e0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc144cd0 con 0x7fccdc1564e0 2024-03-14T09:01:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003c60 con 0x7fccdc1564e0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.377+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc140ec0 con 0x7fccdc1564e0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.378+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc0c4500 con 0x7fccdc1564e0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.379+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcccc01a420 con 0x7fccdc1564e0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.379+0000 7fcccaffd700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 0x7fccbc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.379+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcccc020070 con 0x7fccdc1564e0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.379+0000 7fccda59c700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 0x7fccbc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.380+0000 7fccda59c700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 0x7fccbc06ccf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fccd0006fd0 tx=0x7fccd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.384+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcccc0556b0 con 0x7fccdc1564e0 2024-03-14T09:01:43.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.578+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fccdc0697d0 con 0x7fccdc1564e0 2024-03-14T09:01:43.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.579+0000 7fcccaffd700 1 -- 172.21.15.154:0/2860812403 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcccc0150a0 con 0x7fccdc1564e0 2024-03-14T09:01:43.579 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 msgr2=0x7fccbc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 0x7fccbc06ccf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fccd0006fd0 tx=0x7fccd0008040 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 msgr2=0x7fccdc1e8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcccc014040 tx=0x7fcccc004750 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 shutdown_connections 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fccdc156960 0x7fccdc1412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccdc0a9650 0x7fccdc0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fccbc06a830 0x7fccbc06ccf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 --2- 172.21.15.154:0/2860812403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc1564e0 0x7fccdc1e8970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 >> 172.21.15.154:0/2860812403 conn(0x7fccdc0c3880 msgr2=0x7fccdc0aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 shutdown_connections 2024-03-14T09:01:43.582 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.581+0000 7fcce0c42700 1 -- 172.21.15.154:0/2860812403 wait complete. 2024-03-14T09:01:43.627 INFO:tasks.cephfs_test_runner:Starting test: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-03-14T09:01:43.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-03-14T09:01:43.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.750+0000 7f79367fc700 1 Processor -- start 2024-03-14T09:01:43.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.750+0000 7f79367fc700 1 -- start start 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f79367fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f79300c4380 0x7f793014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f79367fc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300acd10 con 0x7f79300c5d60 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f79367fc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79300ac9c0 con 0x7f79300702f0 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f79367fc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7930150110 con 0x7f79300c4380 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f7934d99700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f79300c5d60 0x7f793015cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48472/0 (socket says 172.21.15.154:48472) 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f7934d99700 1 -- 172.21.15.154:0/1149981338 learned_addr learned my addr 172.21.15.154:0/1149981338 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f792f7fe700 1 --2- 172.21.15.154:0/1149981338 >> v2:172.21.15.191:3300/0 conn(0x7f79300c4380 0x7f793014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672380244 0 0) 0x7f79300acd10 con 0x7f79300c5d60 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.751+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79180035b0 con 0x7f79300c5d60 2024-03-14T09:01:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792f7fe700 1 -- 172.21.15.154:0/1149981338 >> v1:172.21.15.191:6789/0 conn(0x7f79300702f0 legacy=0x7f79300c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792f7fe700 1 -- 172.21.15.154:0/1149981338 >> v1:172.21.15.154:6789/0 conn(0x7f79300c5d60 legacy=0x7f793015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252144534 0 0) 0x7f79300ac9c0 con 0x7f79300702f0 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792f7fe700 1 -- 172.21.15.154:0/1149981338 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79300707f0 con 0x7f79300c4380 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792f7fe700 1 --2- 172.21.15.154:0/1149981338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c4380 0x7f793014fbd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7920009840 tx=0x7f7920009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf4e0dfc3dec0cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928001970 con 0x7f79300c4380 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920004030 con 0x7f79300c4380 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.752+0000 7f792effd700 1 -- 172.21.15.154:0/1149981338 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79200043c0 con 0x7f79300c4380 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 -- 172.21.15.154:0/1149981338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c4380 msgr2=0x7f793014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 --2- 172.21.15.154:0/1149981338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c4380 0x7f793014fbd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7920009840 tx=0x7f7920009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 -- 172.21.15.154:0/1149981338 shutdown_connections 2024-03-14T09:01:43.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 --2- 172.21.15.154:0/1149981338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c4380 0x7f793014fbd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 -- 172.21.15.154:0/1149981338 >> 172.21.15.154:0/1149981338 conn(0x7f79300a9bf0 msgr2=0x7f79300a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:43.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 -- 172.21.15.154:0/1149981338 shutdown_connections 2024-03-14T09:01:43.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.753+0000 7f79367fc700 1 -- 172.21.15.154:0/1149981338 wait complete. 2024-03-14T09:01:43.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.754+0000 7f79367fc700 1 Processor -- start 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.754+0000 7f79367fc700 1 -- start start 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79300c4380 0x7f79300c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c5d60 0x7f79301ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7930154550 con 0x7f79300702f0 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7930144ec0 con 0x7f79300c5d60 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79301451a0 con 0x7f79300c4380 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59226/0 (socket says 172.21.15.154:59226) 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 -- 172.21.15.154:0/3521312266 learned_addr learned my addr 172.21.15.154:0/3521312266 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f7934d99700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c5d60 0x7f79301ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792f7fe700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79300c4380 0x7f79300c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 -- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79300c4380 msgr2=0x7f79300c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79300c4380 0x7f79300c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 -- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c5d60 msgr2=0x7f79301ed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c5d60 0x7f79301ed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 -- 172.21.15.154:0/3521312266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200094a0 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792ffff700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f792400c730 tx=0x7f792400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240054e0 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7924003680 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793006d480 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924003a30 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.755+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79300c1a20 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.756+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793006d2e0 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.757+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7924003b90 con 0x7f79300702f0 2024-03-14T09:01:43.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.757+0000 7f792cff9700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 0x7f791c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:43.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.757+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79240881e0 con 0x7f79300702f0 2024-03-14T09:01:43.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.757+0000 7f792f7fe700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 0x7f791c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:43.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.758+0000 7f792f7fe700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 0x7f791c06ccf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7920009b90 tx=0x7f7920008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:43.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:43.761+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f793006d2e0 con 0x7f79300702f0 2024-03-14T09:01:44.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.036+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7f793006d2e0 con 0x7f79300702f0 2024-03-14T09:01:44.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.601+0000 7f792cff9700 1 -- 172.21.15.154:0/3521312266 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v42) v1 ==== 162+0+0 (secure 0 0 0) 0x7f793006d2e0 con 0x7f79300702f0 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 msgr2=0x7f791c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 0x7f791c06ccf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7920009b90 tx=0x7f7920008000 comp rx=0 tx=0).stop 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 msgr2=0x7f7930154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f792400c730 tx=0x7f792400ca40 comp rx=0 tx=0).stop 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 shutdown_connections 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f79300c5d60 0x7f79301ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f791c06a830 0x7f791c06ccf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79300702f0 0x7f7930154010 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 --2- 172.21.15.154:0/3521312266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79300c4380 0x7f79300c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 >> 172.21.15.154:0/3521312266 conn(0x7f79300a9bf0 msgr2=0x7f7930148200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:44.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 shutdown_connections 2024-03-14T09:01:44.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.604+0000 7f79367fc700 1 -- 172.21.15.154:0/3521312266 wait complete. 2024-03-14T09:01:44.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T09:01:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.745+0000 7f2fa6969700 1 Processor -- start 2024-03-14T09:01:44.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 -- start start 2024-03-14T09:01:44.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fa0070a10 0x7f2fa00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fa006d600 0x7f2fa00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00c5f10 con 0x7f2fa0070a10 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00c7d50 con 0x7f2fa006d600 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2fa6969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c8030 con 0x7f2fa006c6e0 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2f9f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fa0070a10 0x7f2fa00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2f9f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fa0070a10 0x7f2fa00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59228/0 (socket says 172.21.15.154:59228) 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.746+0000 7f2f9f7fe700 1 -- 172.21.15.154:0/1795622232 learned_addr learned my addr 172.21.15.154:0/1795622232 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9ffff700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2fa4f06700 1 --2- 172.21.15.154:0/1795622232 >> v2:172.21.15.191:3300/0 conn(0x7f2fa006d600 0x7f2fa00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 -- 172.21.15.154:0/1795622232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 msgr2=0x7f2fa006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 -- 172.21.15.154:0/1795622232 >> v2:172.21.15.191:3300/0 conn(0x7f2fa006d600 msgr2=0x7f2fa00c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 --2- 172.21.15.154:0/1795622232 >> v2:172.21.15.191:3300/0 conn(0x7f2fa006d600 0x7f2fa00c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 -- 172.21.15.154:0/1795622232 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00c8310 con 0x7f2fa0070a10 2024-03-14T09:01:44.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9f7fe700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00c63a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00a7b30 tx=0x7f2f88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b9751d388bb50e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9effd700 1 -- 172.21.15.154:0/1795622232 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800b870 con 0x7f2fa0070a10 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9effd700 1 -- 172.21.15.154:0/1795622232 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8800b9d0 con 0x7f2fa0070a10 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2f9effd700 1 -- 172.21.15.154:0/1795622232 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8800bd60 con 0x7f2fa0070a10 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2fa6969700 1 -- 172.21.15.154:0/1795622232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 msgr2=0x7f2fa00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.747+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00c63a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00a7b30 tx=0x7f2f88009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 -- 172.21.15.154:0/1795622232 shutdown_connections 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1795622232 >> v2:172.21.15.191:3300/0 conn(0x7f2fa006d600 0x7f2fa00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1795622232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00c63a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 -- 172.21.15.154:0/1795622232 >> 172.21.15.154:0/1795622232 conn(0x7f2fa00a9bf0 msgr2=0x7f2fa00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 -- 172.21.15.154:0/1795622232 shutdown_connections 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.748+0000 7f2fa6969700 1 -- 172.21.15.154:0/1795622232 wait complete. 2024-03-14T09:01:44.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.749+0000 7f2fa6969700 1 Processor -- start 2024-03-14T09:01:44.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.749+0000 7f2fa6969700 1 -- start start 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa01da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006d600 0x7f2fa01ed360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01ee030 con 0x7f2fa0070a10 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa009a500 con 0x7f2fa006d600 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01edd50 con 0x7f2fa006c6e0 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59238/0 (socket says 172.21.15.154:59238) 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 -- 172.21.15.154:0/1898609067 learned_addr learned my addr 172.21.15.154:0/1898609067 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:44.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2f9ffff700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa01da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2f9f7fe700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006d600 0x7f2fa01ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 -- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 msgr2=0x7f2fa01da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa01da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 -- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006d600 msgr2=0x7f2fa01ed360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006d600 0x7f2fa01ed360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 -- 172.21.15.154:0/1898609067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009550 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa4f06700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2f98007e70 tx=0x7f2f9800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800ee50 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98014420 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa009a190 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980147d0 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.750+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa01ed8a0 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.751+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa01edb20 con 0x7f2fa0070a10 2024-03-14T09:01:44.754 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.752+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2f98014930 con 0x7f2fa0070a10 2024-03-14T09:01:44.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.752+0000 7f2f9cff9700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:44.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.752+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f980894e0 con 0x7f2fa0070a10 2024-03-14T09:01:44.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.752+0000 7f2f9ffff700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:44.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.753+0000 7f2f9ffff700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00ad4a0 tx=0x7f2f90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:44.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.756+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2f98054910 con 0x7f2fa0070a10 2024-03-14T09:01:44.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.950+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fa00c3190 con 0x7f2fa0070a10 2024-03-14T09:01:44.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.955+0000 7f2f9cff9700 1 -- 172.21.15.154:0/1898609067 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f98057f30 con 0x7f2fa0070a10 2024-03-14T09:01:44.956 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 msgr2=0x7f2f8006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00ad4a0 tx=0x7f2f90008040 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 msgr2=0x7f2fa00abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2f98007e70 tx=0x7f2f9800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 shutdown_connections 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f8006a830 0x7f2f8006ccf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa0070a10 0x7f2fa00abc60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006c6e0 0x7f2fa01da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 --2- 172.21.15.154:0/1898609067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006d600 0x7f2fa01ed360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 >> 172.21.15.154:0/1898609067 conn(0x7f2fa00a9bf0 msgr2=0x7f2fa00c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 shutdown_connections 2024-03-14T09:01:44.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:44.957+0000 7f2fa6969700 1 -- 172.21.15.154:0/1898609067 wait complete. 2024-03-14T09:01:44.968 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:01:44.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:01:45.016 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:01:45.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:01:45.072 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:45.072 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:45.089 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:45.089 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:45.147 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:01:45.148 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:45.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:45.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.368+0000 7f89dc7d7700 1 Processor -- start 2024-03-14T09:01:45.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.368+0000 7f89dc7d7700 1 -- start start 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89dc7d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89d40a8540 0x7f89d413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89dc7d7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89d40c3250 0x7f89d4154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89dc7d7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d413ade0 con 0x7f89d41550b0 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89dc7d7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d406fb30 con 0x7f89d40c3250 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89dc7d7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d406ec30 con 0x7f89d40a8540 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89d40c3250 0x7f89d4154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89d40c3250 0x7f89d4154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59248/0 (socket says 172.21.15.154:59248) 2024-03-14T09:01:45.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 -- 172.21.15.154:0/4269532153 learned_addr learned my addr 172.21.15.154:0/4269532153 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89da573700 1 --2- 172.21.15.154:0/4269532153 >> v2:172.21.15.191:3300/0 conn(0x7f89d40a8540 0x7f89d413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 -- 172.21.15.154:0/4269532153 >> v2:172.21.15.191:3300/0 conn(0x7f89d40a8540 msgr2=0x7f89d413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 --2- 172.21.15.154:0/4269532153 >> v2:172.21.15.191:3300/0 conn(0x7f89d40a8540 0x7f89d413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 -- 172.21.15.154:0/4269532153 >> v1:172.21.15.191:6789/0 conn(0x7f89d41550b0 legacy=0x7f89d40c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 -- 172.21.15.154:0/4269532153 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d406ef10 con 0x7f89d40c3250 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9d72700 1 --2- 172.21.15.154:0/4269532153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d4154b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89d4070da0 tx=0x7f89c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01702e60d3edec server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9571700 1 -- 172.21.15.154:0/4269532153 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4004230 con 0x7f89d40c3250 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9571700 1 -- 172.21.15.154:0/4269532153 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4004390 con 0x7f89d40c3250 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.369+0000 7f89d9571700 1 -- 172.21.15.154:0/4269532153 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40036a0 con 0x7f89d40c3250 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 -- 172.21.15.154:0/4269532153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 msgr2=0x7f89d4154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/4269532153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d4154b70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89d4070da0 tx=0x7f89c4009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 -- 172.21.15.154:0/4269532153 shutdown_connections 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/4269532153 >> v2:172.21.15.191:3300/0 conn(0x7f89d40a8540 0x7f89d413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/4269532153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d4154b70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 -- 172.21.15.154:0/4269532153 >> 172.21.15.154:0/4269532153 conn(0x7f89d40c3880 msgr2=0x7f89d40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:45.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 -- 172.21.15.154:0/4269532153 shutdown_connections 2024-03-14T09:01:45.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.370+0000 7f89dc7d7700 1 -- 172.21.15.154:0/4269532153 wait complete. 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 Processor -- start 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 -- start start 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d40a8540 0x7f89d40c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d41550b0 0x7f89d4137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41515d0 con 0x7f89d40c3250 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d414ed00 con 0x7f89d41550b0 2024-03-14T09:01:45.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.372+0000 7f89dc7d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40aa510 con 0x7f89d40a8540 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59252/0 (socket says 172.21.15.154:59252) 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 -- 172.21.15.154:0/2195490724 learned_addr learned my addr 172.21.15.154:0/2195490724 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89da573700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d40a8540 0x7f89d40c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89dad74700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d41550b0 0x7f89d4137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 -- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d40a8540 msgr2=0x7f89d40c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d40a8540 0x7f89d40c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 -- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d41550b0 msgr2=0x7f89d4137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d41550b0 0x7f89d4137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 -- 172.21.15.154:0/2195490724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c4009550 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89d9d72700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009c10 tx=0x7f89c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4003d80 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c401f070 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d41512f0 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4011850 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.373+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d40aa1a0 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.374+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d40a9d20 con 0x7f89d40c3250 2024-03-14T09:01:45.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.375+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f89c4021030 con 0x7f89d40c3250 2024-03-14T09:01:45.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.375+0000 7f89caffd700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 0x7f89b806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.375+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89c408a000 con 0x7f89d40c3250 2024-03-14T09:01:45.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.375+0000 7f89da573700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 0x7f89b806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.376+0000 7f89da573700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 0x7f89b806cd20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009930 tx=0x7f89d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.379+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f89d40a9d20 con 0x7f89d40c3250 2024-03-14T09:01:45.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.573+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89d40abac0 con 0x7f89d40c3250 2024-03-14T09:01:45.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.574+0000 7f89caffd700 1 -- 172.21.15.154:0/2195490724 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f89d40abac0 con 0x7f89d40c3250 2024-03-14T09:01:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:45.574 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:45.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 msgr2=0x7f89b806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 0x7f89b806cd20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009930 tx=0x7f89d0008040 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 msgr2=0x7f89d41374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009c10 tx=0x7f89c4004750 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 shutdown_connections 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d41550b0 0x7f89d4137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d40a8540 0x7f89d40c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.576+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f89b806a860 0x7f89b806cd20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.577+0000 7f89dc7d7700 1 --2- 172.21.15.154:0/2195490724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89d40c3250 0x7f89d41374f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.577+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 >> 172.21.15.154:0/2195490724 conn(0x7f89d40c3880 msgr2=0x7f89d40c2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.577+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 shutdown_connections 2024-03-14T09:01:45.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.577+0000 7f89dc7d7700 1 -- 172.21.15.154:0/2195490724 wait complete. 2024-03-14T09:01:45.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:45.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:45.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.713+0000 7f372146a700 1 Processor -- start 2024-03-14T09:01:45.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 -- start start 2024-03-14T09:01:45.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0ab730 0x7f371c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f371c155260 0x7f371c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f371c0bf9c0 0x7f371c0bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c06b6c0 con 0x7f371c155260 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c06a770 con 0x7f371c0bf9c0 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f372146a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c06df70 con 0x7f371c0ab730 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f371a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f371c155260 0x7f371c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f371a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f371c155260 0x7f371c06e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59264/0 (socket says 172.21.15.154:59264) 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.714+0000 7f371a7fc700 1 -- 172.21.15.154:0/1478702511 learned_addr learned my addr 172.21.15.154:0/1478702511 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371affd700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0ab730 0x7f371c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371b7fe700 1 --2- 172.21.15.154:0/1478702511 >> v2:172.21.15.191:3300/0 conn(0x7f371c0bf9c0 0x7f371c0bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 -- 172.21.15.154:0/1478702511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0ab730 msgr2=0x7f371c0aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0ab730 0x7f371c0aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 -- 172.21.15.154:0/1478702511 >> v2:172.21.15.191:3300/0 conn(0x7f371c0bf9c0 msgr2=0x7f371c0bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1478702511 >> v2:172.21.15.191:3300/0 conn(0x7f371c0bf9c0 0x7f371c0bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 -- 172.21.15.154:0/1478702511 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c06dc40 con 0x7f371c155260 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c06e400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f371c066db0 tx=0x7f3710009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8702085f11b25e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f3719ffb700 1 -- 172.21.15.154:0/1478702511 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000b870 con 0x7f371c155260 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f3719ffb700 1 -- 172.21.15.154:0/1478702511 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000b9d0 con 0x7f371c155260 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f3719ffb700 1 -- 172.21.15.154:0/1478702511 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371000bd60 con 0x7f371c155260 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.715+0000 7f372146a700 1 -- 172.21.15.154:0/1478702511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 msgr2=0x7f371c06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c06e400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f371c066db0 tx=0x7f3710009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 -- 172.21.15.154:0/1478702511 shutdown_connections 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 --2- 172.21.15.154:0/1478702511 >> v2:172.21.15.191:3300/0 conn(0x7f371c0bf9c0 0x7f371c0bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c0ab730 0x7f371c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 --2- 172.21.15.154:0/1478702511 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c06e400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 -- 172.21.15.154:0/1478702511 >> 172.21.15.154:0/1478702511 conn(0x7f371c0c3090 msgr2=0x7f371c0c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:45.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 -- 172.21.15.154:0/1478702511 shutdown_connections 2024-03-14T09:01:45.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.716+0000 7f372146a700 1 -- 172.21.15.154:0/1478702511 wait complete. 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.717+0000 7f372146a700 1 Processor -- start 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.717+0000 7f372146a700 1 -- start start 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0bf9c0 0x7f371c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c14dc80 0x7f371c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1566d0 con 0x7f371c155260 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c156220 con 0x7f371c0bf9c0 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c0c4e70 con 0x7f371c14dc80 2024-03-14T09:01:45.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59272/0 (socket says 172.21.15.154:59272) 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 -- 172.21.15.154:0/1548610017 learned_addr learned my addr 172.21.15.154:0/1548610017 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371b7fe700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c14dc80 0x7f371c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371affd700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0bf9c0 0x7f371c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 -- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c14dc80 msgr2=0x7f371c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c14dc80 0x7f371c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 -- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0bf9c0 msgr2=0x7f371c0c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0bf9c0 0x7f371c0c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 -- 172.21.15.154:0/1548610017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710009550 con 0x7f371c155260 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f371a7fc700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f371c0701d0 tx=0x7f3710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710004480 con 0x7f371c155260 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37100045e0 con 0x7f371c155260 2024-03-14T09:01:45.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100038e0 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c155c60 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.718+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c0a8e40 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.719+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c074120 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.720+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3710021070 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.720+0000 7f37037fe700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 0x7f370806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.721+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f371008a4f0 con 0x7f371c155260 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.721+0000 7f371affd700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 0x7f370806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:45.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.721+0000 7f371affd700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 0x7f370806ccf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f370c006b20 tx=0x7f370c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:45.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.724+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3710055a80 con 0x7f371c155260 2024-03-14T09:01:45.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.916+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f371c0697d0 con 0x7f371c155260 2024-03-14T09:01:45.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.917+0000 7f37037fe700 1 -- 172.21.15.154:0/1548610017 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37100150a0 con 0x7f371c155260 2024-03-14T09:01:45.916 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 msgr2=0x7f370806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 0x7f370806ccf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f370c006b20 tx=0x7f370c008040 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 msgr2=0x7f371c0c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f371c0701d0 tx=0x7f3710004750 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 shutdown_connections 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f371c14dc80 0x7f371c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f371c0bf9c0 0x7f371c0c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f370806a830 0x7f370806ccf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 --2- 172.21.15.154:0/1548610017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f371c155260 0x7f371c0c6ce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 >> 172.21.15.154:0/1548610017 conn(0x7f371c0c3090 msgr2=0x7f371c153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 shutdown_connections 2024-03-14T09:01:45.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:45.919+0000 7f372146a700 1 -- 172.21.15.154:0/1548610017 wait complete. 2024-03-14T09:01:45.930 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:46.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.055+0000 7f44b8b0a700 1 Processor -- start 2024-03-14T09:01:46.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.055+0000 7f44b8b0a700 1 -- start start 2024-03-14T09:01:46.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b8b0a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44b40c31c0 0x7f44b40c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b8b0a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40c5f10 con 0x7f44b406c6e0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b8b0a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40c7d50 con 0x7f44b4070a10 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b8b0a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b40c8030 con 0x7f44b40c31c0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44b40c31c0 0x7f44b40c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f44b406c6e0 0x7f44b406d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48488/0 (socket says 172.21.15.154:48488) 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b259c700 1 -- 172.21.15.154:0/1774933248 learned_addr learned my addr 172.21.15.154:0/1774933248 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b2d9d700 1 -- 172.21.15.154:0/1774933248 >> v1:172.21.15.191:6789/0 conn(0x7f44b4070a10 legacy=0x7f44b40c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b2d9d700 1 -- 172.21.15.154:0/1774933248 >> v1:172.21.15.154:6789/0 conn(0x7f44b406c6e0 legacy=0x7f44b406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b2d9d700 1 -- 172.21.15.154:0/1774933248 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40c8310 con 0x7f44b40c31c0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b2d9d700 1 --2- 172.21.15.154:0/1774933248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b40c31c0 0x7f44b40c6ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009d90 tx=0x7f44ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b706d4bf66dd9c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b159a700 1 -- 172.21.15.154:0/1774933248 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00e070 con 0x7f44b40c31c0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b159a700 1 -- 172.21.15.154:0/1774933248 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac00bb90 con 0x7f44b40c31c0 2024-03-14T09:01:46.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.056+0000 7f44b159a700 1 -- 172.21.15.154:0/1774933248 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac003a20 con 0x7f44b40c31c0 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 -- 172.21.15.154:0/1774933248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b40c31c0 msgr2=0x7f44b40c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/1774933248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b40c31c0 0x7f44b40c6ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009d90 tx=0x7f44ac00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 -- 172.21.15.154:0/1774933248 shutdown_connections 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/1774933248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b40c31c0 0x7f44b40c6ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 -- 172.21.15.154:0/1774933248 >> 172.21.15.154:0/1774933248 conn(0x7f44b40a9bf0 msgr2=0x7f44b40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 -- 172.21.15.154:0/1774933248 shutdown_connections 2024-03-14T09:01:46.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.057+0000 7f44b8b0a700 1 -- 172.21.15.154:0/1774933248 wait complete. 2024-03-14T09:01:46.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.058+0000 7f44b8b0a700 1 Processor -- start 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.058+0000 7f44b8b0a700 1 -- start start 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 0x7f44b41da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 0x7f44b41daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b409a430 con 0x7f44b406c6e0 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b409a680 con 0x7f44b4070a10 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b8b0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b4099c10 con 0x7f44b40c31c0 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59290/0 (socket says 172.21.15.154:59290) 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 -- 172.21.15.154:0/2472274025 learned_addr learned my addr 172.21.15.154:0/2472274025 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b2d9d700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 0x7f44b41daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b1d9b700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 0x7f44b41da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 -- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 msgr2=0x7f44b41daaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 0x7f44b41daaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 -- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 msgr2=0x7f44b41da560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 0x7f44b41da560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b259c700 1 -- 172.21.15.154:0/2472274025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac009a70 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b1d9b700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 0x7f44b41da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.059+0000 7f44b2d9d700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 0x7f44b41daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44b259c700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c490 tx=0x7f44a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4005d70 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40070d0 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4007460 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b4099ef0 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.060+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b406f5a0 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.061+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b4099a70 con 0x7f44b406c6e0 2024-03-14T09:01:46.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.061+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f44a40075c0 con 0x7f44b406c6e0 2024-03-14T09:01:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.061+0000 7f44a2ffd700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 0x7f449406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.062+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44a40881e0 con 0x7f44b406c6e0 2024-03-14T09:01:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.062+0000 7f44b1d9b700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 0x7f449406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.062+0000 7f44b1d9b700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 0x7f449406ccf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44b40681a0 tx=0x7f44a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.066+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f44b4099a70 con 0x7f44b406c6e0 2024-03-14T09:01:46.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.259+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44b4099a70 con 0x7f44b406c6e0 2024-03-14T09:01:46.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.259+0000 7f44a2ffd700 1 -- 172.21.15.154:0/2472274025 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44b4099a70 con 0x7f44b406c6e0 2024-03-14T09:01:46.259 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 msgr2=0x7f449406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 0x7f449406ccf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44b40681a0 tx=0x7f44a8009040 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 msgr2=0x7f44b406f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c490 tx=0x7f44a400c850 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 shutdown_connections 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f44b40c31c0 0x7f44b41daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f449406a830 0x7f449406ccf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44b406c6e0 0x7f44b406f060 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 --2- 172.21.15.154:0/2472274025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44b4070a10 0x7f44b41da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 >> 172.21.15.154:0/2472274025 conn(0x7f44b40a9bf0 msgr2=0x7f44b40a8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 shutdown_connections 2024-03-14T09:01:46.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.262+0000 7f44b8b0a700 1 -- 172.21.15.154:0/2472274025 wait complete. 2024-03-14T09:01:46.274 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:46.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.398+0000 7f23fb5b7700 1 Processor -- start 2024-03-14T09:01:46.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23fb5b7700 1 -- start start 2024-03-14T09:01:46.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23fb5b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23f414edf0 0x7f23f41518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23fb5b7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f4155a40 con 0x7f23f4151e00 2024-03-14T09:01:46.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23fb5b7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f4072a90 con 0x7f23f40c26c0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23fb5b7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f40c2f10 con 0x7f23f414edf0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23f8b52700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23f414edf0 0x7f23f41518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23f9b54700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f23f4151e00 0x7f23f415be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48492/0 (socket says 172.21.15.154:48492) 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.399+0000 7f23f9b54700 1 -- 172.21.15.154:0/4097829351 learned_addr learned my addr 172.21.15.154:0/4097829351 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23f8b52700 1 -- 172.21.15.154:0/4097829351 >> v1:172.21.15.191:6789/0 conn(0x7f23f40c26c0 legacy=0x7f23f40c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23f8b52700 1 -- 172.21.15.154:0/4097829351 >> v1:172.21.15.154:6789/0 conn(0x7f23f4151e00 legacy=0x7f23f415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23f8b52700 1 -- 172.21.15.154:0/4097829351 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f41556d0 con 0x7f23f414edf0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23ebfff700 1 -- 172.21.15.154:0/4097829351 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283249345 0 0) 0x7f23f4155a40 con 0x7f23f4151e00 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23f8b52700 1 --2- 172.21.15.154:0/4097829351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f414edf0 0x7f23f41518c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009840 tx=0x7f23f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72beec415e178d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23ebfff700 1 -- 172.21.15.154:0/4097829351 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0004230 con 0x7f23f414edf0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23ebfff700 1 -- 172.21.15.154:0/4097829351 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f0004390 con 0x7f23f414edf0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.400+0000 7f23ebfff700 1 -- 172.21.15.154:0/4097829351 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00036a0 con 0x7f23f414edf0 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 -- 172.21.15.154:0/4097829351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f414edf0 msgr2=0x7f23f41518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/4097829351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f414edf0 0x7f23f41518c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009840 tx=0x7f23f0009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:46.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 -- 172.21.15.154:0/4097829351 shutdown_connections 2024-03-14T09:01:46.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/4097829351 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f414edf0 0x7f23f41518c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 -- 172.21.15.154:0/4097829351 >> 172.21.15.154:0/4097829351 conn(0x7f23f406a770 msgr2=0x7f23f406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 -- 172.21.15.154:0/4097829351 shutdown_connections 2024-03-14T09:01:46.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.401+0000 7f23fb5b7700 1 -- 172.21.15.154:0/4097829351 wait complete. 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 Processor -- start 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 -- start start 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23f414edf0 0x7f23f4071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4151e00 0x7f23f41ed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4153030 con 0x7f23f40c26c0 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f40719a0 con 0x7f23f4151e00 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.402+0000 7f23fb5b7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4071c80 con 0x7f23f414edf0 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59310/0 (socket says 172.21.15.154:59310) 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 -- 172.21.15.154:0/86323737 learned_addr learned my addr 172.21.15.154:0/86323737 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 -- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23f414edf0 msgr2=0x7f23f4071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23f414edf0 0x7f23f4071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 -- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4151e00 msgr2=0x7f23f41ed4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4151e00 0x7f23f41ed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 -- 172.21.15.154:0/86323737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f00094a0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23f9353700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c4d0 tx=0x7f23e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011790 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40118f0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f41e9ff0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4011ca0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.403+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f41ea5b0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.404+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f40697d0 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.405+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f23e401c460 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.405+0000 7f23e9ffb700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 0x7f23d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.405+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23e4088b40 con 0x7f23f40c26c0 2024-03-14T09:01:46.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.405+0000 7f23f8b52700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 0x7f23d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.406+0000 7f23f8b52700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 0x7f23d806ccf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23f4152a40 tx=0x7f23f0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.410+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f23e4053ff0 con 0x7f23f40c26c0 2024-03-14T09:01:46.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.602+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23f4066860 con 0x7f23f40c26c0 2024-03-14T09:01:46.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.602+0000 7f23e9ffb700 1 -- 172.21.15.154:0/86323737 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23e4057610 con 0x7f23f40c26c0 2024-03-14T09:01:46.602 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.604+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 msgr2=0x7f23d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.604+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 0x7f23d806ccf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f23f4152a40 tx=0x7f23f0010000 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.604+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 msgr2=0x7f23f41eaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.604+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f23e400c4d0 tx=0x7f23e400c890 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 shutdown_connections 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4151e00 0x7f23f41ed4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f23d806a830 0x7f23d806ccf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23f40c26c0 0x7f23f41eaad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 --2- 172.21.15.154:0/86323737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23f414edf0 0x7f23f4071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 >> 172.21.15.154:0/86323737 conn(0x7f23f406a770 msgr2=0x7f23f414c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 shutdown_connections 2024-03-14T09:01:46.606 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.605+0000 7f23fb5b7700 1 -- 172.21.15.154:0/86323737 wait complete. 2024-03-14T09:01:46.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:01:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 Processor -- start 2024-03-14T09:01:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 -- start start 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcbc414edf0 0x7fcbc41518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbc4151e00 0x7fcbc415be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc4155a40 con 0x7fcbc40c26c0 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc40c2f10 con 0x7fcbc4151e00 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.740+0000 7fcbca907700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc4072a90 con 0x7fcbc414edf0 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbc4151e00 0x7fcbc415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc3fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcbc40c26c0 0x7fcbc40c55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48508/0 (socket says 172.21.15.154:48508) 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc3fff700 1 -- 172.21.15.154:0/460167559 learned_addr learned my addr 172.21.15.154:0/460167559 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/460167559 >> v2:172.21.15.191:3300/0 conn(0x7fcbc414edf0 msgr2=0x7fcbc41518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 --2- 172.21.15.154:0/460167559 >> v2:172.21.15.191:3300/0 conn(0x7fcbc414edf0 0x7fcbc41518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/460167559 >> v1:172.21.15.154:6789/0 conn(0x7fcbc40c26c0 legacy=0x7fcbc40c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/460167559 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729528349 0 0) 0x7fcbc4155a40 con 0x7fcbc40c26c0 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/460167559 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc41556d0 con 0x7fcbc4151e00 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc8ea4700 1 --2- 172.21.15.154:0/460167559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc415be50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbc406c6f0 tx=0x7fcbbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d672c716bcd1da server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/460167559 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00e070 con 0x7fcbc4151e00 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/460167559 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc00bbd0 con 0x7fcbc4151e00 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.741+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/460167559 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc012430 con 0x7fcbc4151e00 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 -- 172.21.15.154:0/460167559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 msgr2=0x7fcbc415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 --2- 172.21.15.154:0/460167559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc415be50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbc406c6f0 tx=0x7fcbbc00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 -- 172.21.15.154:0/460167559 shutdown_connections 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 --2- 172.21.15.154:0/460167559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc415be50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 --2- 172.21.15.154:0/460167559 >> v2:172.21.15.191:3300/0 conn(0x7fcbc414edf0 0x7fcbc41518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 -- 172.21.15.154:0/460167559 >> 172.21.15.154:0/460167559 conn(0x7fcbc406a770 msgr2=0x7fcbc406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 -- 172.21.15.154:0/460167559 shutdown_connections 2024-03-14T09:01:46.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.742+0000 7fcbca907700 1 -- 172.21.15.154:0/460167559 wait complete. 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.743+0000 7fcbca907700 1 Processor -- start 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.743+0000 7fcbca907700 1 -- start start 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc40c26c0 0x7fcbc41ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc414edf0 0x7fcbc41ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc415df80 con 0x7fcbc4151e00 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc415e260 con 0x7fcbc40c26c0 2024-03-14T09:01:46.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41ed990 con 0x7fcbc414edf0 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59326/0 (socket says 172.21.15.154:59326) 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/1941046813 learned_addr learned my addr 172.21.15.154:0/1941046813 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc414edf0 msgr2=0x7fcbc41ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc414edf0 0x7fcbc41ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc40c26c0 msgr2=0x7fcbc41ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc40c26c0 0x7fcbc41ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 -- 172.21.15.154:0/1941046813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc009a70 con 0x7fcbc4151e00 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc8ea4700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40740c0 tx=0x7fcbbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc012430 con 0x7fcbc4151e00 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc020070 con 0x7fcbc4151e00 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc41edc70 con 0x7fcbc4151e00 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc004280 con 0x7fcbc4151e00 2024-03-14T09:01:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.744+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc40a93f0 con 0x7fcbc4151e00 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.745+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc41ee190 con 0x7fcbc4151e00 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.746+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcbbc0043e0 con 0x7fcbc4151e00 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.746+0000 7fcbc0ff9700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 0x7fcbb006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.746+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbbc022070 con 0x7fcbc4151e00 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.746+0000 7fcbc3fff700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 0x7fcbb006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.747+0000 7fcbc3fff700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 0x7fcbb006ccf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80011e0 tx=0x7fcbb80079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.751+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcbc41ee190 con 0x7fcbc4151e00 2024-03-14T09:01:46.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.958+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcbc41ee190 con 0x7fcbc4151e00 2024-03-14T09:01:46.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.959+0000 7fcbc0ff9700 1 -- 172.21.15.154:0/1941046813 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcbc41ee190 con 0x7fcbc4151e00 2024-03-14T09:01:46.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 msgr2=0x7fcbb006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 0x7fcbb006ccf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80011e0 tx=0x7fcbb80079b0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 msgr2=0x7fcbc40a7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40740c0 tx=0x7fcbbc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 shutdown_connections 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcbb006a830 0x7fcbb006ccf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc4151e00 0x7fcbc40a7330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcbc40c26c0 0x7fcbc41ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 --2- 172.21.15.154:0/1941046813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcbc414edf0 0x7fcbc41ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 >> 172.21.15.154:0/1941046813 conn(0x7fcbc406a770 msgr2=0x7fcbc40c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 shutdown_connections 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:46.961+0000 7fcbca907700 1 -- 172.21.15.154:0/1941046813 wait complete. 2024-03-14T09:01:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:01:46.972 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T09:01:46.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:46.973 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-14T09:01:47.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:01:47.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:47.012 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T09:01:47.086 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:47.086 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:47.107 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:47.108 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:47.169 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:01:47.169 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:01:47.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:47.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.351+0000 7f0a6f43f700 1 Processor -- start 2024-03-14T09:01:47.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.351+0000 7f0a6f43f700 1 -- start start 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6f43f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0a680a8540 0x7f0a6813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6f43f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6813ade0 con 0x7f0a680c1780 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6f43f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6806fb30 con 0x7f0a680c3250 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6f43f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6806ec30 con 0x7f0a680a8540 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6d9dc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0a680c1780 0x7f0a680c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48512/0 (socket says 172.21.15.154:48512) 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.352+0000 7f0a6d9dc700 1 -- 172.21.15.154:0/1085039648 learned_addr learned my addr 172.21.15.154:0/1085039648 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:47.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a6d1db700 1 --2- 172.21.15.154:0/1085039648 >> v2:172.21.15.191:3300/0 conn(0x7f0a680a8540 0x7f0a6813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493423500 0 0) 0x7f0a60001490 con 0x7f0a680c1780 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a540035b0 con 0x7f0a680c1780 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818253331 0 0) 0x7f0a540035b0 con 0x7f0a680c1780 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 >> v2:172.21.15.191:3300/0 conn(0x7f0a680a8540 msgr2=0x7f0a6813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 --2- 172.21.15.154:0/1085039648 >> v2:172.21.15.191:3300/0 conn(0x7f0a680a8540 0x7f0a6813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 >> v1:172.21.15.191:6789/0 conn(0x7f0a680c3250 legacy=0x7f0a68154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6806ef10 con 0x7f0a680c1780 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0a60001b80 con 0x7f0a680c1780 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 >> v1:172.21.15.154:6789/0 conn(0x7f0a680c1780 legacy=0x7f0a680c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a54003a00 0x7f0a54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a54004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a54005330 0x7f0a5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806ec30 con 0x7f0a54004850 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5400c200 con 0x7f0a54005330 2024-03-14T09:01:47.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5400c3c0 con 0x7f0a54003a00 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a54004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a6d1db700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a54003a00 0x7f0a54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.353+0000 7f0a6d9dc700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a54005330 0x7f0a5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 -- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a54003a00 msgr2=0x7f0a54003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a54003a00 0x7f0a54003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 -- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a54005330 msgr2=0x7f0a5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a54005330 0x7f0a5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6d9dc700 1 -- 172.21.15.154:0/1085039648 reap_dead start 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 -- 172.21.15.154:0/1085039648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540035b0 con 0x7f0a54004850 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a54004cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007fd0 tx=0x7f0a54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470f35c652eb2cb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a640052d0 con 0x7f0a54004850 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a64005450 con 0x7f0a54004850 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a5bfff700 1 -- 172.21.15.154:0/1085039648 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a640038c0 con 0x7f0a54004850 2024-03-14T09:01:47.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6f43f700 1 -- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 msgr2=0x7f0a54004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a54004cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007fd0 tx=0x7f0a54005880 comp rx=0 tx=0).stop 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6f43f700 1 -- 172.21.15.154:0/1085039648 shutdown_connections 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/1085039648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a54004cd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.354+0000 7f0a6f43f700 1 -- 172.21.15.154:0/1085039648 >> 172.21.15.154:0/1085039648 conn(0x7f0a680c3880 msgr2=0x7f0a680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.355+0000 7f0a6f43f700 1 -- 172.21.15.154:0/1085039648 shutdown_connections 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.355+0000 7f0a6f43f700 1 -- 172.21.15.154:0/1085039648 wait complete. 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 Processor -- start 2024-03-14T09:01:47.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 -- start start 2024-03-14T09:01:47.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a68154700 0x7f0a681ea630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a681eab70 0x7f0a681331b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681eeb00 con 0x7f0a54004850 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681edd40 con 0x7f0a68154700 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.356+0000 7f0a6f43f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681ee020 con 0x7f0a681eab70 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59334/0 (socket says 172.21.15.154:59334) 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 -- 172.21.15.154:0/618031865 learned_addr learned my addr 172.21.15.154:0/618031865 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d9dc700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a681eab70 0x7f0a681331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a68154700 0x7f0a681ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 -- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a681eab70 msgr2=0x7f0a681331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a681eab70 0x7f0a681331b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 -- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a68154700 msgr2=0x7f0a681ea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a68154700 0x7f0a681ea630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 -- 172.21.15.154:0/618031865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a681ee300 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6d1db700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011aa0 tx=0x7f0a5c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c00f9e0 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c00fb40 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a681ee5e0 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c014440 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.357+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a68130bd0 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.358+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680a92f0 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.359+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0a5c0145a0 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.359+0000 7f0a59ffb700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 0x7f0a4c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.359+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a5c089540 con 0x7f0a54004850 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.359+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 0x7f0a4c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.360+0000 7f0a6c9da700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 0x7f0a4c06cd20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001eb0 tx=0x7f0a64009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.364+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0a5c0549f0 con 0x7f0a54004850 2024-03-14T09:01:47.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.557+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a680c4200 con 0x7f0a54004850 2024-03-14T09:01:47.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.558+0000 7f0a59ffb700 1 -- 172.21.15.154:0/618031865 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f0a5c058010 con 0x7f0a54004850 2024-03-14T09:01:47.558 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:47.558 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:47.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 msgr2=0x7f0a4c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 0x7f0a4c06cd20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001eb0 tx=0x7f0a64009030 comp rx=0 tx=0).stop 2024-03-14T09:01:47.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 msgr2=0x7f0a68131730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c011aa0 tx=0x7f0a5c011e60 comp rx=0 tx=0).stop 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 shutdown_connections 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a681eab70 0x7f0a681331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0a4c06a860 0x7f0a4c06cd20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a54004850 0x7f0a68131730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 --2- 172.21.15.154:0/618031865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a68154700 0x7f0a681ea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 >> 172.21.15.154:0/618031865 conn(0x7f0a680c3880 msgr2=0x7f0a681368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 shutdown_connections 2024-03-14T09:01:47.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.560+0000 7f0a6f43f700 1 -- 172.21.15.154:0/618031865 wait complete. 2024-03-14T09:01:47.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:47.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:47.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.699+0000 7ff0fb647700 1 Processor -- start 2024-03-14T09:01:47.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.699+0000 7ff0fb647700 1 -- start start 2024-03-14T09:01:47.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0fb647700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff0f415bfd0 0x7ff0f415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0fb647700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4156bf0 con 0x7ff0f40a9650 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0fb647700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4156050 con 0x7ff0f40c1780 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0fb647700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40c1c00 con 0x7ff0f415bfd0 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0f93e3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff0f40a9650 0x7ff0f41564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48528/0 (socket says 172.21.15.154:48528) 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0f93e3700 1 -- 172.21.15.154:0/3718899837 learned_addr learned my addr 172.21.15.154:0/3718899837 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/3718899837 >> v2:172.21.15.191:3300/0 conn(0x7ff0f415bfd0 0x7ff0f415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12588197 0 0) 0x7ff0e4001110 con 0x7ff0f40a9650 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.700+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e00035b0 con 0x7ff0f40a9650 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 352362121 0 0) 0x7ff0e00035b0 con 0x7ff0f40a9650 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 >> v2:172.21.15.191:3300/0 conn(0x7ff0f415bfd0 msgr2=0x7ff0f415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 --2- 172.21.15.154:0/3718899837 >> v2:172.21.15.191:3300/0 conn(0x7ff0f415bfd0 0x7ff0f415e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 >> v1:172.21.15.191:6789/0 conn(0x7ff0f40c1780 legacy=0x7ff0f406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40a86f0 con 0x7ff0f40a9650 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7ff0e4001800 con 0x7ff0f40a9650 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 >> v1:172.21.15.154:6789/0 conn(0x7ff0f40a9650 legacy=0x7ff0f41564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 0x7ff0e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0e0004850 0x7ff0e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0005330 0x7ff0e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40c1c00 con 0x7ff0e0003a00 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c200 con 0x7ff0e0004850 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000c3c0 con 0x7ff0e0005330 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 0x7ff0e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f8be2700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0e0004850 0x7ff0e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0005330 0x7ff0e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 -- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0005330 msgr2=0x7ff0e000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0005330 0x7ff0e000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 -- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0e0004850 msgr2=0x7ff0e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0e0004850 0x7ff0e0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 -- 172.21.15.154:0/3718899837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00035b0 con 0x7ff0e0003a00 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0005330 0x7ff0e000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f9be4700 1 -- 172.21.15.154:0/3718899837 reap_dead start 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.701+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 0x7ff0e0003e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c210 tx=0x7ff0e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fafe081e8e5038 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4007d80 con 0x7ff0e0003a00 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e400cd70 con 0x7ff0e0003a00 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0ebfff700 1 -- 172.21.15.154:0/3718899837 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4005ba0 con 0x7ff0e0003a00 2024-03-14T09:01:47.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 -- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 msgr2=0x7ff0e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 0x7ff0e0003e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c210 tx=0x7ff0e0005880 comp rx=0 tx=0).stop 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 -- 172.21.15.154:0/3718899837 shutdown_connections 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 --2- 172.21.15.154:0/3718899837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e0003a00 0x7ff0e0003e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 -- 172.21.15.154:0/3718899837 >> 172.21.15.154:0/3718899837 conn(0x7ff0f40c3880 msgr2=0x7ff0f40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 -- 172.21.15.154:0/3718899837 shutdown_connections 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.702+0000 7ff0fb647700 1 -- 172.21.15.154:0/3718899837 wait complete. 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.703+0000 7ff0fb647700 1 Processor -- start 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.703+0000 7ff0fb647700 1 -- start start 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0003a00 0x7ff0f413ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f40c65f0 0x7ff0f414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4143c10 con 0x7ff0f414bc20 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f413d740 con 0x7ff0f40c65f0 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0fb647700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f413f180 con 0x7ff0e0003a00 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59364/0 (socket says 172.21.15.154:59364) 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 -- 172.21.15.154:0/2653823805 learned_addr learned my addr 172.21.15.154:0/2653823805 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0003a00 0x7ff0f413ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f8be2700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f40c65f0 0x7ff0f414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 -- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0003a00 msgr2=0x7ff0f413ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0003a00 0x7ff0f413ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 -- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f40c65f0 msgr2=0x7ff0f414b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f40c65f0 0x7ff0f414b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.704+0000 7ff0f9be4700 1 -- 172.21.15.154:0/2653823805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4143870 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0f9be4700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40ad1f0 tx=0x7ff0ec0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec01b070 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec00f040 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00e050 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f41461f0 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.705+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f413e960 con 0x7ff0f414bc20 2024-03-14T09:01:47.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.706+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f4143d50 con 0x7ff0f414bc20 2024-03-14T09:01:47.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.707+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff0ec003a20 con 0x7ff0f414bc20 2024-03-14T09:01:47.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.707+0000 7ff0e9ffb700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 0x7ff0d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:47.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.707+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0ec089bd0 con 0x7ff0f414bc20 2024-03-14T09:01:47.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.708+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 0x7ff0d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:47.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.708+0000 7ff0f93e3700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 0x7ff0d806ccf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4000c00 tx=0x7ff0e400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:47.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.711+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff0ec055160 con 0x7ff0f414bc20 2024-03-14T09:01:47.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.906+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0f4143d50 con 0x7ff0f414bc20 2024-03-14T09:01:47.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.906+0000 7ff0e9ffb700 1 -- 172.21.15.154:0/2653823805 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0f4143d50 con 0x7ff0f414bc20 2024-03-14T09:01:47.906 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:47.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.908+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 msgr2=0x7ff0d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.908+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 0x7ff0d806ccf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4000c00 tx=0x7ff0e400a000 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.908+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 msgr2=0x7ff0f40c5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.908+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40ad1f0 tx=0x7ff0ec0104a0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 shutdown_connections 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0d806a830 0x7ff0d806ccf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f414bc20 0x7ff0f40c5b70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0e0003a00 0x7ff0f413ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 --2- 172.21.15.154:0/2653823805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f40c65f0 0x7ff0f414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 >> 172.21.15.154:0/2653823805 conn(0x7ff0f40c3880 msgr2=0x7ff0f40c1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 shutdown_connections 2024-03-14T09:01:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:47.909+0000 7ff0fb647700 1 -- 172.21.15.154:0/2653823805 wait complete. 2024-03-14T09:01:47.920 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:48.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.045+0000 7f7aae639700 1 Processor -- start 2024-03-14T09:01:48.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.045+0000 7f7aae639700 1 -- start start 2024-03-14T09:01:48.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aae639700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7aa80702f0 0x7f7aa80c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.047 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aae639700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7aa80c5d60 0x7f7aa80c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aae639700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa80acd10 con 0x7f7aa814fbd0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aae639700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa80c66b0 con 0x7f7aa80702f0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aae639700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa80ac7b0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7aa80702f0 0x7f7aa80c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7aa80702f0 0x7f7aa80c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59378/0 (socket says 172.21.15.154:59378) 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 -- 172.21.15.154:0/2394481329 learned_addr learned my addr 172.21.15.154:0/2394481329 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aacbd6700 1 --2- 172.21.15.154:0/2394481329 >> v2:172.21.15.191:3300/0 conn(0x7f7aa80c5d60 0x7f7aa80c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 -- 172.21.15.154:0/2394481329 >> v2:172.21.15.191:3300/0 conn(0x7f7aa80c5d60 msgr2=0x7f7aa80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 --2- 172.21.15.154:0/2394481329 >> v2:172.21.15.191:3300/0 conn(0x7f7aa80c5d60 0x7f7aa80c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 -- 172.21.15.154:0/2394481329 >> v1:172.21.15.191:6789/0 conn(0x7f7aa814fbd0 legacy=0x7f7aa80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 -- 172.21.15.154:0/2394481329 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8150050 con 0x7f7aa80702f0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa7fff700 1 --2- 172.21.15.154:0/2394481329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80702f0 0x7f7aa80c4380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80aac90 tx=0x7f7a9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9729aacc04742991 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa6ffd700 1 -- 172.21.15.154:0/2394481329 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004230 con 0x7f7aa80702f0 2024-03-14T09:01:48.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa6ffd700 1 -- 172.21.15.154:0/2394481329 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c004390 con 0x7f7aa80702f0 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.046+0000 7f7aa6ffd700 1 -- 172.21.15.154:0/2394481329 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0036a0 con 0x7f7aa80702f0 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 -- 172.21.15.154:0/2394481329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80702f0 msgr2=0x7f7aa80c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 --2- 172.21.15.154:0/2394481329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80702f0 0x7f7aa80c4380 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80aac90 tx=0x7f7a9c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 -- 172.21.15.154:0/2394481329 shutdown_connections 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 --2- 172.21.15.154:0/2394481329 >> v2:172.21.15.191:3300/0 conn(0x7f7aa80c5d60 0x7f7aa80c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 --2- 172.21.15.154:0/2394481329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80702f0 0x7f7aa80c4380 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 -- 172.21.15.154:0/2394481329 >> 172.21.15.154:0/2394481329 conn(0x7f7aa80a9bf0 msgr2=0x7f7aa80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 -- 172.21.15.154:0/2394481329 shutdown_connections 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.047+0000 7f7aae639700 1 -- 172.21.15.154:0/2394481329 wait complete. 2024-03-14T09:01:48.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.048+0000 7f7aae639700 1 Processor -- start 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.048+0000 7f7aae639700 1 -- start start 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa80702f0 0x7f7aa8148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa814fbd0 0x7f7aa8142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81508a0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81491c0 con 0x7f7aa80702f0 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aae639700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81edb10 con 0x7f7aa814fbd0 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59394/0 (socket says 172.21.15.154:59394) 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 -- 172.21.15.154:0/1765704850 learned_addr learned my addr 172.21.15.154:0/1765704850 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa7fff700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa80702f0 0x7f7aa8148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aacbd6700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa814fbd0 0x7f7aa8142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 -- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa814fbd0 msgr2=0x7f7aa8142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa814fbd0 0x7f7aa8142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 -- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa80702f0 msgr2=0x7f7aa8148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa80702f0 0x7f7aa8148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.049+0000 7f7aa77fe700 1 -- 172.21.15.154:0/1765704850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c009550 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aa77fe700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b470 tx=0x7f7a9800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98011830 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a980119b0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa81eddf0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98011d60 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.050+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81504c0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.051+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8148a80 con 0x7f7aa80c5d60 2024-03-14T09:01:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.051+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7a9801a400 con 0x7f7aa80c5d60 2024-03-14T09:01:48.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.052+0000 7f7aa4ff9700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 0x7f7a9406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.052+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a98088ca0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.052+0000 7f7aa7fff700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 0x7f7a9406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.052+0000 7f7aa7fff700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 0x7f7a9406ccf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009130 tx=0x7f7a9c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.056+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7a98054020 con 0x7f7aa80c5d60 2024-03-14T09:01:48.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.249+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7aa80697d0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.249+0000 7f7aa4ff9700 1 -- 172.21.15.154:0/1765704850 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a980150a0 con 0x7f7aa80c5d60 2024-03-14T09:01:48.249 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:48.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.251+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 msgr2=0x7f7a9406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.251+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 0x7f7a9406ccf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009130 tx=0x7f7a9c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.251+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 msgr2=0x7f7aa806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.251+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b470 tx=0x7f7a9800b780 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.251+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 shutdown_connections 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7aa814fbd0 0x7f7aa8142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7aa80702f0 0x7f7aa8148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7a9406a830 0x7f7a9406ccf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 --2- 172.21.15.154:0/1765704850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa80c5d60 0x7f7aa806c510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 >> 172.21.15.154:0/1765704850 conn(0x7f7aa80a9bf0 msgr2=0x7f7aa80c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 shutdown_connections 2024-03-14T09:01:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.252+0000 7f7aae639700 1 -- 172.21.15.154:0/1765704850 wait complete. 2024-03-14T09:01:48.263 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:48.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.388+0000 7fe86df18700 1 Processor -- start 2024-03-14T09:01:48.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.388+0000 7fe86df18700 1 -- start start 2024-03-14T09:01:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe86806acb0 0x7fe868155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe868156330 0x7fe868157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86806b310 con 0x7fe868156330 2024-03-14T09:01:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680ac8e0 con 0x7fe86806acb0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe86df18700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86806de30 con 0x7fe86814eaf0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe868156330 0x7fe868157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe868156330 0x7fe868157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59400/0 (socket says 172.21.15.154:59400) 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 -- 172.21.15.154:0/2345460444 learned_addr learned my addr 172.21.15.154:0/2345460444 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe8677fe700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe866ffd700 1 --2- 172.21.15.154:0/2345460444 >> v2:172.21.15.191:3300/0 conn(0x7fe86806acb0 0x7fe868155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 -- 172.21.15.154:0/2345460444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 msgr2=0x7fe86806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 -- 172.21.15.154:0/2345460444 >> v2:172.21.15.191:3300/0 conn(0x7fe86806acb0 msgr2=0x7fe868155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 --2- 172.21.15.154:0/2345460444 >> v2:172.21.15.191:3300/0 conn(0x7fe86806acb0 0x7fe868155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 -- 172.21.15.154:0/2345460444 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680acb30 con 0x7fe868156330 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.389+0000 7fe867fff700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe868156330 0x7fe868157ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe868070e20 tx=0x7fe86000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ca08d64cee15a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe8667fc700 1 -- 172.21.15.154:0/2345460444 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86000e070 con 0x7fe868156330 2024-03-14T09:01:48.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe8667fc700 1 -- 172.21.15.154:0/2345460444 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86000b9c0 con 0x7fe868156330 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe8667fc700 1 -- 172.21.15.154:0/2345460444 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86000bd70 con 0x7fe868156330 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 -- 172.21.15.154:0/2345460444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe868156330 msgr2=0x7fe868157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe868156330 0x7fe868157ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe868070e20 tx=0x7fe86000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 -- 172.21.15.154:0/2345460444 shutdown_connections 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe868156330 0x7fe868157ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 --2- 172.21.15.154:0/2345460444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 --2- 172.21.15.154:0/2345460444 >> v2:172.21.15.191:3300/0 conn(0x7fe86806acb0 0x7fe868155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 -- 172.21.15.154:0/2345460444 >> 172.21.15.154:0/2345460444 conn(0x7fe8680c3880 msgr2=0x7fe8680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 -- 172.21.15.154:0/2345460444 shutdown_connections 2024-03-14T09:01:48.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.390+0000 7fe86df18700 1 -- 172.21.15.154:0/2345460444 wait complete. 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.391+0000 7fe86df18700 1 Processor -- start 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 -- start start 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe868156330 0x7fe868144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868155090 con 0x7fe86806acb0 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe868154190 con 0x7fe868156330 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe86df18700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe868154ad0 con 0x7fe86814eaf0 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe8677fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe8677fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59402/0 (socket says 172.21.15.154:59402) 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.392+0000 7fe8677fe700 1 -- 172.21.15.154:0/895633451 learned_addr learned my addr 172.21.15.154:0/895633451 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe866ffd700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe867fff700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe868156330 0x7fe868144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 -- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 msgr2=0x7fe86814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86814f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 -- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe868156330 msgr2=0x7fe868144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe868156330 0x7fe868144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 -- 172.21.15.154:0/895633451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860009b30 con 0x7fe86806acb0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe8677fe700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d800 tx=0x7fe85000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8500125f0 con 0x7fe86806acb0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe850012750 con 0x7fe86806acb0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850012b00 con 0x7fe86806acb0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe868154db0 con 0x7fe86806acb0 2024-03-14T09:01:48.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.393+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe868150750 con 0x7fe86806acb0 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.394+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8680697d0 con 0x7fe86806acb0 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.394+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe850012c60 con 0x7fe86806acb0 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.395+0000 7fe85ffff700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 0x7fe84806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.395+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe850089110 con 0x7fe86806acb0 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.395+0000 7fe866ffd700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 0x7fe84806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.395+0000 7fe866ffd700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 0x7fe84806ccf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe868071c60 tx=0x7fe858008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.399+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe850054530 con 0x7fe86806acb0 2024-03-14T09:01:48.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.593+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe86806b470 con 0x7fe86806acb0 2024-03-14T09:01:48.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.594+0000 7fe85ffff700 1 -- 172.21.15.154:0/895633451 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe850057b50 con 0x7fe86806acb0 2024-03-14T09:01:48.594 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 msgr2=0x7fe84806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 0x7fe84806ccf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe868071c60 tx=0x7fe858008040 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 msgr2=0x7fe868150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d800 tx=0x7fe85000dbc0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 shutdown_connections 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe868156330 0x7fe868144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe84806a830 0x7fe84806ccf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe86806acb0 0x7fe868150210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 --2- 172.21.15.154:0/895633451 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe86814eaf0 0x7fe86814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 >> 172.21.15.154:0/895633451 conn(0x7fe8680c3880 msgr2=0x7fe86814c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 shutdown_connections 2024-03-14T09:01:48.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.596+0000 7fe86df18700 1 -- 172.21.15.154:0/895633451 wait complete. 2024-03-14T09:01:48.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:01:48.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.730+0000 7fb94a38a700 1 Processor -- start 2024-03-14T09:01:48.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.730+0000 7fb94a38a700 1 -- start start 2024-03-14T09:01:48.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb94a38a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9440a8540 0x7fb94413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb94a38a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440c1780 0x7fb9440c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb94a38a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94413ade0 con 0x7fb9440c3250 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb94a38a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94406fb30 con 0x7fb9440a8540 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb94a38a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94406ec30 con 0x7fb9440c1780 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb943fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9440a8540 0x7fb94413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb943fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9440a8540 0x7fb94413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59408/0 (socket says 172.21.15.154:59408) 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.731+0000 7fb943fff700 1 -- 172.21.15.154:0/172486395 learned_addr learned my addr 172.21.15.154:0/172486395 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb948927700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440c1780 0x7fb9440c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb943fff700 1 -- 172.21.15.154:0/172486395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440c1780 msgr2=0x7fb9440c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb943fff700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440c1780 0x7fb9440c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb943fff700 1 -- 172.21.15.154:0/172486395 >> v1:172.21.15.191:6789/0 conn(0x7fb9440c3250 legacy=0x7fb944154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb943fff700 1 -- 172.21.15.154:0/172486395 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94406ef10 con 0x7fb9440a8540 2024-03-14T09:01:48.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb943fff700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440a8540 0x7fb94413a8a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb93800bb90 tx=0x7fb93800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2746aca88503875 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb942ffd700 1 -- 172.21.15.154:0/172486395 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800dd10 con 0x7fb9440a8540 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb942ffd700 1 -- 172.21.15.154:0/172486395 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93800de70 con 0x7fb9440a8540 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.732+0000 7fb942ffd700 1 -- 172.21.15.154:0/172486395 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938004230 con 0x7fb9440a8540 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 -- 172.21.15.154:0/172486395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440a8540 msgr2=0x7fb94413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440a8540 0x7fb94413a8a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb93800bb90 tx=0x7fb93800bea0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 -- 172.21.15.154:0/172486395 shutdown_connections 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440c1780 0x7fb9440c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 --2- 172.21.15.154:0/172486395 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440a8540 0x7fb94413a8a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 -- 172.21.15.154:0/172486395 >> 172.21.15.154:0/172486395 conn(0x7fb9440c3880 msgr2=0x7fb9440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 -- 172.21.15.154:0/172486395 shutdown_connections 2024-03-14T09:01:48.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.733+0000 7fb94a38a700 1 -- 172.21.15.154:0/172486395 wait complete. 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.734+0000 7fb94a38a700 1 Processor -- start 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.734+0000 7fb94a38a700 1 -- start start 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440a8540 0x7fb9440aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9440c3250 0x7fb944154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440aae10 con 0x7fb9440c1780 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ab520 con 0x7fb9440c3250 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9441ee730 con 0x7fb9440a8540 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59422/0 (socket says 172.21.15.154:59422) 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 -- 172.21.15.154:0/3295534496 learned_addr learned my addr 172.21.15.154:0/3295534496 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb948927700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9440c3250 0x7fb944154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb943fff700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440a8540 0x7fb9440aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 -- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440a8540 msgr2=0x7fb9440aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440a8540 0x7fb9440aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 -- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9440c3250 msgr2=0x7fb944154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9440c3250 0x7fb944154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 -- 172.21.15.154:0/3295534496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93800b7f0 con 0x7fb9440c1780 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb9437fe700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b4a0 tx=0x7fb93400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011830 con 0x7fb9440c1780 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011990 con 0x7fb9440c1780 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94406e7b0 con 0x7fb9440c1780 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d40 con 0x7fb9440c1780 2024-03-14T09:01:48.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.735+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9441edc50 con 0x7fb9440c1780 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.736+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9440697d0 con 0x7fb9440c1780 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.737+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb93401a400 con 0x7fb9440c1780 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.737+0000 7fb940ff9700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 0x7fb93006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.737+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9340886b0 con 0x7fb9440c1780 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.738+0000 7fb943fff700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 0x7fb93006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:48.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.738+0000 7fb943fff700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 0x7fb93006ccf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb93800b340 tx=0x7fb93801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:48.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.742+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb934053b60 con 0x7fb9440c1780 2024-03-14T09:01:48.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.949+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb94406e7b0 con 0x7fb9440c1780 2024-03-14T09:01:48.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.949+0000 7fb940ff9700 1 -- 172.21.15.154:0/3295534496 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb94406e7b0 con 0x7fb9440c1780 2024-03-14T09:01:48.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 msgr2=0x7fb93006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 0x7fb93006ccf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb93800b340 tx=0x7fb93801c040 comp rx=0 tx=0).stop 2024-03-14T09:01:48.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 msgr2=0x7fb9440c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b4a0 tx=0x7fb93400b7b0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 shutdown_connections 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9440c3250 0x7fb944154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb9440a8540 0x7fb9440aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb93006a830 0x7fb93006ccf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 --2- 172.21.15.154:0/3295534496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9440c1780 0x7fb9440c0520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 >> 172.21.15.154:0/3295534496 conn(0x7fb9440c3880 msgr2=0x7fb9441dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 shutdown_connections 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:48.951+0000 7fb94a38a700 1 -- 172.21.15.154:0/3295534496 wait complete. 2024-03-14T09:01:48.952 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:01:48.963 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:01:48.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:48.963 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-14T09:01:49.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:01:49.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:01:49.005 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:01:49.074 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-14T09:01:49.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:49.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.196+0000 7fbfeba3e700 1 Processor -- start 2024-03-14T09:01:49.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.196+0000 7fbfeba3e700 1 -- start start 2024-03-14T09:01:49.198 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfeba3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 0x7fbfe40c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfeba3e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfe40c6680 0x7fbfe415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfeba3e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe40c7bb0 con 0x7fbfe406c670 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfeba3e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe40c3e00 con 0x7fbfe40c6680 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfeba3e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40c7e90 con 0x7fbfe40c31c0 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfe9fdb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfe40c6680 0x7fbfe415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfe97da700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbfe406c670 0x7fbfe406cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48536/0 (socket says 172.21.15.154:48536) 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfe9fdb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfe40c6680 0x7fbfe415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59436/0 (socket says 172.21.15.154:59436) 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfe97da700 1 -- 172.21.15.154:0/3511166195 learned_addr learned my addr 172.21.15.154:0/3511166195 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.197+0000 7fbfe8fd9700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 0x7fbfe40c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe9fdb700 1 -- 172.21.15.154:0/3511166195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 msgr2=0x7fbfe40c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe9fdb700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 0x7fbfe40c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe9fdb700 1 -- 172.21.15.154:0/3511166195 >> v1:172.21.15.154:6789/0 conn(0x7fbfe406c670 legacy=0x7fbfe406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe9fdb700 1 -- 172.21.15.154:0/3511166195 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40c8170 con 0x7fbfe40c6680 2024-03-14T09:01:49.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfdbfff700 1 -- 172.21.15.154:0/3511166195 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182371696 0 0) 0x7fbfe40c7bb0 con 0x7fbfe406c670 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe8fd9700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 0x7fbfe40c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfe9fdb700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe40c6680 0x7fbfe415ed60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009d90 tx=0x7fbfdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246bf5f74613270e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfdbfff700 1 -- 172.21.15.154:0/3511166195 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e070 con 0x7fbfe40c6680 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfdbfff700 1 -- 172.21.15.154:0/3511166195 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc00bbd0 con 0x7fbfe40c6680 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfdbfff700 1 -- 172.21.15.154:0/3511166195 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc012430 con 0x7fbfe40c6680 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfeba3e700 1 -- 172.21.15.154:0/3511166195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe40c6680 msgr2=0x7fbfe415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.198+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe40c6680 0x7fbfe415ed60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009d90 tx=0x7fbfdc00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 -- 172.21.15.154:0/3511166195 shutdown_connections 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe40c6680 0x7fbfe415ed60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/3511166195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c31c0 0x7fbfe40c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 -- 172.21.15.154:0/3511166195 >> 172.21.15.154:0/3511166195 conn(0x7fbfe40a9bf0 msgr2=0x7fbfe40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 -- 172.21.15.154:0/3511166195 shutdown_connections 2024-03-14T09:01:49.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.199+0000 7fbfeba3e700 1 -- 172.21.15.154:0/3511166195 wait complete. 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 Processor -- start 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 -- start start 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe40c31c0 0x7fbfe41e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c6680 0x7fbfe41df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41de1f0 con 0x7fbfe406c670 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41dbe30 con 0x7fbfe40c31c0 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.200+0000 7fbfeba3e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41db900 con 0x7fbfe40c6680 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59444/0 (socket says 172.21.15.154:59444) 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 -- 172.21.15.154:0/1573451887 learned_addr learned my addr 172.21.15.154:0/1573451887 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe9fdb700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c6680 0x7fbfe41df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe8fd9700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe40c31c0 0x7fbfe41e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 -- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c6680 msgr2=0x7fbfe41df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c6680 0x7fbfe41df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 -- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe40c31c0 msgr2=0x7fbfe41e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe40c31c0 0x7fbfe41e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 -- 172.21.15.154:0/1573451887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009a70 con 0x7fbfe406c670 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfe97da700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400c550 tx=0x7fbfd400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4004c80 con 0x7fbfe406c670 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4004e00 con 0x7fbfe406c670 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe41deeb0 con 0x7fbfe406c670 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4007310 con 0x7fbfe406c670 2024-03-14T09:01:49.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.201+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe41d7eb0 con 0x7fbfe406c670 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.202+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe40ae1e0 con 0x7fbfe406c670 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.203+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbfd4007470 con 0x7fbfe406c670 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.203+0000 7fbfd9ffb700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 0x7fbfc806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.204+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfd4088940 con 0x7fbfe406c670 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.204+0000 7fbfe8fd9700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 0x7fbfc806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.204+0000 7fbfe8fd9700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 0x7fbfc806ccf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe41e13b0 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.208+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbfd4053df0 con 0x7fbfe406c670 2024-03-14T09:01:49.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.447+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfe40c5a30 con 0x7fbfe406c670 2024-03-14T09:01:49.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.448+0000 7fbfd9ffb700 1 -- 172.21.15.154:0/1573451887 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fbfd4057410 con 0x7fbfe406c670 2024-03-14T09:01:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:49.447 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:49.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 msgr2=0x7fbfc806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 0x7fbfc806ccf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe41e13b0 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 msgr2=0x7fbfe41e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400c550 tx=0x7fbfd400c860 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 shutdown_connections 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbfe40c6680 0x7fbfe41df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbfc806a830 0x7fbfc806ccf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfe406c670 0x7fbfe41e9c50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 --2- 172.21.15.154:0/1573451887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfe40c31c0 0x7fbfe41e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 >> 172.21.15.154:0/1573451887 conn(0x7fbfe40a9bf0 msgr2=0x7fbfe406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 shutdown_connections 2024-03-14T09:01:49.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.450+0000 7fbfeba3e700 1 -- 172.21.15.154:0/1573451887 wait complete. 2024-03-14T09:01:49.451 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:49.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:49.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.583+0000 7fbb56f0d700 1 Processor -- start 2024-03-14T09:01:49.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.583+0000 7fbb56f0d700 1 -- start start 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb56f0d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb5014c5c0 0x7fbb50152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb56f0d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb56f0d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb50070e10 con 0x7fbb50072290 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb56f0d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500c1bb0 con 0x7fbb5014c5c0 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb56f0d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500c6170 con 0x7fbb50072710 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb54ca9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb5014c5c0 0x7fbb50152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb54ca9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb5014c5c0 0x7fbb50152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35120/0 (socket says 172.21.15.154:35120) 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2434854541 learned_addr learned my addr 172.21.15.154:0/2434854541 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.584+0000 7fbb4ffff700 1 --1- 172.21.15.154:0/2434854541 >> v1:172.21.15.154:6789/0 conn(0x7fbb50072290 0x7fbb500c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36308/0 (socket says 172.21.15.154:36308) 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb554aa700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2434854541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 msgr2=0x7fbb500c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb54ca9700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2434854541 >> v1:172.21.15.154:6789/0 conn(0x7fbb50072290 legacy=0x7fbb500c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2434854541 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5014c9a0 con 0x7fbb5014c5c0 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb4f7fe700 1 -- 172.21.15.154:0/2434854541 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148597807 0 0) 0x7fbb50070e10 con 0x7fbb50072290 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb54ca9700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb5014c5c0 0x7fbb50152f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb440098f0 tx=0x7fbb44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60779a77969f783d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb4f7fe700 1 -- 172.21.15.154:0/2434854541 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400b8a0 con 0x7fbb5014c5c0 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb4f7fe700 1 -- 172.21.15.154:0/2434854541 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4400ba00 con 0x7fbb5014c5c0 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb4f7fe700 1 -- 172.21.15.154:0/2434854541 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400bd90 con 0x7fbb5014c5c0 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2434854541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb5014c5c0 msgr2=0x7fbb50152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.585+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb5014c5c0 0x7fbb50152f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb440098f0 tx=0x7fbb44009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2434854541 shutdown_connections 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2434854541 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb5014c5c0 0x7fbb50152f80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2434854541 >> 172.21.15.154:0/2434854541 conn(0x7fbb5006a770 msgr2=0x7fbb5006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:49.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2434854541 shutdown_connections 2024-03-14T09:01:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.586+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2434854541 wait complete. 2024-03-14T09:01:49.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 Processor -- start 2024-03-14T09:01:49.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 -- start start 2024-03-14T09:01:49.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb5014c5c0 0x7fbb5014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501dc1d0 con 0x7fbb50072290 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501dbef0 con 0x7fbb5014c5c0 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.587+0000 7fbb56f0d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501eb1a0 con 0x7fbb50072710 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35130/0 (socket says 172.21.15.154:35130) 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2994218244 learned_addr learned my addr 172.21.15.154:0/2994218244 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb4ffff700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb554aa700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb5014c5c0 0x7fbb5014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 msgr2=0x7fbb500aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb5014c5c0 msgr2=0x7fbb5014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb5014c5c0 0x7fbb5014f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 -- 172.21.15.154:0/2994218244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009550 con 0x7fbb50072290 2024-03-14T09:01:49.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb54ca9700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009c40 tx=0x7fbb44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440045d0 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4401e070 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb501eb480 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440038e0 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.588+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb500abfc0 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.589+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb501eb000 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.590+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbb44003a40 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.590+0000 7fbb4d7fa700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 0x7fbb3406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.590+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb44020070 con 0x7fbb50072290 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.590+0000 7fbb4ffff700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 0x7fbb3406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.591+0000 7fbb4ffff700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 0x7fbb3406ccf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb5014e9d0 tx=0x7fbb3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.594+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbb501eb000 con 0x7fbb50072290 2024-03-14T09:01:49.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.792+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb501eb000 con 0x7fbb50072290 2024-03-14T09:01:49.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.793+0000 7fbb4d7fa700 1 -- 172.21.15.154:0/2994218244 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fbb501eb000 con 0x7fbb50072290 2024-03-14T09:01:49.793 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:49.793 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 msgr2=0x7fbb3406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 0x7fbb3406ccf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb5014e9d0 tx=0x7fbb3c009040 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 msgr2=0x7fbb501db910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009c40 tx=0x7fbb44004750 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 shutdown_connections 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb5014c5c0 0x7fbb5014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb3406a830 0x7fbb3406ccf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb50072290 0x7fbb501db910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 --2- 172.21.15.154:0/2994218244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb50072710 0x7fbb500aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 >> 172.21.15.154:0/2994218244 conn(0x7fbb5006a770 msgr2=0x7fbb500c2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 shutdown_connections 2024-03-14T09:01:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.796+0000 7fbb56f0d700 1 -- 172.21.15.154:0/2994218244 wait complete. 2024-03-14T09:01:49.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:01:17.135537+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:01:18.077046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:01:49.808 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:01:49.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.932+0000 7fbc531bb700 1 Processor -- start 2024-03-14T09:01:49.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.932+0000 7fbc531bb700 1 -- start start 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc531bb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbc4c0702f0 0x7fbc4c0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc531bb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 0x7fbc4c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc531bb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c0acd10 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc531bb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c0ac7b0 con 0x7fbc4c0702f0 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc531bb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c150050 con 0x7fbc4c14fbd0 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc50f57700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbc4c0702f0 0x7fbc4c0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc50f57700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbc4c0702f0 0x7fbc4c0c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35132/0 (socket says 172.21.15.154:35132) 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc50f57700 1 -- 172.21.15.154:0/688087755 learned_addr learned my addr 172.21.15.154:0/688087755 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc4bfff700 1 --2- 172.21.15.154:0/688087755 >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 0x7fbc4c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc50f57700 1 -- 172.21.15.154:0/688087755 >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 msgr2=0x7fbc4c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.933+0000 7fbc50f57700 1 --2- 172.21.15.154:0/688087755 >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 0x7fbc4c0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc50f57700 1 -- 172.21.15.154:0/688087755 >> v1:172.21.15.191:6789/0 conn(0x7fbc4c0c78c0 legacy=0x7fbc4c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc50f57700 1 -- 172.21.15.154:0/688087755 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0ac9c0 con 0x7fbc4c0702f0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc4bfff700 1 --2- 172.21.15.154:0/688087755 >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 0x7fbc4c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc50f57700 1 --2- 172.21.15.154:0/688087755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c0c4380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0c5980 tx=0x7fbc40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896b26618d9f552f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc4b7fe700 1 -- 172.21.15.154:0/688087755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40004230 con 0x7fbc4c0702f0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc4b7fe700 1 -- 172.21.15.154:0/688087755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc40004390 con 0x7fbc4c0702f0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc4b7fe700 1 -- 172.21.15.154:0/688087755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400036a0 con 0x7fbc4c0702f0 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 -- 172.21.15.154:0/688087755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0702f0 msgr2=0x7fbc4c0c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 --2- 172.21.15.154:0/688087755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c0c4380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0c5980 tx=0x7fbc40009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 -- 172.21.15.154:0/688087755 shutdown_connections 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 --2- 172.21.15.154:0/688087755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c0c4380 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 --2- 172.21.15.154:0/688087755 >> v2:172.21.15.191:3300/0 conn(0x7fbc4c14fbd0 0x7fbc4c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 -- 172.21.15.154:0/688087755 >> 172.21.15.154:0/688087755 conn(0x7fbc4c0a9bf0 msgr2=0x7fbc4c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:49.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 -- 172.21.15.154:0/688087755 shutdown_connections 2024-03-14T09:01:49.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.934+0000 7fbc531bb700 1 -- 172.21.15.154:0/688087755 wait complete. 2024-03-14T09:01:49.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.935+0000 7fbc531bb700 1 Processor -- start 2024-03-14T09:01:49.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 -- start start 2024-03-14T09:01:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc4c14fbd0 0x7fbc4c146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1452a0 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c145520 con 0x7fbc4c0702f0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc531bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c145800 con 0x7fbc4c14fbd0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35142/0 (socket says 172.21.15.154:35142) 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.936+0000 7fbc4bfff700 1 -- 172.21.15.154:0/2131083096 learned_addr learned my addr 172.21.15.154:0/2131083096 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 -- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc4c14fbd0 msgr2=0x7fbc4c146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc4c14fbd0 0x7fbc4c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 -- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc4c0702f0 msgr2=0x7fbc4c144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 -- 172.21.15.154:0/2131083096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40009550 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc4bfff700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c7f0 tx=0x7fbc3800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38011800 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc38011960 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38011d10 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c06df80 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.937+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c06e540 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.938+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c06e760 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.938+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbc38018360 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.939+0000 7fbc497fa700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 0x7fbc3006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.939+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc38088cf0 con 0x7fbc4c0c78c0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.939+0000 7fbc50f57700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 0x7fbc3006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.939+0000 7fbc50f57700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 0x7fbc3006ccf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0c8810 tx=0x7fbc40010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:49.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:49.943+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbc3808d070 con 0x7fbc4c0c78c0 2024-03-14T09:01:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.133+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc4c0697d0 con 0x7fbc4c0c78c0 2024-03-14T09:01:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.134+0000 7fbc497fa700 1 -- 172.21.15.154:0/2131083096 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc38057c30 con 0x7fbc4c0c78c0 2024-03-14T09:01:50.133 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 msgr2=0x7fbc3006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 0x7fbc3006ccf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0c8810 tx=0x7fbc40010000 comp rx=0 tx=0).stop 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 msgr2=0x7fbc4c06ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800c7f0 tx=0x7fbc3800cbb0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 shutdown_connections 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbc4c14fbd0 0x7fbc4c146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc4c0702f0 0x7fbc4c144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbc3006a830 0x7fbc3006ccf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 --2- 172.21.15.154:0/2131083096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc4c0c78c0 0x7fbc4c06ef20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 >> 172.21.15.154:0/2131083096 conn(0x7fbc4c0a9bf0 msgr2=0x7fbc4c148370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 shutdown_connections 2024-03-14T09:01:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.136+0000 7fbc531bb700 1 -- 172.21.15.154:0/2131083096 wait complete. 2024-03-14T09:01:50.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:01:50.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 Processor -- start 2024-03-14T09:01:50.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 -- start start 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf581515d0 0x7fdf5815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5806d940 con 0x7fdf5806c740 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580aa800 con 0x7fdf5814e550 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.272+0000 7fdf5d9e9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5814e9d0 con 0x7fdf581515d0 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf56ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdf5806c740 0x7fdf580c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36314/0 (socket says 172.21.15.154:36314) 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf56ffd700 1 -- 172.21.15.154:0/2106346747 learned_addr learned my addr 172.21.15.154:0/2106346747 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf577fe700 1 --2- 172.21.15.154:0/2106346747 >> v2:172.21.15.191:3300/0 conn(0x7fdf581515d0 0x7fdf5815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf55ffb700 1 -- 172.21.15.154:0/2106346747 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971707553 0 0) 0x7fdf5806d940 con 0x7fdf5806c740 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf55ffb700 1 -- 172.21.15.154:0/2106346747 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf440035b0 con 0x7fdf5806c740 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf577fe700 1 -- 172.21.15.154:0/2106346747 >> v1:172.21.15.191:6789/0 conn(0x7fdf5814e550 legacy=0x7fdf58150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf577fe700 1 -- 172.21.15.154:0/2106346747 >> v1:172.21.15.154:6789/0 conn(0x7fdf5806c740 legacy=0x7fdf580c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf577fe700 1 -- 172.21.15.154:0/2106346747 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5806cd30 con 0x7fdf581515d0 2024-03-14T09:01:50.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf577fe700 1 --2- 172.21.15.154:0/2106346747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf5815c550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88331da486f4164d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf55ffb700 1 -- 172.21.15.154:0/2106346747 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000e070 con 0x7fdf581515d0 2024-03-14T09:01:50.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf55ffb700 1 -- 172.21.15.154:0/2106346747 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000bb20 con 0x7fdf581515d0 2024-03-14T09:01:50.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.273+0000 7fdf55ffb700 1 -- 172.21.15.154:0/2106346747 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50012430 con 0x7fdf581515d0 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2106346747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 msgr2=0x7fdf5815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2106346747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf5815c550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009d90 tx=0x7fdf5000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2106346747 shutdown_connections 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2106346747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf5815c550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2106346747 >> 172.21.15.154:0/2106346747 conn(0x7fdf580c5e70 msgr2=0x7fdf580c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2106346747 shutdown_connections 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.274+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2106346747 wait complete. 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.275+0000 7fdf5d9e9700 1 Processor -- start 2024-03-14T09:01:50.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.275+0000 7fdf5d9e9700 1 -- start start 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5814e550 0x7fdf581eb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf581ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580bb310 con 0x7fdf5806c740 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580bffc0 con 0x7fdf581515d0 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf5d9e9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581413c0 con 0x7fdf5814e550 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35152/0 (socket says 172.21.15.154:35152) 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 -- 172.21.15.154:0/2370020857 learned_addr learned my addr 172.21.15.154:0/2370020857 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf577fe700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf581ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf567fc700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5814e550 0x7fdf581eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 -- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5814e550 msgr2=0x7fdf581eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5814e550 0x7fdf581eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 -- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 msgr2=0x7fdf581ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf581ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 -- 172.21.15.154:0/2370020857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009a70 con 0x7fdf5806c740 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.276+0000 7fdf56ffd700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf48001eb0 tx=0x7fdf4800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.277+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48005d70 con 0x7fdf5806c740 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.277+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf480070d0 con 0x7fdf5806c740 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.277+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf580c5a70 con 0x7fdf5806c740 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.277+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48007460 con 0x7fdf5806c740 2024-03-14T09:01:50.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.277+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf580baf30 con 0x7fdf5806c740 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.278+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf580bb450 con 0x7fdf5806c740 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.278+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdf480075c0 con 0x7fdf5806c740 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.278+0000 7fdf4ffff700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 0x7fdf3806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.278+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf48088150 con 0x7fdf5806c740 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.279+0000 7fdf567fc700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 0x7fdf3806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.279+0000 7fdf567fc700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 0x7fdf3806ccf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf58071540 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.283+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdf48053580 con 0x7fdf5806c740 2024-03-14T09:01:50.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.477+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf58073f10 con 0x7fdf5806c740 2024-03-14T09:01:50.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.477+0000 7fdf4ffff700 1 -- 172.21.15.154:0/2370020857 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf48056ba0 con 0x7fdf5806c740 2024-03-14T09:01:50.477 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 msgr2=0x7fdf3806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 0x7fdf3806ccf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf58071540 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 msgr2=0x7fdf58143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf48001eb0 tx=0x7fdf4800c7b0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 shutdown_connections 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf581515d0 0x7fdf581ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdf3806a830 0x7fdf3806ccf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf5806c740 0x7fdf58143ba0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 --2- 172.21.15.154:0/2370020857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf5814e550 0x7fdf581eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 >> 172.21.15.154:0/2370020857 conn(0x7fdf580c5e70 msgr2=0x7fdf580701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 shutdown_connections 2024-03-14T09:01:50.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.480+0000 7fdf5d9e9700 1 -- 172.21.15.154:0/2370020857 wait complete. 2024-03-14T09:01:50.492 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:01:50.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.617+0000 7fa6cf3ef700 1 Processor -- start 2024-03-14T09:01:50.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.617+0000 7fa6cf3ef700 1 -- start start 2024-03-14T09:01:50.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cf3ef700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa6c814eaf0 0x7fa6c806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cf3ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 0x7fa6c8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cf3ef700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80ac8e0 con 0x7fa6c8156330 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cf3ef700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c806b310 con 0x7fa6c814eaf0 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cf3ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c806de30 con 0x7fa6c806acb0 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd98c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa6c8156330 0x7fa6c8157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36324/0 (socket says 172.21.15.154:36324) 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd98c700 1 -- 172.21.15.154:0/3379695583 learned_addr learned my addr 172.21.15.154:0/3379695583 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/3379695583 >> v2:172.21.15.191:3300/0 conn(0x7fa6c814eaf0 0x7fa6c806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cc98a700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 0x7fa6c8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd18b700 1 -- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 msgr2=0x7fa6c8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 0x7fa6c8155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd18b700 1 -- 172.21.15.154:0/3379695583 >> v1:172.21.15.154:6789/0 conn(0x7fa6c8156330 legacy=0x7fa6c8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cd18b700 1 -- 172.21.15.154:0/3379695583 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80acb30 con 0x7fa6c814eaf0 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.618+0000 7fa6cc98a700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 0x7fa6c8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:50.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.619+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c814eaf0 0x7fa6c806a770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009840 tx=0x7fa6c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978cc3b7d6da222f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.620+0000 7fa6bffff700 1 -- 172.21.15.154:0/3379695583 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011830 con 0x7fa6c814eaf0 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.620+0000 7fa6bffff700 1 -- 172.21.15.154:0/3379695583 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4011990 con 0x7fa6c814eaf0 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.620+0000 7fa6bffff700 1 -- 172.21.15.154:0/3379695583 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011d20 con 0x7fa6c814eaf0 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.620+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c814eaf0 msgr2=0x7fa6c806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.620+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c814eaf0 0x7fa6c806a770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009840 tx=0x7fa6c4009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/3379695583 shutdown_connections 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c814eaf0 0x7fa6c806a770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/3379695583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c806acb0 0x7fa6c8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/3379695583 >> 172.21.15.154:0/3379695583 conn(0x7fa6c80c3880 msgr2=0x7fa6c80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/3379695583 shutdown_connections 2024-03-14T09:01:50.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.621+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/3379695583 wait complete. 2024-03-14T09:01:50.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.622+0000 7fa6cf3ef700 1 Processor -- start 2024-03-14T09:01:50.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.622+0000 7fa6cf3ef700 1 -- start start 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c806acb0 0x7fa6c8148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c8156330 0x7fa6c8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c81515e0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8144e20 con 0x7fa6c806acb0 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cf3ef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c81492d0 con 0x7fa6c8156330 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35166/0 (socket says 172.21.15.154:35166) 2024-03-14T09:01:50.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 -- 172.21.15.154:0/1971286312 learned_addr learned my addr 172.21.15.154:0/1971286312 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c806acb0 0x7fa6c8148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cd98c700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c8156330 0x7fa6c8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 -- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c8156330 msgr2=0x7fa6c8153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c8156330 0x7fa6c8153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 -- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c806acb0 msgr2=0x7fa6c8148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c806acb0 0x7fa6c8148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.623+0000 7fa6cc98a700 1 -- 172.21.15.154:0/1971286312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80094a0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6cc98a700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800ebe0 tx=0x7fa6b800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8009760 con 0x7fa6c814eaf0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b800c850 con 0x7fa6c814eaf0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40094a0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800cbe0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.624+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c814a630 con 0x7fa6c814eaf0 2024-03-14T09:01:50.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.625+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8067970 con 0x7fa6c814eaf0 2024-03-14T09:01:50.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.625+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa6b800cd40 con 0x7fa6c814eaf0 2024-03-14T09:01:50.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.626+0000 7fa6bdffb700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 0x7fa6ac06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.626+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6b8088ca0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.626+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 0x7fa6ac06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.627+0000 7fa6cd18b700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 0x7fa6ac06ccf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000c00 tx=0x7fa6c401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.630+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa6b8054150 con 0x7fa6c814eaf0 2024-03-14T09:01:50.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.825+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6c8067dc0 con 0x7fa6c814eaf0 2024-03-14T09:01:50.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.825+0000 7fa6bdffb700 1 -- 172.21.15.154:0/1971286312 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6b8057770 con 0x7fa6c814eaf0 2024-03-14T09:01:50.825 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 msgr2=0x7fa6ac06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 0x7fa6ac06ccf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000c00 tx=0x7fa6c401c040 comp rx=0 tx=0).stop 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 msgr2=0x7fa6c81eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800ebe0 tx=0x7fa6b800eef0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 shutdown_connections 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6c8156330 0x7fa6c8153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6c806acb0 0x7fa6c8148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6ac06a830 0x7fa6ac06ccf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 --2- 172.21.15.154:0/1971286312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6c814eaf0 0x7fa6c81eb080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 >> 172.21.15.154:0/1971286312 conn(0x7fa6c80c3880 msgr2=0x7fa6c81574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 shutdown_connections 2024-03-14T09:01:50.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.827+0000 7fa6cf3ef700 1 -- 172.21.15.154:0/1971286312 wait complete. 2024-03-14T09:01:50.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:50.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.960+0000 7fd628a42700 1 Processor -- start 2024-03-14T09:01:50.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.960+0000 7fd628a42700 1 -- start start 2024-03-14T09:01:50.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.961+0000 7fd628a42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd6240c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.961+0000 7fd628a42700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd624151860 con 0x7fd6240c1d90 2024-03-14T09:01:50.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.961+0000 7fd628a42700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd624150f30 con 0x7fd6240c1830 2024-03-14T09:01:50.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.961+0000 7fd628a42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd624151210 con 0x7fd62406d3c0 2024-03-14T09:01:50.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd621d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd6240c1d90 0x7fd6240c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36328/0 (socket says 172.21.15.154:36328) 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd621d9b700 1 -- 172.21.15.154:0/1246391867 learned_addr learned my addr 172.21.15.154:0/1246391867 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62259c700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd6240c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087316055 0 0) 0x7fd624151860 con 0x7fd6240c1d90 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6100035b0 con 0x7fd6240c1d90 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028124458 0 0) 0x7fd6100035b0 con 0x7fd6240c1d90 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 msgr2=0x7fd6240c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd6240c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 >> v1:172.21.15.191:6789/0 conn(0x7fd6240c1830 legacy=0x7fd62415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6241514f0 con 0x7fd6240c1d90 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fd60c001670 con 0x7fd6240c1d90 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 >> v1:172.21.15.154:6789/0 conn(0x7fd6240c1d90 legacy=0x7fd6240c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610003a30 0x7fd610003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd610004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd610004d90 0x7fd610005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624151210 con 0x7fd610004d90 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61000bac0 con 0x7fd610003a30 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61000bc80 con 0x7fd62406d3c0 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd622d9d700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd610004d90 0x7fd610005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd62259c700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610003a30 0x7fd610003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.962+0000 7fd621d9b700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd610004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 -- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 msgr2=0x7fd610004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd62406d3c0 0x7fd610004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 -- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610003a30 msgr2=0x7fd610003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610003a30 0x7fd610003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 -- 172.21.15.154:0/1246391867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100035b0 con 0x7fd610004d90 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd610004d90 0x7fd610005210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001ed0 tx=0x7fd610005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563d10eead529125 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00cc70 con 0x7fd610004d90 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c00cdd0 con 0x7fd610004d90 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd62159a700 1 -- 172.21.15.154:0/1246391867 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c003c40 con 0x7fd610004d90 2024-03-14T09:01:50.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd628a42700 1 -- 172.21.15.154:0/1246391867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd610004d90 msgr2=0x7fd610005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd628a42700 1 --2- 172.21.15.154:0/1246391867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd610004d90 0x7fd610005210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001ed0 tx=0x7fd610005880 comp rx=0 tx=0).stop 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd622d9d700 1 -- 172.21.15.154:0/1246391867 reap_dead start 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.963+0000 7fd628a42700 1 -- 172.21.15.154:0/1246391867 shutdown_connections 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.964+0000 7fd628a42700 1 -- 172.21.15.154:0/1246391867 >> 172.21.15.154:0/1246391867 conn(0x7fd62406a770 msgr2=0x7fd62406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.964+0000 7fd628a42700 1 -- 172.21.15.154:0/1246391867 shutdown_connections 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.964+0000 7fd628a42700 1 -- 172.21.15.154:0/1246391867 wait complete. 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 Processor -- start 2024-03-14T09:01:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 -- start start 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 0x7fd6241e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 0x7fd62414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6241e6d50 con 0x7fd6240ac0b0 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240a9ba0 con 0x7fd610004d90 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.965+0000 7fd628a42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240a9e80 con 0x7fd6241e7200 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35188/0 (socket says 172.21.15.154:35188) 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 -- 172.21.15.154:0/3725010881 learned_addr learned my addr 172.21.15.154:0/3725010881 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd62259c700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 0x7fd6241e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd621d9b700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 0x7fd62414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 -- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 msgr2=0x7fd62414fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 0x7fd62414fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 -- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 msgr2=0x7fd6241e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 0x7fd6241e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 -- 172.21.15.154:0/3725010881 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240aa160 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd62259c700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 0x7fd6241e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd621d9b700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 0x7fd62414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd622d9d700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6100047b0 tx=0x7fd61c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c001240 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c0052e0 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c005670 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240ab8b0 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.966+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624150250 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.967+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6240697d0 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.968+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd61c023070 con 0x7fd6240ac0b0 2024-03-14T09:01:50.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.968+0000 7fd61affd700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 0x7fd60406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:50.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.968+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd61c08c3d0 con 0x7fd6240ac0b0 2024-03-14T09:01:50.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.968+0000 7fd62259c700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 0x7fd60406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:50.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.969+0000 7fd62259c700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 0x7fd60406ccf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6240c0bd0 tx=0x7fd614009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:50.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:50.972+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd61c057880 con 0x7fd6240ac0b0 2024-03-14T09:01:51.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.216+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62406af60 con 0x7fd6240ac0b0 2024-03-14T09:01:51.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.216+0000 7fd61affd700 1 -- 172.21.15.154:0/3725010881 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7fd61c05aea0 con 0x7fd6240ac0b0 2024-03-14T09:01:51.216 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:51.216 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:51.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 msgr2=0x7fd60406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 0x7fd60406ccf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6240c0bd0 tx=0x7fd614009210 comp rx=0 tx=0).stop 2024-03-14T09:01:51.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 msgr2=0x7fd6240ac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6100047b0 tx=0x7fd61c0014a0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 shutdown_connections 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd60406a830 0x7fd60406ccf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6240ac0b0 0x7fd6240ac540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd610004d90 0x7fd6241e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 --2- 172.21.15.154:0/3725010881 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6241e7200 0x7fd62414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 >> 172.21.15.154:0/3725010881 conn(0x7fd62406a770 msgr2=0x7fd62406b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 shutdown_connections 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.218+0000 7fd628a42700 1 -- 172.21.15.154:0/3725010881 wait complete. 2024-03-14T09:01:51.219 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:51.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:51.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 Processor -- start 2024-03-14T09:01:51.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 -- start start 2024-03-14T09:01:51.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0bc0158a80 0x7f0bc00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc014ed60 con 0x7f0bc00c4a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc00c6cf0 con 0x7f0bc014e980 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bc573f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc00c6b10 con 0x7f0bc0158a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bbe7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0bc00c4a80 0x7f0bc0158370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36334/0 (socket says 172.21.15.154:36334) 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.352+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3104936340 learned_addr learned my addr 172.21.15.154:0/3104936340 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565079823 0 0) 0x7f0bc014ed60 con 0x7f0bc00c4a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbf7fe700 1 --2- 172.21.15.154:0/3104936340 >> v2:172.21.15.191:3300/0 conn(0x7f0bc0158a80 0x7f0bc00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba80035b0 con 0x7f0bc00c4a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795988303 0 0) 0x7f0ba80035b0 con 0x7f0bc00c4a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 >> v2:172.21.15.191:3300/0 conn(0x7f0bc0158a80 msgr2=0x7f0bc00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 --2- 172.21.15.154:0/3104936340 >> v2:172.21.15.191:3300/0 conn(0x7f0bc0158a80 0x7f0bc00c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 >> v1:172.21.15.191:6789/0 conn(0x7f0bc014e980 legacy=0x7f0bc01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00c6fd0 con 0x7f0bc00c4a80 2024-03-14T09:01:51.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0bb4001670 con 0x7f0bc00c4a80 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 >> v1:172.21.15.154:6789/0 conn(0x7f0bc00c4a80 legacy=0x7f0bc0158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a00 0x7f0ba8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 0x7f0ba8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8005210 0x7f0ba800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00c6b10 con 0x7f0ba8004850 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba800c1b0 con 0x7f0ba8003a00 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba800c370 con 0x7f0ba8005210 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 0x7f0ba8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbeffd700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a00 0x7f0ba8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbf7fe700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8005210 0x7f0ba800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8005210 msgr2=0x7f0ba800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8005210 0x7f0ba800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a00 msgr2=0x7f0ba8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a00 0x7f0ba8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.353+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3104936340 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80035b0 con 0x7f0ba8004850 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbf7fe700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8005210 0x7f0ba800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbf7fe700 1 -- 172.21.15.154:0/3104936340 reap_dead start 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 0x7f0ba8004cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c2a0 tx=0x7f0ba8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c61bee318f1b228 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4007c10 con 0x7f0ba8004850 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400cdc0 con 0x7f0ba8004850 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bbdffb700 1 -- 172.21.15.154:0/3104936340 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4005ba0 con 0x7f0ba8004850 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 -- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 msgr2=0x7f0ba8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 0x7f0ba8004cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c2a0 tx=0x7f0ba8005880 comp rx=0 tx=0).stop 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 -- 172.21.15.154:0/3104936340 shutdown_connections 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3104936340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8004850 0x7f0ba8004cd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 -- 172.21.15.154:0/3104936340 >> 172.21.15.154:0/3104936340 conn(0x7f0bc00abcf0 msgr2=0x7f0bc00ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 -- 172.21.15.154:0/3104936340 shutdown_connections 2024-03-14T09:01:51.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.354+0000 7f0bc573f700 1 -- 172.21.15.154:0/3104936340 wait complete. 2024-03-14T09:01:51.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 Processor -- start 2024-03-14T09:01:51.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 -- start start 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8004850 0x7f0bc00bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc00accc0 0x7f0bc01470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01ee0a0 con 0x7f0bc0155a40 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01ee380 con 0x7f0bc00accc0 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bc573f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00c0660 con 0x7f0ba8004850 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.356+0000 7f0bbe7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35208/0 (socket says 172.21.15.154:35208) 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3163417597 learned_addr learned my addr 172.21.15.154:0/3163417597 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:51.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbf7fe700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc00accc0 0x7f0bc01470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbeffd700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8004850 0x7f0bc00bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8004850 msgr2=0x7f0bc00bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8004850 0x7f0bc00bfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc00accc0 msgr2=0x7f0bc01470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc00accc0 0x7f0bc01470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 -- 172.21.15.154:0/3163417597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00c0940 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bbe7fc700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800c180 tx=0x7f0bc00c5ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4001640 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb40070d0 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc00bef40 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40074a0 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.357+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0152030 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.358+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0152220 con 0x7f0bc0155a40 2024-03-14T09:01:51.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.359+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0bb4007600 con 0x7f0bc0155a40 2024-03-14T09:01:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.359+0000 7f0ba77fe700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 0x7f0bac06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.359+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bb408a020 con 0x7f0bc0155a40 2024-03-14T09:01:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.359+0000 7f0bbeffd700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 0x7f0bac06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.360+0000 7f0bbeffd700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 0x7f0bac06ccf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0007fd0 tx=0x7f0bb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.364+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0bc0152220 con 0x7f0bc0155a40 2024-03-14T09:01:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.559+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bc00c04c0 con 0x7f0bc0155a40 2024-03-14T09:01:51.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.560+0000 7f0ba77fe700 1 -- 172.21.15.154:0/3163417597 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7f0bc00c04c0 con 0x7f0bc0155a40 2024-03-14T09:01:51.559 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:51.560 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 msgr2=0x7f0bac06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 0x7f0bac06ccf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0007fd0 tx=0x7f0bb0009040 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 msgr2=0x7f0bc00ab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800c180 tx=0x7f0bc00c5ff0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 shutdown_connections 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc00accc0 0x7f0bc01470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ba8004850 0x7f0bc00bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0bac06a830 0x7f0bac06ccf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 --2- 172.21.15.154:0/3163417597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc0155a40 0x7f0bc00ab260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 >> 172.21.15.154:0/3163417597 conn(0x7f0bc00abcf0 msgr2=0x7f0bc00c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 shutdown_connections 2024-03-14T09:01:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.562+0000 7f0bc573f700 1 -- 172.21.15.154:0/3163417597 wait complete. 2024-03-14T09:01:51.574 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T09:01:51.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:51.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 Processor -- start 2024-03-14T09:01:51.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 -- start start 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f28740702f0 0x7f28740c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28740acd10 con 0x7f287414fbd0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2874150050 con 0x7f28740c5d60 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2879b37700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28740ac7b0 con 0x7f28740702f0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2872ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f287414fbd0 0x7f28740c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36350/0 (socket says 172.21.15.154:36350) 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.697+0000 7f2872ffd700 1 -- 172.21.15.154:0/1141367264 learned_addr learned my addr 172.21.15.154:0/1141367264 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849390609 0 0) 0x7f28740acd10 con 0x7f287414fbd0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f285c0035b0 con 0x7f287414fbd0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 --2- 172.21.15.154:0/1141367264 >> v2:172.21.15.191:3300/0 conn(0x7f28740702f0 0x7f28740c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1656053235 0 0) 0x7f285c0035b0 con 0x7f287414fbd0 2024-03-14T09:01:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 >> v2:172.21.15.191:3300/0 conn(0x7f28740702f0 msgr2=0x7f28740c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 --2- 172.21.15.154:0/1141367264 >> v2:172.21.15.191:3300/0 conn(0x7f28740702f0 0x7f28740c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 >> v1:172.21.15.191:6789/0 conn(0x7f28740c5d60 legacy=0x7f287415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740ac9c0 con 0x7f287414fbd0 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2868001670 con 0x7f287414fbd0 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 >> v1:172.21.15.154:6789/0 conn(0x7f287414fbd0 legacy=0x7f28740c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f285c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f285c004850 0x7f285c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f285c005210 0x7f285c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740ac7b0 con 0x7f285c003a00 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c00c0b0 con 0x7f285c005210 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c00c270 con 0x7f285c004850 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f285c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f2872ffd700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f285c004850 0x7f285c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f2873fff700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f285c005210 0x7f285c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 -- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f285c004850 msgr2=0x7f285c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f285c004850 0x7f285c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 -- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f285c005210 msgr2=0x7f285c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f285c005210 0x7f285c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f2873fff700 1 -- 172.21.15.154:0/1141367264 reap_dead start 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f28737fe700 1 -- 172.21.15.154:0/1141367264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c0035b0 con 0x7f285c003a00 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.698+0000 7f2872ffd700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f285c004850 0x7f285c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f28737fe700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f285c003e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2864009b60 tx=0x7f285c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ac2ee6edc61687 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286400e6c0 con 0x7f285c003a00 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286400e820 con 0x7f285c003a00 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f28727fc700 1 -- 172.21.15.154:0/1141367264 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286400eb90 con 0x7f285c003a00 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 -- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 msgr2=0x7f285c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f285c003e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2864009b60 tx=0x7f285c005880 comp rx=0 tx=0).stop 2024-03-14T09:01:51.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 -- 172.21.15.154:0/1141367264 shutdown_connections 2024-03-14T09:01:51.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 --2- 172.21.15.154:0/1141367264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f285c003e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 -- 172.21.15.154:0/1141367264 >> 172.21.15.154:0/1141367264 conn(0x7f28740a9bf0 msgr2=0x7f28740a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:51.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 -- 172.21.15.154:0/1141367264 shutdown_connections 2024-03-14T09:01:51.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.699+0000 7f2879b37700 1 -- 172.21.15.154:0/1141367264 wait complete. 2024-03-14T09:01:51.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.701+0000 7f2879b37700 1 Processor -- start 2024-03-14T09:01:51.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.701+0000 7f2879b37700 1 -- start start 2024-03-14T09:01:51.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 0x7f28741eb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28741eb940 0x7f2874147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287414a700 con 0x7f285c003a00 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740a9a90 con 0x7f287413f3e0 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2879b37700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28741508b0 con 0x7f28741eb940 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35224/0 (socket says 172.21.15.154:35224) 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 -- 172.21.15.154:0/3081025641 learned_addr learned my addr 172.21.15.154:0/3081025641 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2872ffd700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 0x7f28741eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2873fff700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28741eb940 0x7f2874147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 -- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28741eb940 msgr2=0x7f2874147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28741eb940 0x7f2874147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 -- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 msgr2=0x7f28741eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 0x7f28741eb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f28737fe700 1 -- 172.21.15.154:0/3081025641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874147470 con 0x7f285c003a00 2024-03-14T09:01:51.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.702+0000 7f2872ffd700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 0x7f28741eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f28737fe700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c080 tx=0x7f28640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286400e6c0 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864003ad0 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28741ec140 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864003e80 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.703+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2874150da0 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.704+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28740697d0 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.705+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2864013470 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.705+0000 7f285bfff700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 0x7f286006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.705+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f286408ad70 con 0x7f285c003a00 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.706+0000 7f2872ffd700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 0x7f286006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:51.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.706+0000 7f2872ffd700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 0x7f286006ccf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2868001c90 tx=0x7f2868009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:51.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.709+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2864056200 con 0x7f285c003a00 2024-03-14T09:01:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.947+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28740c7160 con 0x7f285c003a00 2024-03-14T09:01:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.948+0000 7f285bfff700 1 -- 172.21.15.154:0/3081025641 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7f2864017090 con 0x7f285c003a00 2024-03-14T09:01:51.947 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:51.947 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:51.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.949+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 msgr2=0x7f286006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.949+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 0x7f286006ccf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2868001c90 tx=0x7f2868009040 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 msgr2=0x7f28740a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f285c00c080 tx=0x7f28640047c0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 shutdown_connections 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28741eb940 0x7f2874147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f286006a830 0x7f286006ccf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285c003a00 0x7f28740a9550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 --2- 172.21.15.154:0/3081025641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f287413f3e0 0x7f28741eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 >> 172.21.15.154:0/3081025641 conn(0x7f28740a9bf0 msgr2=0x7f28740a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 shutdown_connections 2024-03-14T09:01:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:51.950+0000 7f2879b37700 1 -- 172.21.15.154:0/3081025641 wait complete. 2024-03-14T09:01:51.951 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:51.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:01:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.083+0000 7f75b51a6700 1 Processor -- start 2024-03-14T09:01:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.083+0000 7f75b51a6700 1 -- start start 2024-03-14T09:01:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75b51a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75b014e550 0x7f75b0150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75b51a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b0151400 0x7f75b015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75b51a6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b006d940 con 0x7f75b006c740 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75b51a6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b014e9d0 con 0x7f75b014e550 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75b51a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00aa800 con 0x7f75b0151400 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75ae59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75b014e550 0x7f75b0150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75aed9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f75b006c740 0x7f75b00c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36356/0 (socket says 172.21.15.154:36356) 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75aed9d700 1 -- 172.21.15.154:0/197712749 learned_addr learned my addr 172.21.15.154:0/197712749 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75af59e700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b0151400 0x7f75b015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75ae59c700 1 -- 172.21.15.154:0/197712749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b0151400 msgr2=0x7f75b015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75ae59c700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b0151400 0x7f75b015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75ae59c700 1 -- 172.21.15.154:0/197712749 >> v1:172.21.15.154:6789/0 conn(0x7f75b006c740 legacy=0x7f75b00c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75ae59c700 1 -- 172.21.15.154:0/197712749 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b006cd30 con 0x7f75b014e550 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.084+0000 7f75add9b700 1 -- 172.21.15.154:0/197712749 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442561347 0 0) 0x7f75b006d940 con 0x7f75b006c740 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75ae59c700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b0150ec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75a000a9e0 tx=0x7f75a000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce25b5c6e318d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75add9b700 1 -- 172.21.15.154:0/197712749 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cb70 con 0x7f75b014e550 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75add9b700 1 -- 172.21.15.154:0/197712749 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000ccd0 con 0x7f75b014e550 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75add9b700 1 -- 172.21.15.154:0/197712749 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0018620 con 0x7f75b014e550 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 -- 172.21.15.154:0/197712749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 msgr2=0x7f75b0150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b0150ec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75a000a9e0 tx=0x7f75a000acf0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 -- 172.21.15.154:0/197712749 shutdown_connections 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b0151400 0x7f75b015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 --2- 172.21.15.154:0/197712749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b0150ec0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 -- 172.21.15.154:0/197712749 >> 172.21.15.154:0/197712749 conn(0x7f75b00c5e70 msgr2=0x7f75b00c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 -- 172.21.15.154:0/197712749 shutdown_connections 2024-03-14T09:01:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.085+0000 7f75b51a6700 1 -- 172.21.15.154:0/197712749 wait complete. 2024-03-14T09:01:52.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.087+0000 7f75b51a6700 1 Processor -- start 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.087+0000 7f75b51a6700 1 -- start start 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b006c740 0x7f75b014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75b0151400 0x7f75b014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01e9f10 con 0x7f75b014e550 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0153cb0 con 0x7f75b0151400 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01dcf50 con 0x7f75b006c740 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35242/0 (socket says 172.21.15.154:35242) 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 -- 172.21.15.154:0/1261306372 learned_addr learned my addr 172.21.15.154:0/1261306372 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 -- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b006c740 msgr2=0x7f75b014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b006c740 0x7f75b014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 -- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75b0151400 msgr2=0x7f75b014b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75b0151400 0x7f75b014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 -- 172.21.15.154:0/1261306372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a000a640 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75ae59c700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75a000a1c0 tx=0x7f75a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0003b70 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0003cd0 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b01dbbd0 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00216a0 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.088+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b01ea100 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.089+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0073f10 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.090+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f75a0018400 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.090+0000 7f75a77fe700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 0x7f759006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.090+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75a008a390 con 0x7f75b014e550 2024-03-14T09:01:52.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.090+0000 7f75aed9d700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 0x7f759006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.091+0000 7f75aed9d700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 0x7f759006ccf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7598001610 tx=0x7f75980079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.095+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f75a0055840 con 0x7f75b014e550 2024-03-14T09:01:52.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.333+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b00aa5b0 con 0x7f75b014e550 2024-03-14T09:01:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.333+0000 7f75a77fe700 1 -- 172.21.15.154:0/1261306372 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4737 (secure 0 0 0) 0x7f75a0058e60 con 0x7f75b014e550 2024-03-14T09:01:52.333 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:52.333 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1438901536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1438901536},{"type":"v1","addr":"172.21.15.191:6837","nonce":1438901536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/853041480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":853041480},{"type":"v1","addr":"172.21.15.154:6835","nonce":853041480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3063700329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3063700329},{"type":"v1","addr":"172.21.15.154:6834","nonce":3063700329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:01:19.106939+0000","modified":"2024-03-14T09:01:22.301841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1086778085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1086778085},{"type":"v1","addr":"172.21.15.191:6835","nonce":1086778085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-14T09:01:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 msgr2=0x7f759006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 0x7f759006ccf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7598001610 tx=0x7f75980079f0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 msgr2=0x7f75b01ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75a000a1c0 tx=0x7f75a00047c0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 shutdown_connections 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75b0151400 0x7f75b014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75b006c740 0x7f75b014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f759006a830 0x7f759006ccf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 --2- 172.21.15.154:0/1261306372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75b014e550 0x7f75b01ea620 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.335+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 >> 172.21.15.154:0/1261306372 conn(0x7f75b00c5e70 msgr2=0x7f75b00c0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.336+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 shutdown_connections 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.336+0000 7f75b51a6700 1 -- 172.21.15.154:0/1261306372 wait complete. 2024-03-14T09:01:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 8 2024-03-14T09:01:52.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:01:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.469+0000 7fe2578f6700 1 Processor -- start 2024-03-14T09:01:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 -- start start 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe25014eaf0 0x7fe25006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe250155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2500ac8e0 con 0x7fe250156330 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25006b310 con 0x7fe25014eaf0 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe2578f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25006de30 con 0x7fe25006acb0 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.470+0000 7fe255e93700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe250156330 0x7fe250157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36358/0 (socket says 172.21.15.154:36358) 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255e93700 1 -- 172.21.15.154:0/3519635098 learned_addr learned my addr 172.21.15.154:0/3519635098 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 --2- 172.21.15.154:0/3519635098 >> v2:172.21.15.191:3300/0 conn(0x7fe25014eaf0 0x7fe25006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472044392 0 0) 0x7fe2500ac8e0 con 0x7fe250156330 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe254e91700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe250155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe23c0035b0 con 0x7fe250156330 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594571901 0 0) 0x7fe23c0035b0 con 0x7fe250156330 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 msgr2=0x7fe250155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe250155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 >> v2:172.21.15.191:3300/0 conn(0x7fe25014eaf0 msgr2=0x7fe25006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 --2- 172.21.15.154:0/3519635098 >> v2:172.21.15.191:3300/0 conn(0x7fe25014eaf0 0x7fe25006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500acb30 con 0x7fe250156330 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fe2480019f0 con 0x7fe250156330 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 >> v1:172.21.15.154:6789/0 conn(0x7fe250156330 legacy=0x7fe250157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe23c003a00 0x7fe23c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe23c004850 0x7fe23c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe23c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25006b310 con 0x7fe23c003a00 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25006de30 con 0x7fe23c004850 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c00ba90 con 0x7fe25006acb0 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe23c003a00 0x7fe23c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe254e91700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe23c004850 0x7fe23c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255e93700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe23c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 -- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 msgr2=0x7fe23c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe23c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 -- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe23c004850 msgr2=0x7fe23c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.471+0000 7fe255692700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe23c004850 0x7fe23c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe255692700 1 -- 172.21.15.154:0/3519635098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0035b0 con 0x7fe23c003a00 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe255e93700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe25006acb0 0x7fe23c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe255692700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe23c003a00 0x7fe23c003e20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe244009b60 tx=0x7fe23c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4718f74d53c532c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24400e6c0 con 0x7fe23c003a00 2024-03-14T09:01:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24400e820 con 0x7fe23c003a00 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe243fff700 1 -- 172.21.15.154:0/3519635098 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24400eb90 con 0x7fe23c003a00 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 -- 172.21.15.154:0/3519635098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe23c003a00 msgr2=0x7fe23c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 --2- 172.21.15.154:0/3519635098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe23c003a00 0x7fe23c003e20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe244009b60 tx=0x7fe23c005880 comp rx=0 tx=0).stop 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe255e93700 1 -- 172.21.15.154:0/3519635098 reap_dead start 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 -- 172.21.15.154:0/3519635098 shutdown_connections 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 -- 172.21.15.154:0/3519635098 >> 172.21.15.154:0/3519635098 conn(0x7fe2500c3880 msgr2=0x7fe2500c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 -- 172.21.15.154:0/3519635098 shutdown_connections 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.472+0000 7fe2578f6700 1 -- 172.21.15.154:0/3519635098 wait complete. 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.473+0000 7fe2578f6700 1 Processor -- start 2024-03-14T09:01:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 -- start start 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe23c003a00 0x7fe2500c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2500c1470 0x7fe2500c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2500c18f0 con 0x7fe250141a60 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25014d880 con 0x7fe2500c1470 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe2578f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25014ddb0 con 0x7fe23c003a00 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe255e93700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe255e93700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35274/0 (socket says 172.21.15.154:35274) 2024-03-14T09:01:52.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe255e93700 1 -- 172.21.15.154:0/4222857767 learned_addr learned my addr 172.21.15.154:0/4222857767 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe255692700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe23c003a00 0x7fe2500c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.474+0000 7fe254e91700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2500c1470 0x7fe2500c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 -- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe23c003a00 msgr2=0x7fe2500c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe23c003a00 0x7fe2500c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 -- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2500c1470 msgr2=0x7fe2500c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2500c1470 0x7fe2500c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 -- 172.21.15.154:0/4222857767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25014dad0 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe255e93700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe248000f80 tx=0x7fe24800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248017070 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24800eb50 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25014c980 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800ed20 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.475+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25014cf40 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.476+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25014d100 con 0x7fe250141a60 2024-03-14T09:01:52.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.476+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe248013420 con 0x7fe250141a60 2024-03-14T09:01:52.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.477+0000 7fe241ffb700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 0x7fe23406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.477+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe248088860 con 0x7fe250141a60 2024-03-14T09:01:52.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.477+0000 7fe255692700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 0x7fe23406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.478+0000 7fe255692700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 0x7fe23406cd20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2440090b0 tx=0x7fe244008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.481+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe248053c80 con 0x7fe250141a60 2024-03-14T09:01:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.680+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2501433f0 con 0x7fe250141a60 2024-03-14T09:01:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.681+0000 7fe241ffb700 1 -- 172.21.15.154:0/4222857767 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16085 (secure 0 0 0) 0x7fe2480572a0 con 0x7fe250141a60 2024-03-14T09:01:52.681 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:52.681 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:20.108768+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:01:17.135537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-14T09:01:18.077046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:01:52.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 msgr2=0x7fe23406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 0x7fe23406cd20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2440090b0 tx=0x7fe244008030 comp rx=0 tx=0).stop 2024-03-14T09:01:52.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 msgr2=0x7fe250141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe248000f80 tx=0x7fe24800e440 comp rx=0 tx=0).stop 2024-03-14T09:01:52.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 shutdown_connections 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe23406a860 0x7fe23406cd20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe250141a60 0x7fe250141ee0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe23c003a00 0x7fe2500c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 --2- 172.21.15.154:0/4222857767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2500c1470 0x7fe2500c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 >> 172.21.15.154:0/4222857767 conn(0x7fe2500c3880 msgr2=0x7fe2500c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 shutdown_connections 2024-03-14T09:01:52.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.683+0000 7fe2578f6700 1 -- 172.21.15.154:0/4222857767 wait complete. 2024-03-14T09:01:52.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T09:01:52.818 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.818+0000 7f1f50c19700 1 Processor -- start 2024-03-14T09:01:52.818 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.818+0000 7f1f50c19700 1 -- start start 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f50c19700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f50c19700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0c5d60 0x7f1f4c0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f50c19700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c0acd10 con 0x7f1f4c0702f0 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f50c19700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c150110 con 0x7f1f4c0c4380 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f50c19700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c66b0 con 0x7f1f4c0c5d60 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.819+0000 7f1f49d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35290/0 (socket says 172.21.15.154:35290) 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 -- 172.21.15.154:0/2111230925 learned_addr learned my addr 172.21.15.154:0/2111230925 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f4a59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1f4c0702f0 0x7f1f4c0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36366/0 (socket says 172.21.15.154:36366) 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f4ad9d700 1 --2- 172.21.15.154:0/2111230925 >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0c5d60 0x7f1f4c0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 -- 172.21.15.154:0/2111230925 >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0c5d60 msgr2=0x7f1f4c0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/2111230925 >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0c5d60 0x7f1f4c0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 -- 172.21.15.154:0/2111230925 >> v1:172.21.15.154:6789/0 conn(0x7f1f4c0702f0 legacy=0x7f1f4c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 -- 172.21.15.154:0/2111230925 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0ac9c0 con 0x7f1f4c0c4380 2024-03-14T09:01:52.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/2111230925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0aac90 tx=0x7f1f3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb3b06a9d3a2f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f4959a700 1 -- 172.21.15.154:0/2111230925 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00b870 con 0x7f1f4c0c4380 2024-03-14T09:01:52.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f4959a700 1 -- 172.21.15.154:0/2111230925 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00b9d0 con 0x7f1f4c0c4380 2024-03-14T09:01:52.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.820+0000 7f1f4959a700 1 -- 172.21.15.154:0/2111230925 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00bd60 con 0x7f1f4c0c4380 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 -- 172.21.15.154:0/2111230925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0c4380 msgr2=0x7f1f4c14fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 --2- 172.21.15.154:0/2111230925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0aac90 tx=0x7f1f3c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 -- 172.21.15.154:0/2111230925 shutdown_connections 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 --2- 172.21.15.154:0/2111230925 >> v2:172.21.15.191:3300/0 conn(0x7f1f4c0c5d60 0x7f1f4c0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 --2- 172.21.15.154:0/2111230925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c14fbd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 -- 172.21.15.154:0/2111230925 >> 172.21.15.154:0/2111230925 conn(0x7f1f4c0a9bf0 msgr2=0x7f1f4c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:52.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 -- 172.21.15.154:0/2111230925 shutdown_connections 2024-03-14T09:01:52.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.821+0000 7f1f50c19700 1 -- 172.21.15.154:0/2111230925 wait complete. 2024-03-14T09:01:52.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.822+0000 7f1f50c19700 1 Processor -- start 2024-03-14T09:01:52.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.822+0000 7f1f50c19700 1 -- start start 2024-03-14T09:01:52.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c1dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0c5d60 0x7f1f4c1dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c148fc0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c3160 con 0x7f1f4c0c4380 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f50c19700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c33b0 con 0x7f1f4c0c5d60 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f4a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f4a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35298/0 (socket says 172.21.15.154:35298) 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.823+0000 7f1f4a59c700 1 -- 172.21.15.154:0/1576829646 learned_addr learned my addr 172.21.15.154:0/1576829646 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c1dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 -- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0c5d60 msgr2=0x7f1f4c1dc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0c5d60 0x7f1f4c1dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 -- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 msgr2=0x7f1f4c1dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c1dbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 -- 172.21.15.154:0/1576829646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c009550 con 0x7f1f4c0702f0 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c1dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f4a59c700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c490 tx=0x7f1f4000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40005cb0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f40005e10 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c1ebef0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400072d0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.824+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c14b8a0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.825+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c1ec410 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.825+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1f40007430 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.826+0000 7f1f3affd700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 0x7f1f2c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.826+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f40087fd0 con 0x7f1f4c0702f0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.826+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 0x7f1f2c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:52.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.827+0000 7f1f49d9b700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 0x7f1f2c06cd20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c14df70 tx=0x7f1f3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:52.829 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:52.830+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1f40053530 con 0x7f1f4c0702f0 2024-03-14T09:01:53.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.070+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1f4c0697d0 con 0x7f1f4c0702f0 2024-03-14T09:01:53.070 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:01:53.071+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:01:53.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.081+0000 7f1f3affd700 1 -- 172.21.15.154:0/1576829646 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1f40056b50 con 0x7f1f4c0702f0 2024-03-14T09:01:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 msgr2=0x7f1f2c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 0x7f1f2c06cd20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c14df70 tx=0x7f1f3c008030 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 msgr2=0x7f1f4c14b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c490 tx=0x7f1f4000c850 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 shutdown_connections 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f4c0c5d60 0x7f1f4c1dc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1f2c06a860 0x7f1f2c06cd20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0702f0 0x7f1f4c14b360 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 --2- 172.21.15.154:0/1576829646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f4c0c4380 0x7f1f4c1dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.083+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 >> 172.21.15.154:0/1576829646 conn(0x7f1f4c0a9bf0 msgr2=0x7f1f4c14d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.084+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 shutdown_connections 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.084+0000 7f1f50c19700 1 -- 172.21.15.154:0/1576829646 wait complete. 2024-03-14T09:01:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:01:53.095 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T09:01:53.123 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:01:53.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.220+0000 7f3dab009700 1 Processor -- start 2024-03-14T09:01:53.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.220+0000 7f3dab009700 1 -- start start 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3dab009700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3da40702f0 0x7f3da40c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3dab009700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da40acd10 con 0x7f3da414fbd0 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3dab009700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da4150050 con 0x7f3da40c5d60 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3dab009700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da40ac7b0 con 0x7f3da40702f0 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3da3fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3da414fbd0 0x7f3da40c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36368/0 (socket says 172.21.15.154:36368) 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3da3fff700 1 -- 172.21.15.154:0/3570044947 learned_addr learned my addr 172.21.15.154:0/3570044947 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:53.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3da8da5700 1 --2- 172.21.15.154:0/3570044947 >> v2:172.21.15.191:3300/0 conn(0x7f3da40702f0 0x7f3da40c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.221+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141016041 0 0) 0x7f3da40acd10 con 0x7f3da414fbd0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d940035b0 con 0x7f3da414fbd0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368025472 0 0) 0x7f3d940035b0 con 0x7f3da414fbd0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 >> v2:172.21.15.191:3300/0 conn(0x7f3da40702f0 msgr2=0x7f3da40c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 --2- 172.21.15.154:0/3570044947 >> v2:172.21.15.191:3300/0 conn(0x7f3da40702f0 0x7f3da40c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 >> v1:172.21.15.191:6789/0 conn(0x7f3da40c5d60 legacy=0x7f3da415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40ac9c0 con 0x7f3da414fbd0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f3d90001670 con 0x7f3da414fbd0 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 >> v1:172.21.15.154:6789/0 conn(0x7f3da414fbd0 legacy=0x7f3da40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94003a00 0x7f3d94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d94004850 0x7f3d94004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 0x7f3d9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40ac7b0 con 0x7f3d94005210 2024-03-14T09:01:53.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9400c0b0 con 0x7f3d94004850 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9400c270 con 0x7f3d94003a00 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 0x7f3d9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da8da5700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94003a00 0x7f3d94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da3fff700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d94004850 0x7f3d94004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 -- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94003a00 msgr2=0x7f3d94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94003a00 0x7f3d94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 -- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d94004850 msgr2=0x7f3d94004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d94004850 0x7f3d94004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 -- 172.21.15.154:0/3570044947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940035b0 con 0x7f3d94005210 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 -- 172.21.15.154:0/3570044947 reap_dead start 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.222+0000 7f3da95a6700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 0x7f3d9400ba10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c240 tx=0x7f3d94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea90ada9d6e052 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c001df0 con 0x7f3d94005210 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00cdc0 con 0x7f3d94005210 2024-03-14T09:01:53.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3da37fe700 1 -- 172.21.15.154:0/3570044947 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c003c40 con 0x7f3d94005210 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 -- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 msgr2=0x7f3d9400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 0x7f3d9400ba10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00c240 tx=0x7f3d94005880 comp rx=0 tx=0).stop 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 -- 172.21.15.154:0/3570044947 shutdown_connections 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 --2- 172.21.15.154:0/3570044947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d94005210 0x7f3d9400ba10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 -- 172.21.15.154:0/3570044947 >> 172.21.15.154:0/3570044947 conn(0x7f3da40a9bf0 msgr2=0x7f3da40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 -- 172.21.15.154:0/3570044947 shutdown_connections 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.223+0000 7f3dab009700 1 -- 172.21.15.154:0/3570044947 wait complete. 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.224+0000 7f3dab009700 1 Processor -- start 2024-03-14T09:01:53.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.224+0000 7f3dab009700 1 -- start start 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94005210 0x7f3da40c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3da414d340 0x7f3da41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4150320 con 0x7f3da40c1d10 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4150940 con 0x7f3da414d340 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3dab009700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4154b30 con 0x7f3d94005210 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35320/0 (socket says 172.21.15.154:35320) 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3da3fff700 1 -- 172.21.15.154:0/2446946102 learned_addr learned my addr 172.21.15.154:0/2446946102 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3da8da5700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94005210 0x7f3da40c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.225+0000 7f3da95a6700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3da414d340 0x7f3da41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 -- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94005210 msgr2=0x7f3da40c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94005210 0x7f3da40c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 -- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3da414d340 msgr2=0x7f3da41530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3da414d340 0x7f3da41530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 -- 172.21.15.154:0/2446946102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da41e9ec0 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da3fff700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d900072c0 tx=0x7f3d900075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000f660 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9000f7e0 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da41ea1a0 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000fb90 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.226+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da41ea760 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.227+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da41ed160 con 0x7f3da40c1d10 2024-03-14T09:01:53.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.227+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3d9000fcf0 con 0x7f3da40c1d10 2024-03-14T09:01:53.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.228+0000 7f3da17fa700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 0x7f3d8806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:53.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.228+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3d90088bd0 con 0x7f3da40c1d10 2024-03-14T09:01:53.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.228+0000 7f3da8da5700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 0x7f3d8806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:53.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.228+0000 7f3da8da5700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 0x7f3d8806ccf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3da41e79b0 tx=0x7f3d98009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:53.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.232+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3da41ed160 con 0x7f3da40c1d10 2024-03-14T09:01:53.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:53.469+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3da41ed160 con 0x7f3da40c1d10 2024-03-14T09:01:54.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.086+0000 7f3da17fa700 1 -- 172.21.15.154:0/2446946102 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3da41ed160 con 0x7f3da40c1d10 2024-03-14T09:01:54.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.088+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 msgr2=0x7f3d8806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.088+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 0x7f3d8806ccf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3da41e79b0 tx=0x7f3d98009210 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.088+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 msgr2=0x7f3da4152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.088+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d900072c0 tx=0x7f3d900075d0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 shutdown_connections 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3da414d340 0x7f3da41530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3d94005210 0x7f3da40c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3d8806a830 0x7f3d8806ccf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 --2- 172.21.15.154:0/2446946102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da40c1d10 0x7f3da4152b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 >> 172.21.15.154:0/2446946102 conn(0x7f3da40a9bf0 msgr2=0x7f3da40c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 shutdown_connections 2024-03-14T09:01:54.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.089+0000 7f3dab009700 1 -- 172.21.15.154:0/2446946102 wait complete. 2024-03-14T09:01:54.103 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T09:01:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.235+0000 7f5e0bb9a700 1 Processor -- start 2024-03-14T09:01:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.235+0000 7f5e0bb9a700 1 -- start start 2024-03-14T09:01:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0bb9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 0x7f5e041532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0bb9a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5e0406d920 0x7f5e04148c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0bb9a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040ac1f0 con 0x7f5e04066970 2024-03-14T09:01:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0bb9a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040ac4b0 con 0x7f5e0406d920 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0bb9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e041527c0 con 0x7f5e040c1030 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0a137700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5e0406d920 0x7f5e04148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e09936700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 0x7f5e041532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e09135700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f5e04066970 0x7f5e0406b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:51500/0 (socket says 172.21.15.154:51500) 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e09135700 1 -- 172.21.15.154:0/114364406 learned_addr learned my addr 172.21.15.154:0/114364406 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0a137700 1 -- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 msgr2=0x7f5e041532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0a137700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 0x7f5e041532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0a137700 1 -- 172.21.15.154:0/114364406 >> v1:172.21.15.191:6789/0 conn(0x7f5e04066970 legacy=0x7f5e0406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e0a137700 1 -- 172.21.15.154:0/114364406 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04152aa0 con 0x7f5e0406d920 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.236+0000 7f5e09936700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 0x7f5e041532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0a137700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e0406d920 0x7f5e04148c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215bfb62b63ab77d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e08934700 1 -- 172.21.15.154:0/114364406 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00e070 con 0x7f5e0406d920 2024-03-14T09:01:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e08934700 1 -- 172.21.15.154:0/114364406 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc00bb20 con 0x7f5e0406d920 2024-03-14T09:01:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e08934700 1 -- 172.21.15.154:0/114364406 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc012430 con 0x7f5e0406d920 2024-03-14T09:01:54.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e0406d920 msgr2=0x7f5e04148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e0406d920 0x7f5e04148c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009d90 tx=0x7f5dfc00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:54.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/114364406 shutdown_connections 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e0406d920 0x7f5e04148c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/114364406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e040c1030 0x7f5e041532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/114364406 >> 172.21.15.154:0/114364406 conn(0x7f5e040c3130 msgr2=0x7f5e040c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/114364406 shutdown_connections 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.237+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/114364406 wait complete. 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.238+0000 7f5e0bb9a700 1 Processor -- start 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 -- start start 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e04066970 0x7f5e0406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e0406d920 0x7f5e041e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c0050 con 0x7f5e040c1030 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04149110 con 0x7f5e04066970 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0bb9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0413ed60 con 0x7f5e0406d920 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0a137700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0a137700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35332/0 (socket says 172.21.15.154:35332) 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e0a137700 1 -- 172.21.15.154:0/2191301770 learned_addr learned my addr 172.21.15.154:0/2191301770 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e09135700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e0406d920 0x7f5e041e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.239+0000 7f5e09936700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e04066970 0x7f5e0406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 -- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e0406d920 msgr2=0x7f5e041e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e0406d920 0x7f5e041e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 -- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e04066970 msgr2=0x7f5e0406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e04066970 0x7f5e0406ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 -- 172.21.15.154:0/2191301770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df40094a0 con 0x7f5e040c1030 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0a137700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc014040 tx=0x7f5dfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc012430 con 0x7f5e040c1030 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc020070 con 0x7f5e040c1030 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc009a70 con 0x7f5e040c1030 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc003ea0 con 0x7f5e040c1030 2024-03-14T09:01:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.240+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e04072990 con 0x7f5e040c1030 2024-03-14T09:01:54.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.241+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e041ede60 con 0x7f5e040c1030 2024-03-14T09:01:54.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.241+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5dfc004040 con 0x7f5e040c1030 2024-03-14T09:01:54.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.242+0000 7f5dfa7fc700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 0x7f5de806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:54.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.242+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5dfc022070 con 0x7f5e040c1030 2024-03-14T09:01:54.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.242+0000 7f5e09936700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 0x7f5de806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:54.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.243+0000 7f5e09936700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 0x7f5de806ccf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e0414fef0 tx=0x7f5df4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.246+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5e041ede60 con 0x7f5e040c1030 2024-03-14T09:01:54.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:54.438+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e041ede60 con 0x7f5e040c1030 2024-03-14T09:01:55.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.088+0000 7f5dfa7fc700 1 -- 172.21.15.154:0/2191301770 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5e041ede60 con 0x7f5e040c1030 2024-03-14T09:01:55.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 msgr2=0x7f5de806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 0x7f5de806ccf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e0414fef0 tx=0x7f5df4009210 comp rx=0 tx=0).stop 2024-03-14T09:01:55.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 msgr2=0x7f5e041ea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc014040 tx=0x7f5dfc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 shutdown_connections 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5de806a830 0x7f5de806ccf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e040c1030 0x7f5e041ea420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e04066970 0x7f5e0406ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 --2- 172.21.15.154:0/2191301770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e0406d920 0x7f5e041e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 >> 172.21.15.154:0/2191301770 conn(0x7f5e040c3130 msgr2=0x7f5e04151640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 shutdown_connections 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.093+0000 7f5e0bb9a700 1 -- 172.21.15.154:0/2191301770 wait complete. 2024-03-14T09:01:55.094 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2024-03-14T09:01:55.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T09:01:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.243+0000 7fa1edaaf700 1 Processor -- start 2024-03-14T09:01:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.243+0000 7fa1edaaf700 1 -- start start 2024-03-14T09:01:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1edaaf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1e806d050 0x7fa1e8158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1edaaf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 0x7fa1e80c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1edaaf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e8158c90 con 0x7fa1e81554e0 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1edaaf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e806b110 con 0x7fa1e806d050 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1edaaf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e806ab50 con 0x7fa1e8155060 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1e806d050 0x7fa1e8158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1e806d050 0x7fa1e8158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35348/0 (socket says 172.21.15.154:35348) 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 -- 172.21.15.154:0/3206424009 learned_addr learned my addr 172.21.15.154:0/3206424009 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e7fff700 1 --1- 172.21.15.154:0/3206424009 >> v1:172.21.15.154:6789/0 conn(0x7fa1e81554e0 0x7fa1e80c18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36374/0 (socket says 172.21.15.154:36374) 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e6ffd700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 0x7fa1e80c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 -- 172.21.15.154:0/3206424009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 msgr2=0x7fa1e80c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 0x7fa1e80c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 -- 172.21.15.154:0/3206424009 >> v1:172.21.15.154:6789/0 conn(0x7fa1e81554e0 legacy=0x7fa1e80c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 -- 172.21.15.154:0/3206424009 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e806ae30 con 0x7fa1e806d050 2024-03-14T09:01:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e67fc700 1 -- 172.21.15.154:0/3206424009 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647555172 0 0) 0x7fa1e8158c90 con 0x7fa1e81554e0 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e6ffd700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 0x7fa1e80c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.244+0000 7fa1e77fe700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e806d050 0x7fa1e8158750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009e50 tx=0x7fa1d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea0424a7c4102c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1e67fc700 1 -- 172.21.15.154:0/3206424009 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800e070 con 0x7fa1e806d050 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1e67fc700 1 -- 172.21.15.154:0/3206424009 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d800b9c0 con 0x7fa1e806d050 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1e67fc700 1 -- 172.21.15.154:0/3206424009 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800bd70 con 0x7fa1e806d050 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3206424009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e806d050 msgr2=0x7fa1e8158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e806d050 0x7fa1e8158750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009e50 tx=0x7fa1d800b330 comp rx=0 tx=0).stop 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3206424009 shutdown_connections 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e806d050 0x7fa1e8158750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3206424009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e8155060 0x7fa1e80c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3206424009 >> 172.21.15.154:0/3206424009 conn(0x7fa1e80c39f0 msgr2=0x7fa1e80c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3206424009 shutdown_connections 2024-03-14T09:01:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.245+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3206424009 wait complete. 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.246+0000 7fa1edaaf700 1 Processor -- start 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 -- start start 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806d050 0x7fa1e81429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 0x7fa1e80c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8143770 con 0x7fa1e81554e0 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8141770 con 0x7fa1e8155060 2024-03-14T09:01:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1edaaf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8142ef0 con 0x7fa1e806d050 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1e7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1e7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35350/0 (socket says 172.21.15.154:35350) 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1e7fff700 1 -- 172.21.15.154:0/3942480640 learned_addr learned my addr 172.21.15.154:0/3942480640 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1e6ffd700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 0x7fa1e80c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.247+0000 7fa1e77fe700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806d050 0x7fa1e81429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 -- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806d050 msgr2=0x7fa1e81429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806d050 0x7fa1e81429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 -- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 msgr2=0x7fa1e80c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 0x7fa1e80c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 -- 172.21.15.154:0/3942480640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009b30 con 0x7fa1e81554e0 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e6ffd700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 0x7fa1e80c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1e7fff700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000c410 tx=0x7fa1e000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004c80 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e0004de0 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0007310 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8141a10 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.248+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8141fd0 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.249+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8066920 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.249+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa1e0007470 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.250+0000 7fa1dffff700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 0x7fa1c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.250+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa1e00887d0 con 0x7fa1e81554e0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.250+0000 7fa1e77fe700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 0x7fa1c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.251+0000 7fa1e77fe700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 0x7fa1c806ccf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b370 tx=0x7fa1d8009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.254+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa1e0053ea0 con 0x7fa1e81554e0 2024-03-14T09:01:55.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:55.448+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1e80697d0 con 0x7fa1e81554e0 2024-03-14T09:01:55.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:01:55.926 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:01:56.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.093+0000 7fa1dffff700 1 -- 172.21.15.154:0/3942480640 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa1e00574c0 con 0x7fa1e81554e0 2024-03-14T09:01:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 msgr2=0x7fa1c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 0x7fa1c806ccf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b370 tx=0x7fa1d8009000 comp rx=0 tx=0).stop 2024-03-14T09:01:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 msgr2=0x7fa1e8143c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000c410 tx=0x7fa1e000c7d0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 shutdown_connections 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa1c806a830 0x7fa1c806ccf0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b370 tx=0x7fa1d8009000 comp rx=0 tx=0).stop 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1e81554e0 0x7fa1e8143c00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806d050 0x7fa1e81429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 --2- 172.21.15.154:0/3942480640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e8155060 0x7fa1e80c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.098+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 >> 172.21.15.154:0/3942480640 conn(0x7fa1e80c39f0 msgr2=0x7fa1e806e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.099+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 shutdown_connections 2024-03-14T09:01:56.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.099+0000 7fa1edaaf700 1 -- 172.21.15.154:0/3942480640 wait complete. 2024-03-14T09:01:56.100 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2024-03-14T09:01:56.114 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T09:01:56.114 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T09:01:56.114 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T09:01:56.114 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:01:56.115 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T09:01:56.115 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T09:01:56.115 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T09:01:56.115 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:01:56.115 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T09:01:56.115 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T09:01:56.115 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T09:01:56.115 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:01:56.115 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T09:01:56.116 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T09:01:56.116 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T09:01:56.116 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:01:56.124 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T09:01:56.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T09:01:56.125 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T09:01:56.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T09:01:56.127 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T09:01:56.127 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T09:01:56.128 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T09:01:56.128 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T09:01:56.168 INFO:tasks.ceph.mds.a:Started 2024-03-14T09:01:56.169 INFO:tasks.ceph.mds.c:Started 2024-03-14T09:01:56.169 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:56.169 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:56.171 INFO:tasks.ceph.mds.b:Started 2024-03-14T09:01:56.171 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:56.172 INFO:tasks.ceph.mds.d:Started 2024-03-14T09:01:56.172 DEBUG:teuthology.parallel:result is None 2024-03-14T09:01:56.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T09:01:56.248 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:01:56.249 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:01:56.257 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-14T09:01:56.257 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:01:56.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.306+0000 7f93fe3f3700 1 Processor -- start 2024-03-14T09:01:56.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.306+0000 7f93fe3f3700 1 -- start start 2024-03-14T09:01:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fe3f3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f80c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fe3f3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f80c5f10 con 0x7f93f806c6e0 2024-03-14T09:01:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fe3f3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f80c7d50 con 0x7f93f8070a10 2024-03-14T09:01:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fe3f3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f80c8030 con 0x7f93f80c31c0 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93f7fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f93f806c6e0 0x7f93f806d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36390/0 (socket says 172.21.15.154:36390) 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93f7fff700 1 -- 172.21.15.154:0/3308507134 learned_addr learned my addr 172.21.15.154:0/3308507134 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fc990700 1 --2- 172.21.15.154:0/3308507134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f80c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fc990700 1 -- 172.21.15.154:0/3308507134 >> v1:172.21.15.191:6789/0 conn(0x7f93f8070a10 legacy=0x7f93f80c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fc990700 1 -- 172.21.15.154:0/3308507134 >> v1:172.21.15.154:6789/0 conn(0x7f93f806c6e0 legacy=0x7f93f806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.307+0000 7f93fc990700 1 -- 172.21.15.154:0/3308507134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f80c8310 con 0x7f93f80c31c0 2024-03-14T09:01:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.308+0000 7f93fc990700 1 --2- 172.21.15.154:0/3308507134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f80c6ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93f80c3d50 tx=0x7f93f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe86400a459cdac server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93f6ffd700 1 -- 172.21.15.154:0/3308507134 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0016070 con 0x7f93f80c31c0 2024-03-14T09:01:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93f6ffd700 1 -- 172.21.15.154:0/3308507134 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000bbd0 con 0x7f93f80c31c0 2024-03-14T09:01:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93f6ffd700 1 -- 172.21.15.154:0/3308507134 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000e070 con 0x7f93f80c31c0 2024-03-14T09:01:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3308507134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 msgr2=0x7f93f80c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3308507134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f80c6ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93f80c3d50 tx=0x7f93f000b330 comp rx=0 tx=0).stop 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3308507134 shutdown_connections 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3308507134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f80c6ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3308507134 >> 172.21.15.154:0/3308507134 conn(0x7f93f80a9bf0 msgr2=0x7f93f80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3308507134 shutdown_connections 2024-03-14T09:01:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.309+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3308507134 wait complete. 2024-03-14T09:01:56.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.310+0000 7f93fe3f3700 1 Processor -- start 2024-03-14T09:01:56.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 -- start start 2024-03-14T09:01:56.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:56.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93f8070a10 0x7f93f81da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f81d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81d97c0 con 0x7f93f806c6e0 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81e0780 con 0x7f93f8070a10 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fe3f3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f81eac20 con 0x7f93f80c31c0 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93f7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93f7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35390/0 (socket says 172.21.15.154:35390) 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93f7fff700 1 -- 172.21.15.154:0/3754069865 learned_addr learned my addr 172.21.15.154:0/3754069865 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93fc990700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f81d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.311+0000 7f93f77fe700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93f8070a10 0x7f93f81da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 -- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 msgr2=0x7f93f81d7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f81d7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 -- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93f8070a10 msgr2=0x7f93f81da8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93f8070a10 0x7f93f81da8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:56.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 -- 172.21.15.154:0/3754069865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f0009a70 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f7fff700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93e800c550 tx=0x7f93e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8005d70 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e80070d0 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f81dc020 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8007440 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.312+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f81ea8b0 con 0x7f93f806c6e0 2024-03-14T09:01:56.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f80697d0 con 0x7f93f806c6e0 2024-03-14T09:01:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.313+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f93e80075a0 con 0x7f93f806c6e0 2024-03-14T09:01:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.314+0000 7f93f4ff9700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 0x7f93e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.314+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f93e8087c30 con 0x7f93f806c6e0 2024-03-14T09:01:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.314+0000 7f93f77fe700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 0x7f93e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.314+0000 7f93f77fe700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 0x7f93e406ccf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93f8074c30 tx=0x7f93ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:56.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.318+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f93e8053490 con 0x7f93f806c6e0 2024-03-14T09:01:56.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:56.526+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f93f80c2d90 con 0x7f93f806c6e0 2024-03-14T09:01:57.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.310+0000 7f93f4ff9700 1 -- 172.21.15.154:0/3754069865 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f93e8056ab0 con 0x7f93f806c6e0 2024-03-14T09:01:57.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 msgr2=0x7f93e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 0x7f93e406ccf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93f8074c30 tx=0x7f93ec009040 comp rx=0 tx=0).stop 2024-03-14T09:01:57.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 msgr2=0x7f93f81db990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93e800c550 tx=0x7f93e800c860 comp rx=0 tx=0).stop 2024-03-14T09:01:57.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 shutdown_connections 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93f80c31c0 0x7f93f81d7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f93e406a830 0x7f93e406ccf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93f806c6e0 0x7f93f81db990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 --2- 172.21.15.154:0/3754069865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93f8070a10 0x7f93f81da8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 >> 172.21.15.154:0/3754069865 conn(0x7f93f80a9bf0 msgr2=0x7f93f806d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 shutdown_connections 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.313+0000 7f93fe3f3700 1 -- 172.21.15.154:0/3754069865 wait complete. 2024-03-14T09:01:57.314 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2024-03-14T09:01:57.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T09:01:57.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 Processor -- start 2024-03-14T09:01:57.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 -- start start 2024-03-14T09:01:57.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa454155df0 0x7fa454156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4540ac8e0 con 0x7fa45415d270 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45406de30 con 0x7fa45414eaf0 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.452+0000 7fa45af27700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45406b310 con 0x7fa454155df0 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa454155df0 0x7fa454156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4594c4700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa45415d270 0x7fa45415f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36396/0 (socket says 172.21.15.154:36396) 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa454155df0 0x7fa454156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35402/0 (socket says 172.21.15.154:35402) 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4594c4700 1 -- 172.21.15.154:0/2882908756 learned_addr learned my addr 172.21.15.154:0/2882908756 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 -- 172.21.15.154:0/2882908756 >> v1:172.21.15.191:6789/0 conn(0x7fa45414eaf0 legacy=0x7fa45406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 -- 172.21.15.154:0/2882908756 >> v1:172.21.15.154:6789/0 conn(0x7fa45415d270 legacy=0x7fa45415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 -- 172.21.15.154:0/2882908756 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540acb30 con 0x7fa454155df0 2024-03-14T09:01:57.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4537fe700 1 -- 172.21.15.154:0/2882908756 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745937033 0 0) 0x7fa4540ac8e0 con 0x7fa45415d270 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa453fff700 1 --2- 172.21.15.154:0/2882908756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa454155df0 0x7fa454156250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa440009840 tx=0x7fa440009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea95ff949128a130 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4537fe700 1 -- 172.21.15.154:0/2882908756 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440011830 con 0x7fa454155df0 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4537fe700 1 -- 172.21.15.154:0/2882908756 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa440011990 con 0x7fa454155df0 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.453+0000 7fa4537fe700 1 -- 172.21.15.154:0/2882908756 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440011d20 con 0x7fa454155df0 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 -- 172.21.15.154:0/2882908756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa454155df0 msgr2=0x7fa454156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 --2- 172.21.15.154:0/2882908756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa454155df0 0x7fa454156250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa440009840 tx=0x7fa440009b50 comp rx=0 tx=0).stop 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 -- 172.21.15.154:0/2882908756 shutdown_connections 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 --2- 172.21.15.154:0/2882908756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa454155df0 0x7fa454156250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 -- 172.21.15.154:0/2882908756 >> 172.21.15.154:0/2882908756 conn(0x7fa4540c3880 msgr2=0x7fa4540c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 -- 172.21.15.154:0/2882908756 shutdown_connections 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.454+0000 7fa45af27700 1 -- 172.21.15.154:0/2882908756 wait complete. 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.455+0000 7fa45af27700 1 Processor -- start 2024-03-14T09:01:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.455+0000 7fa45af27700 1 -- start start 2024-03-14T09:01:57.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa454155df0 0x7fa454142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 0x7fa454143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa454158370 con 0x7fa45414eaf0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa454143740 con 0x7fa454155df0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4541501e0 con 0x7fa45415d270 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35406/0 (socket says 172.21.15.154:35406) 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 -- 172.21.15.154:0/2415068466 learned_addr learned my addr 172.21.15.154:0/2415068466 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa453fff700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa454155df0 0x7fa454142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa4594c4700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 0x7fa454143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 -- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 msgr2=0x7fa454143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 0x7fa454143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 -- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa454155df0 msgr2=0x7fa454142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa454155df0 0x7fa454142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 -- 172.21.15.154:0/2415068466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4400094a0 con 0x7fa45414eaf0 2024-03-14T09:01:57.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa4594c4700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 0x7fa454143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa458cc3700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c450 tx=0x7fa44800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480052d0 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448005430 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4541eb000 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.456+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480038c0 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.457+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa454158000 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.457+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4540697d0 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.458+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa448003a20 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.458+0000 7fa4517fa700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 0x7fa43806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.458+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa448087d60 con 0x7fa45414eaf0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.459+0000 7fa453fff700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 0x7fa43806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.459+0000 7fa453fff700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 0x7fa43806ccf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa4400090b0 tx=0x7fa44001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.463+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa448053560 con 0x7fa45414eaf0 2024-03-14T09:01:57.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.734+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa45415d050 con 0x7fa45414eaf0 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.735+0000 7fa4517fa700 1 -- 172.21.15.154:0/2415068466 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6065 (secure 0 0 0) 0x7fa448056b80 con 0x7fa45414eaf0 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlnCs5CxAA/mf9z7eSw0Fh5yqCxmvhww==", 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:01:57.735 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlq18XGBAA9Nr/cXAtRbeaKmJDCLUz6w==", 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.736 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlvIuREhAAO1gOyqDP2tTJcb1IuOdQ0A==", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlAAuqHxAAy5SQz2lJHPwRad7HCIjjSg==", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:01:57.737 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCvvPJlDzrPMRAAEAP7kHfAaxZGM+W0rY0DIA==", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCyvPJlgS2rMBAAoPtjrTbc7bMLMJAIUKD+vg==", 2024-03-14T09:01:57.738 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC1vPJltO6ULxAAvkKiSoRBoKxFz8zPjGCOnA==", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC4vPJlWIs/LhAAi5LaiHbS1svlQ0UesC9CLg==", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC+vPJl5TleMRAAjIGbvgOjoV3mJYIK1fQoYA==", 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.740 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDBvPJlhiV5MBAAe7XI8s0kWQRAhvWWqMInrg==", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDEvPJlbghELxAASO+DuPJa7zXCH/KkhPi1WA==", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDHvPJlRba5LhAASpiL/U05lgta2f4xKra5ZQ==", 2024-03-14T09:01:57.742 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlhDgFJBAA47spIEFGnChaoxmMQkaA+w==", 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.743 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJl58xgJxAAfqdSCJED8MwHCScl1fqdFA==", 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:01:57.744 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlMjx9JRAAScnZoVr95MollZzYB2rQIA==", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.745 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-mds", 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJlhr+YIBAAFFJ9OobMJ/AoKAPhfCkf0g==", 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-mds" 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.746 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-mgr", 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJlZMiYIBAAM8cQFtF3qa1G9F4geCsM7w==", 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-osd", 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJl1s+YIBAARcH1Dp/cc66HjxRr2e3YsA==", 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-osd" 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rbd", 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJl99iYIBAAMYPDexvbovZGcDq6Kq+ecA==", 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJlfuCYIBAA0vBjuwINXcRVWOtCAD73Zg==", 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rgw", 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDNvPJl9+eYIBAAp9GulpvXNwlIRnIyl/if3A==", 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlSRLFBxAAkIg2TMRbiwcdMggjl+vMWw==", 2024-03-14T09:01:57.749 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlQsV4OxAADoB6pH163AX2+5II4EtdFw==", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:01:57.750 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:01:57.751 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.751 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:01:57.751 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-03-14T09:01:57.751 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:01:57.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 msgr2=0x7fa43806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 0x7fa43806ccf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa4400090b0 tx=0x7fa44001c040 comp rx=0 tx=0).stop 2024-03-14T09:01:57.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 msgr2=0x7fa4540c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa44800c450 tx=0x7fa44800c760 comp rx=0 tx=0).stop 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 shutdown_connections 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa45415d270 0x7fa454143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa43806a830 0x7fa43806ccf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa45414eaf0 0x7fa4540c5e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 --2- 172.21.15.154:0/2415068466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa454155df0 0x7fa454142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 >> 172.21.15.154:0/2415068466 conn(0x7fa4540c3880 msgr2=0x7fa45414d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 shutdown_connections 2024-03-14T09:01:57.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.737+0000 7fa45af27700 1 -- 172.21.15.154:0/2415068466 wait complete. 2024-03-14T09:01:57.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-14T09:01:57.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.877+0000 7fbda3a60700 1 Processor -- start 2024-03-14T09:01:57.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.877+0000 7fbda3a60700 1 -- start start 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda3a60700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda3a60700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda3a60700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c157e80 con 0x7fbd9c0c8350 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda3a60700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c158ac0 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda3a60700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c158120 con 0x7fbd9c0c7f70 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35416/0 (socket says 172.21.15.154:35416) 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda1ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbd9c0c8350 0x7fbd9c154d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36400/0 (socket says 172.21.15.154:36400) 2024-03-14T09:01:57.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 -- 172.21.15.154:0/1052818865 learned_addr learned my addr 172.21.15.154:0/1052818865 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda17fc700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 -- 172.21.15.154:0/1052818865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 msgr2=0x7fbd9c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 -- 172.21.15.154:0/1052818865 >> v1:172.21.15.154:6789/0 conn(0x7fbd9c0c8350 legacy=0x7fbd9c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 -- 172.21.15.154:0/1052818865 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c158400 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbd93fff700 1 -- 172.21.15.154:0/1052818865 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225217080 0 0) 0x7fbd9c157e80 con 0x7fbd9c0c8350 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbda0ffb700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0098f0 tx=0x7fbd8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356db20293cd7653 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.878+0000 7fbd93fff700 1 -- 172.21.15.154:0/1052818865 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00b8a0 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbd93fff700 1 -- 172.21.15.154:0/1052818865 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00ba00 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbd93fff700 1 -- 172.21.15.154:0/1052818865 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00bd90 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 -- 172.21.15.154:0/1052818865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c4f30 msgr2=0x7fbd9c1547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0098f0 tx=0x7fbd8c009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:57.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 -- 172.21.15.154:0/1052818865 shutdown_connections 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 --2- 172.21.15.154:0/1052818865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1547f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 -- 172.21.15.154:0/1052818865 >> 172.21.15.154:0/1052818865 conn(0x7fbd9c072080 msgr2=0x7fbd9c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 -- 172.21.15.154:0/1052818865 shutdown_connections 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.879+0000 7fbda3a60700 1 -- 172.21.15.154:0/1052818865 wait complete. 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.880+0000 7fbda3a60700 1 Processor -- start 2024-03-14T09:01:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.880+0000 7fbda3a60700 1 -- start start 2024-03-14T09:01:57.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1e0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c153580 con 0x7fbd9c0c8350 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0aa230 con 0x7fbd9c0c4f30 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda3a60700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0aa510 con 0x7fbd9c0c7f70 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35422/0 (socket says 172.21.15.154:35422) 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 -- 172.21.15.154:0/263839164 learned_addr learned my addr 172.21.15.154:0/263839164 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda17fc700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda0ffb700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 -- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 msgr2=0x7fbd9c0aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 -- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0c4f30 msgr2=0x7fbd9c1e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.881+0000 7fbda1ffd700 1 -- 172.21.15.154:0/263839164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009550 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbda1ffd700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd94002230 tx=0x7fbd9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400ac20 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9400ada0 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007310 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0aa7f0 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.882+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0bd610 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.883+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c153070 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.883+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbd94007470 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.883+0000 7fbd91ffb700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 0x7fbd8006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.884+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd94088500 con 0x7fbd9c0c8350 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.884+0000 7fbda17fc700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 0x7fbd8006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:57.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.884+0000 7fbda17fc700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 0x7fbd8006ccf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b9b0 tx=0x7fbd98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:57.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:57.888+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbd9c153070 con 0x7fbd9c0c8350 2024-03-14T09:01:58.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.162+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbd9c153070 con 0x7fbd9c0c8350 2024-03-14T09:01:58.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.166+0000 7fbd91ffb700 1 -- 172.21.15.154:0/263839164 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd9c153070 con 0x7fbd9c0c8350 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 msgr2=0x7fbd8006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 0x7fbd8006ccf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800b9b0 tx=0x7fbd98008040 comp rx=0 tx=0).stop 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 msgr2=0x7fbd9c1def60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd94002230 tx=0x7fbd9400a060 comp rx=0 tx=0).stop 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 shutdown_connections 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8006a830 0x7fbd8006ccf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9c0c8350 0x7fbd9c1def60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9c0c4f30 0x7fbd9c1e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 --2- 172.21.15.154:0/263839164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9c0c7f70 0x7fbd9c0aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 >> 172.21.15.154:0/263839164 conn(0x7fbd9c072080 msgr2=0x7fbd9c156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:58.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 shutdown_connections 2024-03-14T09:01:58.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.168+0000 7fbda3a60700 1 -- 172.21.15.154:0/263839164 wait complete. 2024-03-14T09:01:58.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-14T09:01:58.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.304+0000 7f136040a700 1 Processor -- start 2024-03-14T09:01:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.304+0000 7f136040a700 1 -- start start 2024-03-14T09:01:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.304+0000 7f136040a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13580a9650 0x7f13580c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.304+0000 7f136040a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f135806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f136040a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1358156050 con 0x7f1358156960 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f136040a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580a86f0 con 0x7f13580a9650 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f136040a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580c34b0 con 0x7f13581564e0 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13580a9650 0x7f13580c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13580a9650 0x7f13580c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35430/0 (socket says 172.21.15.154:35430) 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 -- 172.21.15.154:0/2721493089 learned_addr learned my addr 172.21.15.154:0/2721493089 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135d9a5700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f135806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 -- 172.21.15.154:0/2721493089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 msgr2=0x7f135806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f135806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 -- 172.21.15.154:0/2721493089 >> v1:172.21.15.191:6789/0 conn(0x7f1358156960 legacy=0x7f1358152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 -- 172.21.15.154:0/2721493089 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580a8990 con 0x7f13580a9650 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135e1a6700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c1780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13540098f0 tx=0x7f1354009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67ccd36fd192edd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135d1a4700 1 -- 172.21.15.154:0/2721493089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354011830 con 0x7f13580a9650 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135d1a4700 1 -- 172.21.15.154:0/2721493089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354011990 con 0x7f13580a9650 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.305+0000 7f135d1a4700 1 -- 172.21.15.154:0/2721493089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354011d20 con 0x7f13580a9650 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 -- 172.21.15.154:0/2721493089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 msgr2=0x7f13580c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c1780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13540098f0 tx=0x7f1354009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 -- 172.21.15.154:0/2721493089 shutdown_connections 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c1780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 --2- 172.21.15.154:0/2721493089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f135806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 -- 172.21.15.154:0/2721493089 >> 172.21.15.154:0/2721493089 conn(0x7f13580c3880 msgr2=0x7f13580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 -- 172.21.15.154:0/2721493089 shutdown_connections 2024-03-14T09:01:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.306+0000 7f136040a700 1 -- 172.21.15.154:0/2721493089 wait complete. 2024-03-14T09:01:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.307+0000 7f136040a700 1 Processor -- start 2024-03-14T09:01:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.307+0000 7f136040a700 1 -- start start 2024-03-14T09:01:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f13580c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358156960 0x7f1358144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358148820 con 0x7f13580a9650 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13581448f0 con 0x7f1358156960 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f136040a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580c6a40 con 0x7f13581564e0 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35444/0 (socket says 172.21.15.154:35444) 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 -- 172.21.15.154:0/3885545283 learned_addr learned my addr 172.21.15.154:0/3885545283 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135d9a5700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f13580c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e9a7700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358156960 0x7f1358144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 -- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 msgr2=0x7f13580c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f13580c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 -- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358156960 msgr2=0x7f1358144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358156960 0x7f1358144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 -- 172.21.15.154:0/3885545283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1354009550 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.308+0000 7f135e1a6700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1354014040 tx=0x7f1354004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.309+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354004450 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.309+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135401e070 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.309+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1358147fc0 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.309+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13540038c0 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.309+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1358148540 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.310+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13580697d0 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.310+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1354003a20 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.311+0000 7f134effd700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 0x7f133c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.311+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1354020070 con 0x7f13580a9650 2024-03-14T09:01:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.311+0000 7f135d9a5700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 0x7f133c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.311+0000 7f135d9a5700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 0x7f133c06ccf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13581ec6e0 tx=0x7f1348009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.316+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1354055370 con 0x7f13580a9650 2024-03-14T09:01:58.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.591+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f135806b470 con 0x7f13580a9650 2024-03-14T09:01:58.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.594+0000 7f134effd700 1 -- 172.21.15.154:0/3885545283 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f13540045b0 con 0x7f13580a9650 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.596+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 msgr2=0x7f133c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.596+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 0x7f133c06ccf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13581ec6e0 tx=0x7f1348009210 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.596+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 msgr2=0x7f13580c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.596+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1354014040 tx=0x7f1354004750 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 shutdown_connections 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358156960 0x7f1358144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f133c06a830 0x7f133c06ccf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13580a9650 0x7f13580c5fc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 --2- 172.21.15.154:0/3885545283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13581564e0 0x7f13580c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 >> 172.21.15.154:0/3885545283 conn(0x7f13580c3880 msgr2=0x7f1358071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 shutdown_connections 2024-03-14T09:01:58.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.597+0000 7f136040a700 1 -- 172.21.15.154:0/3885545283 wait complete. 2024-03-14T09:01:58.609 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-14T09:01:58.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.732+0000 7f52bc222700 1 Processor -- start 2024-03-14T09:01:58.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 -- start start 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 0x7f52b406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52b4156500 0x7f52b415f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b40ac8e0 con 0x7f52b406acb0 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b406b310 con 0x7f52b4156500 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52bc222700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b406de30 con 0x7f52b414eaf0 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.733+0000 7f52ba7bf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52b4156500 0x7f52b415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52b4156500 0x7f52b415f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35460/0 (socket says 172.21.15.154:35460) 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 -- 172.21.15.154:0/612565435 learned_addr learned my addr 172.21.15.154:0/612565435 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52b9fbe700 1 --2- 172.21.15.154:0/612565435 >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 0x7f52b406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 -- 172.21.15.154:0/612565435 >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 msgr2=0x7f52b406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 --2- 172.21.15.154:0/612565435 >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 0x7f52b406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 -- 172.21.15.154:0/612565435 >> v1:172.21.15.191:6789/0 conn(0x7f52b406acb0 legacy=0x7f52b4155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 -- 172.21.15.154:0/612565435 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4156910 con 0x7f52b4156500 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52b9fbe700 1 --2- 172.21.15.154:0/612565435 >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 0x7f52b406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52ba7bf700 1 --2- 172.21.15.154:0/612565435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b4156500 0x7f52b415f280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52b4070da0 tx=0x7f52ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979ab0117ad2409c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52b8fbc700 1 -- 172.21.15.154:0/612565435 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00e070 con 0x7f52b4156500 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52b8fbc700 1 -- 172.21.15.154:0/612565435 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ac003a20 con 0x7f52b4156500 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.734+0000 7f52b8fbc700 1 -- 172.21.15.154:0/612565435 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac003dd0 con 0x7f52b4156500 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 -- 172.21.15.154:0/612565435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b4156500 msgr2=0x7f52b415f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 --2- 172.21.15.154:0/612565435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b4156500 0x7f52b415f280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52b4070da0 tx=0x7f52ac00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 -- 172.21.15.154:0/612565435 shutdown_connections 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 --2- 172.21.15.154:0/612565435 >> v2:172.21.15.191:3300/0 conn(0x7f52b414eaf0 0x7f52b406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 --2- 172.21.15.154:0/612565435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b4156500 0x7f52b415f280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 -- 172.21.15.154:0/612565435 >> 172.21.15.154:0/612565435 conn(0x7f52b40c3880 msgr2=0x7f52b40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 -- 172.21.15.154:0/612565435 shutdown_connections 2024-03-14T09:01:58.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.735+0000 7f52bc222700 1 -- 172.21.15.154:0/612565435 wait complete. 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.736+0000 7f52bc222700 1 Processor -- start 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.736+0000 7f52bc222700 1 -- start start 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b406acb0 0x7f52b40c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b4156500 0x7f52b4144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41dea10 con 0x7f52b414eaf0 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41de370 con 0x7f52b406acb0 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52bc222700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41de090 con 0x7f52b4156500 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52b97bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52b97bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35476/0 (socket says 172.21.15.154:35476) 2024-03-14T09:01:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52b97bd700 1 -- 172.21.15.154:0/741061713 learned_addr learned my addr 172.21.15.154:0/741061713 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.737+0000 7f52b9fbe700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b406acb0 0x7f52b40c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52ba7bf700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b4156500 0x7f52b4144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 -- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b4156500 msgr2=0x7f52b4144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b4156500 0x7f52b4144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 -- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b406acb0 msgr2=0x7f52b40c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b406acb0 0x7f52b40c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 -- 172.21.15.154:0/741061713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac009b30 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52b97bd700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c4d0 tx=0x7f52b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011790 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b00118f0 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0011ca0 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b4146120 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.738+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b41deea0 con 0x7f52b414eaf0 2024-03-14T09:01:58.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.739+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b41deb80 con 0x7f52b414eaf0 2024-03-14T09:01:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.739+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f52b001c460 con 0x7f52b414eaf0 2024-03-14T09:01:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.740+0000 7f52a67fc700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 0x7f529806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.740+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52b00887c0 con 0x7f52b414eaf0 2024-03-14T09:01:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.740+0000 7f52b9fbe700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 0x7f529806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.740+0000 7f52b9fbe700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 0x7f529806ccf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:58.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:58.744+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f52b0053fc0 con 0x7f52b414eaf0 2024-03-14T09:01:59.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.021+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f52b40697d0 con 0x7f52b414eaf0 2024-03-14T09:01:59.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.025+0000 7f52a67fc700 1 -- 172.21.15.154:0/741061713 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f52b00575e0 con 0x7f52b414eaf0 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 msgr2=0x7f529806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 0x7f529806ccf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52a8006fd0 tx=0x7f52a8008040 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 msgr2=0x7f52b41446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c4d0 tx=0x7f52b000c890 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 shutdown_connections 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52b4156500 0x7f52b4144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52b406acb0 0x7f52b40c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f529806a830 0x7f529806ccf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 --2- 172.21.15.154:0/741061713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b414eaf0 0x7f52b41446d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 >> 172.21.15.154:0/741061713 conn(0x7f52b40c3880 msgr2=0x7f52b4157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:59.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 shutdown_connections 2024-03-14T09:01:59.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.028+0000 7f52bc222700 1 -- 172.21.15.154:0/741061713 wait complete. 2024-03-14T09:01:59.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-14T09:01:59.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.163+0000 7f7db8a87700 1 Processor -- start 2024-03-14T09:01:59.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.163+0000 7f7db8a87700 1 -- start start 2024-03-14T09:01:59.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db8a87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db8a87700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7db40c1780 0x7f7db40c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db8a87700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db413ade0 con 0x7f7db40a8540 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db8a87700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db406fb30 con 0x7f7db40c1780 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db8a87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db406ec30 con 0x7f7db40c3250 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7db40c1780 0x7f7db40c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7db40a8540 0x7f7db4154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36414/0 (socket says 172.21.15.154:36414) 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db2d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7db40c1780 0x7f7db40c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35484/0 (socket says 172.21.15.154:35484) 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.164+0000 7f7db259c700 1 -- 172.21.15.154:0/3614277109 learned_addr learned my addr 172.21.15.154:0/3614277109 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db1d9b700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db2d9d700 1 -- 172.21.15.154:0/3614277109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 msgr2=0x7f7db413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db2d9d700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db2d9d700 1 -- 172.21.15.154:0/3614277109 >> v1:172.21.15.154:6789/0 conn(0x7f7db40a8540 legacy=0x7f7db4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db2d9d700 1 -- 172.21.15.154:0/3614277109 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db406ef10 con 0x7f7db40c1780 2024-03-14T09:01:59.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db159a700 1 -- 172.21.15.154:0/3614277109 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855541555 0 0) 0x7f7da8001110 con 0x7f7db40a8540 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db1d9b700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db2d9d700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c1780 0x7f7db40c1c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2f0b948c8a6ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db159a700 1 -- 172.21.15.154:0/3614277109 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00e070 con 0x7f7db40c1780 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db159a700 1 -- 172.21.15.154:0/3614277109 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00bbd0 con 0x7f7db40c1780 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db159a700 1 -- 172.21.15.154:0/3614277109 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac012430 con 0x7f7db40c1780 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db8a87700 1 -- 172.21.15.154:0/3614277109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c1780 msgr2=0x7f7db40c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.165+0000 7f7db8a87700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c1780 0x7f7db40c1c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac00b330 comp rx=0 tx=0).stop 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 -- 172.21.15.154:0/3614277109 shutdown_connections 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40c1780 0x7f7db40c1c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 --2- 172.21.15.154:0/3614277109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 -- 172.21.15.154:0/3614277109 >> 172.21.15.154:0/3614277109 conn(0x7f7db40c3880 msgr2=0x7f7db40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 -- 172.21.15.154:0/3614277109 shutdown_connections 2024-03-14T09:01:59.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.166+0000 7f7db8a87700 1 -- 172.21.15.154:0/3614277109 wait complete. 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.167+0000 7f7db8a87700 1 Processor -- start 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.167+0000 7f7db8a87700 1 -- start start 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.167+0000 7f7db8a87700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db40c1780 0x7f7db414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db414eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41eaf60 con 0x7f7db40a8540 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4152490 con 0x7f7db40c1780 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41eac80 con 0x7f7db40c3250 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35488/0 (socket says 172.21.15.154:35488) 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 -- 172.21.15.154:0/769643274 learned_addr learned my addr 172.21.15.154:0/769643274 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db2d9d700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db414eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db1d9b700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db40c1780 0x7f7db414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 -- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 msgr2=0x7f7db414eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db414eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 -- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db40c1780 msgr2=0x7f7db414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db40c1780 0x7f7db414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 -- 172.21.15.154:0/769643274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac009a70 con 0x7f7db40a8540 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db259c700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e760 tx=0x7f7da800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004a80 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8004c00 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db41e8050 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80070d0 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.168+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db41e8610 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.169+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db41e8830 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.170+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7da8007230 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.170+0000 7f7da2ffd700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 0x7f7d9406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.171+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7da808ad10 con 0x7f7db40a8540 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.171+0000 7f7db1d9b700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 0x7f7d9406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.171+0000 7f7db1d9b700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 0x7f7d9406cd20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7db41544e0 tx=0x7f7da4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.175+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7da8056470 con 0x7f7db40a8540 2024-03-14T09:01:59.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.449+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7db413ade0 con 0x7f7db40a8540 2024-03-14T09:01:59.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.453+0000 7f7da2ffd700 1 -- 172.21.15.154:0/769643274 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7db413ade0 con 0x7f7db40a8540 2024-03-14T09:01:59.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 msgr2=0x7f7d9406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 0x7f7d9406cd20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7db41544e0 tx=0x7f7da4008040 comp rx=0 tx=0).stop 2024-03-14T09:01:59.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 msgr2=0x7f7db41ea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e760 tx=0x7f7da800eb20 comp rx=0 tx=0).stop 2024-03-14T09:01:59.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 shutdown_connections 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db40c3250 0x7f7db414eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d9406a860 0x7f7d9406cd20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7db40a8540 0x7f7db41ea290 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 --2- 172.21.15.154:0/769643274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db40c1780 0x7f7db414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 >> 172.21.15.154:0/769643274 conn(0x7f7db40c3880 msgr2=0x7f7db406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 shutdown_connections 2024-03-14T09:01:59.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.455+0000 7f7db8a87700 1 -- 172.21.15.154:0/769643274 wait complete. 2024-03-14T09:01:59.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-14T09:01:59.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.590+0000 7f962e4a0700 1 Processor -- start 2024-03-14T09:01:59.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.590+0000 7f962e4a0700 1 -- start start 2024-03-14T09:01:59.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962e4a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f962814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962e4a0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96280c55a0 0x7f96280c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962e4a0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9628155a40 con 0x7f96281518c0 2024-03-14T09:01:59.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962e4a0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280c2f10 con 0x7f96280c55a0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962e4a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9628072a90 con 0x7f96280c26c0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96280c55a0 0x7f96280c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96280c55a0 0x7f96280c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57202/0 (socket says 172.21.15.154:57202) 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f962ca3d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f96281518c0 0x7f9628151d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46914/0 (socket says 172.21.15.154:46914) 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 -- 172.21.15.154:0/3668483471 learned_addr learned my addr 172.21.15.154:0/3668483471 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f9627fff700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f962814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 -- 172.21.15.154:0/3668483471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 msgr2=0x7f962814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f962814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 -- 172.21.15.154:0/3668483471 >> v1:172.21.15.154:6789/0 conn(0x7f96281518c0 legacy=0x7f9628151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f96277fe700 1 -- 172.21.15.154:0/3668483471 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96281556d0 con 0x7f96280c55a0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.591+0000 7f9626ffd700 1 -- 172.21.15.154:0/3668483471 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71222567 0 0) 0x7f9628155a40 con 0x7f96281518c0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f96277fe700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96280c5a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96180098f0 tx=0x7f9618009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9855c581ef48c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f9626ffd700 1 -- 172.21.15.154:0/3668483471 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800b8a0 con 0x7f96280c55a0 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f9626ffd700 1 -- 172.21.15.154:0/3668483471 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961800ba00 con 0x7f96280c55a0 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f9626ffd700 1 -- 172.21.15.154:0/3668483471 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800bd90 con 0x7f96280c55a0 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 -- 172.21.15.154:0/3668483471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 msgr2=0x7f96280c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96280c5a00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f96180098f0 tx=0x7f9618009c00 comp rx=0 tx=0).stop 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 -- 172.21.15.154:0/3668483471 shutdown_connections 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f962814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 --2- 172.21.15.154:0/3668483471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96280c5a00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 -- 172.21.15.154:0/3668483471 >> 172.21.15.154:0/3668483471 conn(0x7f962806a770 msgr2=0x7f962806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 -- 172.21.15.154:0/3668483471 shutdown_connections 2024-03-14T09:01:59.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.592+0000 7f962e4a0700 1 -- 172.21.15.154:0/3668483471 wait complete. 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.594+0000 7f962e4a0700 1 Processor -- start 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.594+0000 7f962e4a0700 1 -- start start 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f96280bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 0x7f9628141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9628141f50 con 0x7f96280c55a0 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96281ed970 con 0x7f96281518c0 2024-03-14T09:01:59.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96281edc50 con 0x7f96280c26c0 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57214/0 (socket says 172.21.15.154:57214) 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 -- 172.21.15.154:0/1156370348 learned_addr learned my addr 172.21.15.154:0/1156370348 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 -- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 msgr2=0x7f96280bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f96280bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 -- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 msgr2=0x7f9628141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962ca3d700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 0x7f9628141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 0x7f9628141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 -- 172.21.15.154:0/1156370348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009550 con 0x7f96280c55a0 2024-03-14T09:01:59.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962ca3d700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 0x7f9628141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f96277fe700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9618009c40 tx=0x7f9618004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96180036a0 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618003800 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96281edf30 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618003b90 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.595+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96281ee4f0 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.596+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280697d0 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.597+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9618021070 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.597+0000 7f9624ff9700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 0x7f961406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.598+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9618089f00 con 0x7f96280c55a0 2024-03-14T09:01:59.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.598+0000 7f9627fff700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 0x7f961406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:01:59.600 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.598+0000 7f9627fff700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 0x7f961406ccf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f961c0097f0 tx=0x7f961c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:01:59.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.602+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9618055700 con 0x7f96280c55a0 2024-03-14T09:01:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.874+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9628066860 con 0x7f96280c55a0 2024-03-14T09:01:59.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.878+0000 7f9624ff9700 1 -- 172.21.15.154:0/1156370348 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f96180290a0 con 0x7f96280c55a0 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 msgr2=0x7f961406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 0x7f961406ccf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f961c0097f0 tx=0x7f961c008040 comp rx=0 tx=0).stop 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 msgr2=0x7f96281414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9618009c40 tx=0x7f9618004750 comp rx=0 tx=0).stop 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 shutdown_connections 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96281518c0 0x7f9628141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96280c26c0 0x7f96280bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f961406a830 0x7f961406ccf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 --2- 172.21.15.154:0/1156370348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96280c55a0 0x7f96281414d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 >> 172.21.15.154:0/1156370348 conn(0x7f962806a770 msgr2=0x7f9628071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 shutdown_connections 2024-03-14T09:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:01:59.880+0000 7f962e4a0700 1 -- 172.21.15.154:0/1156370348 wait complete. 2024-03-14T09:01:59.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-14T09:02:00.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.013+0000 7fc747519700 1 Processor -- start 2024-03-14T09:02:00.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 -- start start 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc74014eaf0 0x7fc74006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006acb0 0x7fc740155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7400ac8e0 con 0x7fc740156330 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74006b310 con 0x7fc74014eaf0 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc747519700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74006de30 con 0x7fc74006acb0 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc7452b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc74014eaf0 0x7fc74006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.014+0000 7fc7452b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc74014eaf0 0x7fc74006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57220/0 (socket says 172.21.15.154:57220) 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 -- 172.21.15.154:0/2354815711 learned_addr learned my addr 172.21.15.154:0/2354815711 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc745ab6700 1 --1- 172.21.15.154:0/2354815711 >> v1:172.21.15.154:6789/0 conn(0x7fc740156330 0x7fc740157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46928/0 (socket says 172.21.15.154:46928) 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 -- 172.21.15.154:0/2354815711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006acb0 msgr2=0x7fc740155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc744ab4700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006acb0 0x7fc740155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006acb0 0x7fc740155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc737fff700 1 -- 172.21.15.154:0/2354815711 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619660056 0 0) 0x7fc7400ac8e0 con 0x7fc740156330 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 -- 172.21.15.154:0/2354815711 >> v1:172.21.15.154:6789/0 conn(0x7fc740156330 legacy=0x7fc740157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 -- 172.21.15.154:0/2354815711 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400acb30 con 0x7fc74014eaf0 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc7452b5700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74014eaf0 0x7fc74006a770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b624f61c10a2401e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc737fff700 1 -- 172.21.15.154:0/2354815711 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000b8a0 con 0x7fc74014eaf0 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc737fff700 1 -- 172.21.15.154:0/2354815711 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000ba00 con 0x7fc74014eaf0 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.015+0000 7fc737fff700 1 -- 172.21.15.154:0/2354815711 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000bd90 con 0x7fc74014eaf0 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 -- 172.21.15.154:0/2354815711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74014eaf0 msgr2=0x7fc74006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74014eaf0 0x7fc74006a770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 -- 172.21.15.154:0/2354815711 shutdown_connections 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74014eaf0 0x7fc74006a770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 --2- 172.21.15.154:0/2354815711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006acb0 0x7fc740155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 -- 172.21.15.154:0/2354815711 >> 172.21.15.154:0/2354815711 conn(0x7fc7400c3880 msgr2=0x7fc7400c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 -- 172.21.15.154:0/2354815711 shutdown_connections 2024-03-14T09:02:00.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.016+0000 7fc747519700 1 -- 172.21.15.154:0/2354815711 wait complete. 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 Processor -- start 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 -- start start 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74014eaf0 0x7fc740151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc740156330 0x7fc7401362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7401eb160 con 0x7fc74006acb0 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7401eb440 con 0x7fc74014eaf0 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.017+0000 7fc747519700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74014f290 con 0x7fc740156330 2024-03-14T09:02:00.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57222/0 (socket says 172.21.15.154:57222) 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 -- 172.21.15.154:0/3143976254 learned_addr learned my addr 172.21.15.154:0/3143976254 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 -- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc740156330 msgr2=0x7fc7401362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc740156330 0x7fc7401362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 -- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74014eaf0 msgr2=0x7fc740151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74014eaf0 0x7fc740151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 -- 172.21.15.154:0/3143976254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009550 con 0x7fc74006acb0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc7452b5700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc730009130 tx=0x7fc730004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730004470 con 0x7fc74006acb0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73001e070 con 0x7fc74006acb0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7300038c0 con 0x7fc74006acb0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74014f570 con 0x7fc74006acb0 2024-03-14T09:02:00.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.018+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74014c570 con 0x7fc74006acb0 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.019+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74014fa90 con 0x7fc74006acb0 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.020+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc730020030 con 0x7fc74006acb0 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.020+0000 7fc735ffb700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 0x7fc72406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.020+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7300898a0 con 0x7fc74006acb0 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.020+0000 7fc744ab4700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 0x7fc72406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.021+0000 7fc744ab4700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 0x7fc72406cd20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc740145bc0 tx=0x7fc73c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.025+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc74014fa90 con 0x7fc74006acb0 2024-03-14T09:02:00.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.297+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc74014fa90 con 0x7fc74006acb0 2024-03-14T09:02:00.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.300+0000 7fc735ffb700 1 -- 172.21.15.154:0/3143976254 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc74014fa90 con 0x7fc74006acb0 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.302+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 msgr2=0x7fc72406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.302+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 0x7fc72406cd20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc740145bc0 tx=0x7fc73c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.302+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 msgr2=0x7fc740144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc730009130 tx=0x7fc730004750 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 shutdown_connections 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc740156330 0x7fc7401362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc72406a860 0x7fc72406cd20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc74006acb0 0x7fc740144ca0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 --2- 172.21.15.154:0/3143976254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74014eaf0 0x7fc740151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 >> 172.21.15.154:0/3143976254 conn(0x7fc7400c3880 msgr2=0x7fc74014c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 shutdown_connections 2024-03-14T09:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.303+0000 7fc747519700 1 -- 172.21.15.154:0/3143976254 wait complete. 2024-03-14T09:02:00.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:00.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.436+0000 7f6467727700 1 Processor -- start 2024-03-14T09:02:00.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.436+0000 7f6467727700 1 -- start start 2024-03-14T09:02:00.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f646006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f646006acb0 0x7f6460155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6460156330 0x7f6460157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646006b310 con 0x7f646006acb0 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600ac8e0 con 0x7f6460156330 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6467727700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646006de30 con 0x7f646014eaf0 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f646006acb0 0x7f6460155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f646006acb0 0x7f6460155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57238/0 (socket says 172.21.15.154:57238) 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 -- 172.21.15.154:0/3531015419 learned_addr learned my addr 172.21.15.154:0/3531015419 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6465cc4700 1 --2- 172.21.15.154:0/3531015419 >> v2:172.21.15.191:3300/0 conn(0x7f6460156330 0x7f6460157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f64654c3700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f646006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 -- 172.21.15.154:0/3531015419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 msgr2=0x7f646006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f646006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 -- 172.21.15.154:0/3531015419 >> v2:172.21.15.191:3300/0 conn(0x7f6460156330 msgr2=0x7f6460157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 --2- 172.21.15.154:0/3531015419 >> v2:172.21.15.191:3300/0 conn(0x7f6460156330 0x7f6460157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 -- 172.21.15.154:0/3531015419 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600acb30 con 0x7f646006acb0 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6464cc2700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f646006acb0 0x7f6460155df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6460070e50 tx=0x7f6450009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a8ba6e3a2ee4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6457fff700 1 -- 172.21.15.154:0/3531015419 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645000b870 con 0x7f646006acb0 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6457fff700 1 -- 172.21.15.154:0/3531015419 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645000b9d0 con 0x7f646006acb0 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.437+0000 7f6457fff700 1 -- 172.21.15.154:0/3531015419 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645000bd60 con 0x7f646006acb0 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 -- 172.21.15.154:0/3531015419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f646006acb0 msgr2=0x7f6460155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f646006acb0 0x7f6460155df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6460070e50 tx=0x7f6450009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 -- 172.21.15.154:0/3531015419 shutdown_connections 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 --2- 172.21.15.154:0/3531015419 >> v2:172.21.15.191:3300/0 conn(0x7f6460156330 0x7f6460157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f646006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 --2- 172.21.15.154:0/3531015419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f646006acb0 0x7f6460155df0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 -- 172.21.15.154:0/3531015419 >> 172.21.15.154:0/3531015419 conn(0x7f64600c3880 msgr2=0x7f64600c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 -- 172.21.15.154:0/3531015419 shutdown_connections 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.438+0000 7f6467727700 1 -- 172.21.15.154:0/3531015419 wait complete. 2024-03-14T09:02:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.439+0000 7f6467727700 1 Processor -- start 2024-03-14T09:02:00.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.439+0000 7f6467727700 1 -- start start 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 0x7f6460153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f6460146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460149af0 con 0x7f6460156330 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64601ede00 con 0x7f646006acb0 2024-03-14T09:02:00.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6467727700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64601ee330 con 0x7f646014eaf0 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f64654c3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 0x7f6460153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57252/0 (socket says 172.21.15.154:57252) 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f64654c3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 0x7f6460153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:43132/0 (socket says 172.21.15.154:43132) 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 -- 172.21.15.154:0/176757854 learned_addr learned my addr 172.21.15.154:0/176757854 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6464cc2700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f6460146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 -- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 msgr2=0x7f6460146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f6460146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 -- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 msgr2=0x7f6460153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 0x7f6460153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 -- 172.21.15.154:0/176757854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6450009550 con 0x7f6460156330 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.440+0000 7f6465cc4700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6458007e40 tx=0x7f645800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.441+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800ed30 con 0x7f6460156330 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.441+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6458014420 con 0x7f6460156330 2024-03-14T09:02:00.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.441+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64601ee050 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.441+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580147d0 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.441+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601491a0 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.442+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600697d0 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.442+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6458014930 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.442+0000 7f6455ffb700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 0x7f644406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.442+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6458089040 con 0x7f6460156330 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.443+0000 7f64654c3700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 0x7f644406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.443+0000 7f64654c3700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 0x7f644406ccf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f645c0045f0 tx=0x7f645c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.447+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6458054840 con 0x7f6460156330 2024-03-14T09:02:00.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.643+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f646006b470 con 0x7f6460156330 2024-03-14T09:02:00.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.644+0000 7f6455ffb700 1 -- 172.21.15.154:0/176757854 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f6458057e60 con 0x7f6460156330 2024-03-14T09:02:00.644 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:00.644 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":28,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:01:57.308212+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:00.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 msgr2=0x7f644406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 0x7f644406ccf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f645c0045f0 tx=0x7f645c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 msgr2=0x7f6460146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6458007e40 tx=0x7f645800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 shutdown_connections 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f644406a830 0x7f644406ccf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6460156330 0x7f6460146cd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646006acb0 0x7f6460153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 --2- 172.21.15.154:0/176757854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f646014eaf0 0x7f6460146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 >> 172.21.15.154:0/176757854 conn(0x7f64600c3880 msgr2=0x7f646014d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 shutdown_connections 2024-03-14T09:02:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.646+0000 7f6467727700 1 -- 172.21.15.154:0/176757854 wait complete. 2024-03-14T09:02:00.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:00.658 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:00.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 Processor -- start 2024-03-14T09:02:00.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 -- start start 2024-03-14T09:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fab2c154f00 0x7fab2c160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c157e80 con 0x7fab2c0c7f70 2024-03-14T09:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c158120 con 0x7fab2c0c8350 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab31ed4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c158ac0 con 0x7fab2c154f00 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab2bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fab2c154f00 0x7fab2c160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab2b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fab2c0c7f70 0x7fab2c0c49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46944/0 (socket says 172.21.15.154:46944) 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.784+0000 7fab2b7fe700 1 -- 172.21.15.154:0/3839657740 learned_addr learned my addr 172.21.15.154:0/3839657740 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2bfff700 1 -- 172.21.15.154:0/3839657740 >> v1:172.21.15.191:6789/0 conn(0x7fab2c0c8350 legacy=0x7fab2c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2bfff700 1 -- 172.21.15.154:0/3839657740 >> v1:172.21.15.154:6789/0 conn(0x7fab2c0c7f70 legacy=0x7fab2c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2a7fc700 1 -- 172.21.15.154:0/3839657740 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622502771 0 0) 0x7fab2c157e80 con 0x7fab2c0c7f70 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2bfff700 1 -- 172.21.15.154:0/3839657740 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c158400 con 0x7fab2c154f00 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2bfff700 1 --2- 172.21.15.154:0/3839657740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c154f00 0x7fab2c160180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab24009d90 tx=0x7fab2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9855187794546de server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2a7fc700 1 -- 172.21.15.154:0/3839657740 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2400e070 con 0x7fab2c154f00 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2a7fc700 1 -- 172.21.15.154:0/3839657740 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2400bb90 con 0x7fab2c154f00 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.785+0000 7fab2a7fc700 1 -- 172.21.15.154:0/3839657740 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24003a20 con 0x7fab2c154f00 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 -- 172.21.15.154:0/3839657740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c154f00 msgr2=0x7fab2c160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3839657740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c154f00 0x7fab2c160180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab24009d90 tx=0x7fab2400b330 comp rx=0 tx=0).stop 2024-03-14T09:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 -- 172.21.15.154:0/3839657740 shutdown_connections 2024-03-14T09:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3839657740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c154f00 0x7fab2c160180 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 -- 172.21.15.154:0/3839657740 >> 172.21.15.154:0/3839657740 conn(0x7fab2c072080 msgr2=0x7fab2c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 -- 172.21.15.154:0/3839657740 shutdown_connections 2024-03-14T09:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.786+0000 7fab31ed4700 1 -- 172.21.15.154:0/3839657740 wait complete. 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.787+0000 7fab31ed4700 1 Processor -- start 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.787+0000 7fab31ed4700 1 -- start start 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 0x7fab2c1df900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 0x7fab2c1e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1548e0 con 0x7fab2c0c8350 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0af940 con 0x7fab2c0c7f70 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1df3b0 con 0x7fab2c154f00 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57260/0 (socket says 172.21.15.154:57260) 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 -- 172.21.15.154:0/3043571211 learned_addr learned my addr 172.21.15.154:0/3043571211 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2b7fe700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 0x7fab2c1df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2bfff700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 0x7fab2c1e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 -- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 msgr2=0x7fab2c1e05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 0x7fab2c1e05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 -- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 msgr2=0x7fab2c1df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 0x7fab2c1df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 -- 172.21.15.154:0/3043571211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24009a70 con 0x7fab2c0c8350 2024-03-14T09:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2bfff700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 0x7fab2c1e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2b7fe700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 0x7fab2c1df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab2affd700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c500 tx=0x7fab1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011790 con 0x7fab2c0c8350 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0118f0 con 0x7fab2c0c8350 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c1e1900 con 0x7fab2c0c8350 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.788+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011ca0 con 0x7fab2c0c8350 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.789+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1df600 con 0x7fab2c0c8350 2024-03-14T09:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.789+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c0afaf0 con 0x7fab2c0c8350 2024-03-14T09:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.790+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fab1c01c460 con 0x7fab2c0c8350 2024-03-14T09:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.790+0000 7fab13fff700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 0x7fab1806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.790+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fab1c088780 con 0x7fab2c0c8350 2024-03-14T09:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.791+0000 7fab2b7fe700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 0x7fab1806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.791+0000 7fab2b7fe700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 0x7fab1806ccf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab20000f60 tx=0x7fab20007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.795+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fab2c0afaf0 con 0x7fab2c0c8350 2024-03-14T09:02:00.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.986+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab2c0afaf0 con 0x7fab2c0c8350 2024-03-14T09:02:00.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.986+0000 7fab13fff700 1 -- 172.21.15.154:0/3043571211 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab2c0afaf0 con 0x7fab2c0c8350 2024-03-14T09:02:00.986 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:00.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 msgr2=0x7fab1806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 0x7fab1806ccf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab20000f60 tx=0x7fab20007ce0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 msgr2=0x7fab2c153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00c500 tx=0x7fab1c00c8c0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 shutdown_connections 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab2c154f00 0x7fab2c1e05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab2c0c7f70 0x7fab2c1df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fab1806a830 0x7fab1806ccf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 --2- 172.21.15.154:0/3043571211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab2c0c8350 0x7fab2c153920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 >> 172.21.15.154:0/3043571211 conn(0x7fab2c072080 msgr2=0x7fab2c0ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 shutdown_connections 2024-03-14T09:02:00.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:00.988+0000 7fab31ed4700 1 -- 172.21.15.154:0/3043571211 wait complete. 2024-03-14T09:02:01.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:02:01.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 Processor -- start 2024-03-14T09:02:01.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 -- start start 2024-03-14T09:02:01.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48100c4af0 0x7f48100c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100c55e0 con 0x7f481015d370 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100c8290 con 0x7f481006b300 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.124+0000 7f48158f7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48100c5ac0 con 0x7f48100c4af0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48100c4af0 0x7f48100c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48100c4af0 0x7f48100c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57264/0 (socket says 172.21.15.154:57264) 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 -- 172.21.15.154:0/1706683827 learned_addr learned my addr 172.21.15.154:0/1706683827 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480f7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f481015d370 0x7f481015f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46958/0 (socket says 172.21.15.154:46958) 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 -- 172.21.15.154:0/1706683827 >> v1:172.21.15.191:6789/0 conn(0x7f481006b300 legacy=0x7f481006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 -- 172.21.15.154:0/1706683827 >> v1:172.21.15.154:6789/0 conn(0x7f481015d370 legacy=0x7f481015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 -- 172.21.15.154:0/1706683827 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810154bf0 con 0x7f48100c4af0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480dffb700 1 -- 172.21.15.154:0/1706683827 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114134796 0 0) 0x7f48100c55e0 con 0x7f481015d370 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480effd700 1 --2- 172.21.15.154:0/1706683827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48100c4af0 0x7f48100c4ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb1531f28cb951a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480dffb700 1 -- 172.21.15.154:0/1706683827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004230 con 0x7f48100c4af0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480dffb700 1 -- 172.21.15.154:0/1706683827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800004390 con 0x7f48100c4af0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.125+0000 7f480dffb700 1 -- 172.21.15.154:0/1706683827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000036a0 con 0x7f48100c4af0 2024-03-14T09:02:01.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 -- 172.21.15.154:0/1706683827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48100c4af0 msgr2=0x7f48100c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 --2- 172.21.15.154:0/1706683827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48100c4af0 0x7f48100c4ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48000098f0 tx=0x7f4800009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 -- 172.21.15.154:0/1706683827 shutdown_connections 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 --2- 172.21.15.154:0/1706683827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48100c4af0 0x7f48100c4ed0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 -- 172.21.15.154:0/1706683827 >> 172.21.15.154:0/1706683827 conn(0x7f48100abc60 msgr2=0x7f48100ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 -- 172.21.15.154:0/1706683827 shutdown_connections 2024-03-14T09:02:01.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.126+0000 7f48158f7700 1 -- 172.21.15.154:0/1706683827 wait complete. 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.127+0000 7f48158f7700 1 Processor -- start 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.127+0000 7f48158f7700 1 -- start start 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 0x7f481014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 0x7f4810158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c00a0 con 0x7f481006b300 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48101eed90 con 0x7f48100c4af0 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4810151d30 con 0x7f481015d370 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57280/0 (socket says 172.21.15.154:57280) 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 -- 172.21.15.154:0/1398672245 learned_addr learned my addr 172.21.15.154:0/1398672245 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:01.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480e7fc700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 0x7f481014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 -- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 msgr2=0x7f4810158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480f7fe700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 0x7f4810158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 0x7f4810158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 -- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 msgr2=0x7f481014f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 0x7f481014f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 -- 172.21.15.154:0/1398672245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009550 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480e7fc700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 0x7f481014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480f7fe700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 0x7f4810158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f480effd700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4800000c00 tx=0x7f4800004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800003db0 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480001f070 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48101eab00 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000b8a0 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.128+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481015c5f0 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.129+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810067bd0 con 0x7f481006b300 2024-03-14T09:02:01.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.130+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4800021030 con 0x7f481006b300 2024-03-14T09:02:01.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.130+0000 7f47f7fff700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 0x7f47fc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.130+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f480008a120 con 0x7f481006b300 2024-03-14T09:02:01.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.130+0000 7f480e7fc700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 0x7f47fc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.131+0000 7f480e7fc700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 0x7f47fc06ccf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f48100bdcd0 tx=0x7f4804009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.135+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4800055920 con 0x7f481006b300 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.336+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4810067bd0 con 0x7f481006b300 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.336+0000 7f47f7fff700 1 -- 172.21.15.154:0/1398672245 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4800058f40 con 0x7f481006b300 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:01.336 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 msgr2=0x7f47fc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 0x7f47fc06ccf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f48100bdcd0 tx=0x7f4804009040 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 msgr2=0x7f48101ee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4800000c00 tx=0x7f4800004750 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 shutdown_connections 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f481015d370 0x7f4810158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f47fc06a830 0x7f47fc06ccf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f481006b300 0x7f48101ee850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.338+0000 7f48158f7700 1 --2- 172.21.15.154:0/1398672245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48100c4af0 0x7f481014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.339+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 >> 172.21.15.154:0/1398672245 conn(0x7f48100abc60 msgr2=0x7f48100c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.339+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 shutdown_connections 2024-03-14T09:02:01.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.339+0000 7f48158f7700 1 -- 172.21.15.154:0/1398672245 wait complete. 2024-03-14T09:02:01.350 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T09:02:01.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T09:02:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.476+0000 7f49eac36700 1 Processor -- start 2024-03-14T09:02:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.476+0000 7f49eac36700 1 -- start start 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49eac36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49e40c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49eac36700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40c3130 con 0x7f49e40c2050 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49eac36700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40c1020 con 0x7f49e40c2b40 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49eac36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40c1300 con 0x7f49e40c1560 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e91d3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f49e40c2050 0x7f49e40ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46966/0 (socket says 172.21.15.154:46966) 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e91d3700 1 -- 172.21.15.154:0/2218180140 learned_addr learned my addr 172.21.15.154:0/2218180140 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e89d2700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49e40c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725113307 0 0) 0x7f49e40c3130 con 0x7f49e40c2050 2024-03-14T09:02:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d40035b0 con 0x7f49e40c2050 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.477+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1385388465 0 0) 0x7f49d40035b0 con 0x7f49e40c2050 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 msgr2=0x7f49e40c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49e40c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 >> v1:172.21.15.191:6789/0 conn(0x7f49e40c2b40 legacy=0x7f49e41547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e406d5a0 con 0x7f49e40c2050 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f49dc0019f0 con 0x7f49e40c2050 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 >> v1:172.21.15.154:6789/0 conn(0x7f49e40c2050 legacy=0x7f49e40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49d4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d4003f70 0x7f49d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4004d90 0x7f49d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40c1300 con 0x7f49d4003f70 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d400bb20 con 0x7f49d4004d90 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d400bce0 con 0x7f49e40c1560 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d4003f70 0x7f49d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e89d2700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49d4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e91d3700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4004d90 0x7f49d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 -- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 msgr2=0x7f49d4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e40c1560 0x7f49d4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 -- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4004d90 msgr2=0x7f49d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4004d90 0x7f49d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 -- 172.21.15.154:0/2218180140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40035b0 con 0x7f49d4003f70 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e3fff700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d4003f70 0x7f49d4004850 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f49d0001880 tx=0x7f49d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f225d7b7dc574db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0001370 con 0x7f49d4003f70 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0012640 con 0x7f49d4003f70 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.478+0000 7f49e37fe700 1 -- 172.21.15.154:0/2218180140 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00129f0 con 0x7f49d4003f70 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 -- 172.21.15.154:0/2218180140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d4003f70 msgr2=0x7f49d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 --2- 172.21.15.154:0/2218180140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49d4003f70 0x7f49d4004850 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f49d0001880 tx=0x7f49d4005880 comp rx=0 tx=0).stop 2024-03-14T09:02:01.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49e91d3700 1 -- 172.21.15.154:0/2218180140 reap_dead start 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 -- 172.21.15.154:0/2218180140 shutdown_connections 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 -- 172.21.15.154:0/2218180140 >> 172.21.15.154:0/2218180140 conn(0x7f49e40c3b40 msgr2=0x7f49e40c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 -- 172.21.15.154:0/2218180140 shutdown_connections 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.479+0000 7f49eac36700 1 -- 172.21.15.154:0/2218180140 wait complete. 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.480+0000 7f49eac36700 1 Processor -- start 2024-03-14T09:02:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.480+0000 7f49eac36700 1 -- start start 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4003f70 0x7f49e4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e414c090 0x7f49e41df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41df4c0 con 0x7f49e41dfe20 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41e1c90 con 0x7f49d4003f70 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49eac36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41e16d0 con 0x7f49e414c090 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57296/0 (socket says 172.21.15.154:57296) 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 -- 172.21.15.154:0/1847079185 learned_addr learned my addr 172.21.15.154:0/1847079185 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e89d2700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4003f70 0x7f49e4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e3fff700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e414c090 0x7f49e41df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 -- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e414c090 msgr2=0x7f49e41df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e414c090 0x7f49e41df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 -- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4003f70 msgr2=0x7f49e4155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4003f70 0x7f49e4155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.481+0000 7f49e91d3700 1 -- 172.21.15.154:0/1847079185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e41e19b0 con 0x7f49e41dfe20 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49e91d3700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f49e4072b10 tx=0x7f49dc00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc00ebd0 con 0x7f49e41dfe20 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc00ed30 con 0x7f49e41dfe20 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e41df0e0 con 0x7f49e41dfe20 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc013660 con 0x7f49e41dfe20 2024-03-14T09:02:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.482+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e414b4e0 con 0x7f49e41dfe20 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.483+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e41def40 con 0x7f49e41dfe20 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.483+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f49dc0137c0 con 0x7f49e41dfe20 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.483+0000 7f49e17fa700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 0x7f49c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.484+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49dc088ea0 con 0x7f49e41dfe20 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.484+0000 7f49e89d2700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 0x7f49c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.484+0000 7f49e89d2700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 0x7f49c806ccf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c60 tx=0x7f49d8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:01.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.488+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f49e41def40 con 0x7f49e41dfe20 2024-03-14T09:02:01.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:01.681+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49e41df600 con 0x7f49e41dfe20 2024-03-14T09:02:02.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.314+0000 7f49e17fa700 1 -- 172.21.15.154:0/1847079185 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49e41df600 con 0x7f49e41dfe20 2024-03-14T09:02:02.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 msgr2=0x7f49c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 0x7f49c806ccf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c60 tx=0x7f49d8009210 comp rx=0 tx=0).stop 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 msgr2=0x7f49e414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f49e4072b10 tx=0x7f49dc00e440 comp rx=0 tx=0).stop 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 shutdown_connections 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f49c806a830 0x7f49c806ccf0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c60 tx=0x7f49d8009210 comp rx=0 tx=0).stop 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e41dfe20 0x7f49e414afa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49d4003f70 0x7f49e4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 --2- 172.21.15.154:0/1847079185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49e414c090 0x7f49e41df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.318+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 >> 172.21.15.154:0/1847079185 conn(0x7f49e40c3b40 msgr2=0x7f49e4150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.319+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 shutdown_connections 2024-03-14T09:02:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.319+0000 7f49eac36700 1 -- 172.21.15.154:0/1847079185 wait complete. 2024-03-14T09:02:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2024-03-14T09:02:02.332 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:02:02.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.462+0000 7f685491e700 1 Processor -- start 2024-03-14T09:02:02.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.462+0000 7f685491e700 1 -- start start 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f685491e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f685006d3c0 0x7f68500c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f685491e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 0x7f68500c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f685491e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6850151860 con 0x7f68500c1d90 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f685491e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500c50e0 con 0x7f685006d3c0 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f685491e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6850150f30 con 0x7f68500c1830 2024-03-14T09:02:02.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f685006d3c0 0x7f68500c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f685006d3c0 0x7f68500c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57298/0 (socket says 172.21.15.154:57298) 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 -- 172.21.15.154:0/682936699 learned_addr learned my addr 172.21.15.154:0/682936699 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684ed9d700 1 --2- 172.21.15.154:0/682936699 >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 0x7f68500c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 -- 172.21.15.154:0/682936699 >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 msgr2=0x7f68500c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 --2- 172.21.15.154:0/682936699 >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 0x7f68500c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 -- 172.21.15.154:0/682936699 >> v1:172.21.15.191:6789/0 conn(0x7f68500c1d90 legacy=0x7f68500c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 -- 172.21.15.154:0/682936699 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850151210 con 0x7f685006d3c0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684ed9d700 1 --2- 172.21.15.154:0/682936699 >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 0x7f68500c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684e59c700 1 --2- 172.21.15.154:0/682936699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685006d3c0 0x7f68500c12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68400098f0 tx=0x7f6840009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a9a2b75183c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684d59a700 1 -- 172.21.15.154:0/682936699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840004230 con 0x7f685006d3c0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684d59a700 1 -- 172.21.15.154:0/682936699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840004390 con 0x7f685006d3c0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.463+0000 7f684d59a700 1 -- 172.21.15.154:0/682936699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400036a0 con 0x7f685006d3c0 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 -- 172.21.15.154:0/682936699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685006d3c0 msgr2=0x7f68500c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:02.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 --2- 172.21.15.154:0/682936699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685006d3c0 0x7f68500c12f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68400098f0 tx=0x7f6840009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 -- 172.21.15.154:0/682936699 shutdown_connections 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 --2- 172.21.15.154:0/682936699 >> v2:172.21.15.191:3300/0 conn(0x7f68500c1830 0x7f68500c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 --2- 172.21.15.154:0/682936699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f685006d3c0 0x7f68500c12f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 -- 172.21.15.154:0/682936699 >> 172.21.15.154:0/682936699 conn(0x7f685006a770 msgr2=0x7f685006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 -- 172.21.15.154:0/682936699 shutdown_connections 2024-03-14T09:02:02.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.464+0000 7f685491e700 1 -- 172.21.15.154:0/682936699 wait complete. 2024-03-14T09:02:02.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.465+0000 7f685491e700 1 Processor -- start 2024-03-14T09:02:02.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.465+0000 7f685491e700 1 -- start start 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f685006d3c0 0x7f685014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68500c1830 0x7f685013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685006c060 con 0x7f68500c1d90 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68501e8ba0 con 0x7f685006d3c0 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f685491e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685014eac0 con 0x7f68500c1830 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57314/0 (socket says 172.21.15.154:57314) 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 -- 172.21.15.154:0/1570029827 learned_addr learned my addr 172.21.15.154:0/1570029827 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684e59c700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f685006d3c0 0x7f685014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684dd9b700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68500c1830 0x7f685013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 -- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68500c1830 msgr2=0x7f685013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68500c1830 0x7f685013fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 -- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f685006d3c0 msgr2=0x7f685014d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:02.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f685006d3c0 0x7f685014d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.466+0000 7f684ed9d700 1 -- 172.21.15.154:0/1570029827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009550 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f684ed9d700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6848007ef0 tx=0x7f684800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800dd80 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685014eda0 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800f040 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848003c60 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.467+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685014f360 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.468+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6848004030 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.468+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685014f580 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.469+0000 7f683effd700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 0x7f683006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.469+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6848088c80 con 0x7f68500c1d90 2024-03-14T09:02:02.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.469+0000 7f684e59c700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 0x7f683006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:02.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.469+0000 7f684e59c700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 0x7f683006ccf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6840009130 tx=0x7f6840008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:02.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.473+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6848054200 con 0x7f68500c1d90 2024-03-14T09:02:02.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:02.670+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68500697d0 con 0x7f68500c1d90 2024-03-14T09:02:03.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.316+0000 7f683effd700 1 -- 172.21.15.154:0/1570029827 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6848057820 con 0x7f68500c1d90 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 msgr2=0x7f683006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 0x7f683006ccf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6840009130 tx=0x7f6840008030 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 msgr2=0x7f6850140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6848007ef0 tx=0x7f684800d3b0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 shutdown_connections 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f683006a830 0x7f683006ccf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68500c1d90 0x7f6850140060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f685006d3c0 0x7f685014d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 --2- 172.21.15.154:0/1570029827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68500c1830 0x7f685013fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 >> 172.21.15.154:0/1570029827 conn(0x7f685006a770 msgr2=0x7f685015ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:03.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 shutdown_connections 2024-03-14T09:02:03.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.333+0000 7f685491e700 1 -- 172.21.15.154:0/1570029827 wait complete. 2024-03-14T09:02:03.334 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2024-03-14T09:02:03.346 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T09:02:03.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.471+0000 7f5b2c7b2700 1 Processor -- start 2024-03-14T09:02:03.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.471+0000 7f5b2c7b2700 1 -- start start 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b2c7b2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5b240a9650 0x7f5b240c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b2c7b2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b24156050 con 0x7f5b241564e0 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b2c7b2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240a86f0 con 0x7f5b24156960 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b2c7b2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240c34b0 con 0x7f5b240a9650 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b29d4d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5b241564e0 0x7f5b2406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46976/0 (socket says 172.21.15.154:46976) 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.472+0000 7f5b29d4d700 1 -- 172.21.15.154:0/171314470 learned_addr learned my addr 172.21.15.154:0/171314470 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/171314470 >> v2:172.21.15.191:3300/0 conn(0x7f5b240a9650 0x7f5b240c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2954c700 1 -- 172.21.15.154:0/171314470 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794012901 0 0) 0x7f5b24156050 con 0x7f5b241564e0 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2954c700 1 -- 172.21.15.154:0/171314470 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b100035b0 con 0x7f5b241564e0 2024-03-14T09:02:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2a54e700 1 -- 172.21.15.154:0/171314470 >> v1:172.21.15.191:6789/0 conn(0x7f5b24156960 legacy=0x7f5b2415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2a54e700 1 -- 172.21.15.154:0/171314470 >> v1:172.21.15.154:6789/0 conn(0x7f5b241564e0 legacy=0x7f5b2406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2a54e700 1 -- 172.21.15.154:0/171314470 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240a8990 con 0x7f5b240a9650 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/171314470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b240c1780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009840 tx=0x7f5b14009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f51328dba58bfd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2954c700 1 -- 172.21.15.154:0/171314470 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14011830 con 0x7f5b240a9650 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2954c700 1 -- 172.21.15.154:0/171314470 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14011990 con 0x7f5b240a9650 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.473+0000 7f5b2954c700 1 -- 172.21.15.154:0/171314470 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14011d20 con 0x7f5b240a9650 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/171314470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 msgr2=0x7f5b240c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/171314470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b240c1780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009840 tx=0x7f5b14009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/171314470 shutdown_connections 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/171314470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b240c1780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/171314470 >> 172.21.15.154:0/171314470 conn(0x7f5b240c3880 msgr2=0x7f5b240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:03.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/171314470 shutdown_connections 2024-03-14T09:02:03.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.474+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/171314470 wait complete. 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.475+0000 7f5b2c7b2700 1 Processor -- start 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.475+0000 7f5b2c7b2700 1 -- start start 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b241eae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 0x7f5b24148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241ee720 con 0x7f5b241564e0 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240c5500 con 0x7f5b240a9650 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240c57e0 con 0x7f5b24156960 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57328/0 (socket says 172.21.15.154:57328) 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 -- 172.21.15.154:0/2255266567 learned_addr learned my addr 172.21.15.154:0/2255266567 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:03.479 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b241eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2ad4f700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 0x7f5b24148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 -- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 msgr2=0x7f5b24148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 0x7f5b24148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 -- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 msgr2=0x7f5b241eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b241eae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 -- 172.21.15.154:0/2255266567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140094a0 con 0x7f5b241564e0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b241eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2ad4f700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 0x7f5b24148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b29d4d700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c700 tx=0x7f5b2000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20005d70 con 0x7f5b241564e0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b200070d0 con 0x7f5b241564e0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b241ea370 con 0x7f5b241564e0 2024-03-14T09:02:03.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20007460 con 0x7f5b241564e0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.476+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b241ea930 con 0x7f5b241564e0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.477+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240697d0 con 0x7f5b241564e0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.478+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5b200075c0 con 0x7f5b241564e0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.478+0000 7f5b1affd700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 0x7f5b0806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.478+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5b200882c0 con 0x7f5b241564e0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.479+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 0x7f5b0806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.479+0000 7f5b2a54e700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 0x7f5b0806ccf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b140090b0 tx=0x7f5b1401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.483+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5b20053750 con 0x7f5b241564e0 2024-03-14T09:02:03.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.725+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5b2406b470 con 0x7f5b241564e0 2024-03-14T09:02:03.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.733+0000 7f5b1affd700 1 -- 172.21.15.154:0/2255266567 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5b20056d70 con 0x7f5b241564e0 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 msgr2=0x7f5b0806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 0x7f5b0806ccf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b140090b0 tx=0x7f5b1401c040 comp rx=0 tx=0).stop 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 msgr2=0x7f5b241ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c700 tx=0x7f5b2000cac0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 shutdown_connections 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24156960 0x7f5b24148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b240a9650 0x7f5b241eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5b0806a830 0x7f5b0806ccf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 --2- 172.21.15.154:0/2255266567 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b241564e0 0x7f5b241ee1e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.738+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 >> 172.21.15.154:0/2255266567 conn(0x7f5b240c3880 msgr2=0x7f5b2414d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:03.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.742+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 shutdown_connections 2024-03-14T09:02:03.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.742+0000 7f5b2c7b2700 1 -- 172.21.15.154:0/2255266567 wait complete. 2024-03-14T09:02:03.743 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 4 and data pool 5 2024-03-14T09:02:03.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:03.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.883+0000 7f242b528700 1 Processor -- start 2024-03-14T09:02:03.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.883+0000 7f242b528700 1 -- start start 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f242b528700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f24240c3250 0x7f242413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f242b528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f24240c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f242b528700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242413ade0 con 0x7f24240a8540 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f242b528700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242406fb30 con 0x7f24240c3250 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f242b528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242406ec30 con 0x7f24240c1780 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f24292c4700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f24240a8540 0x7f2424154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46984/0 (socket says 172.21.15.154:46984) 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.884+0000 7f24292c4700 1 -- 172.21.15.154:0/664128510 learned_addr learned my addr 172.21.15.154:0/664128510 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2428ac3700 1 --2- 172.21.15.154:0/664128510 >> v2:172.21.15.191:3300/0 conn(0x7f24240c3250 0x7f242413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f24240c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115258899 0 0) 0x7f2414001110 con 0x7f24240a8540 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24100035b0 con 0x7f24240a8540 2024-03-14T09:02:03.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021070026 0 0) 0x7f24100035b0 con 0x7f24240a8540 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 msgr2=0x7f24240c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f24240c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 >> v2:172.21.15.191:3300/0 conn(0x7f24240c3250 msgr2=0x7f242413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 --2- 172.21.15.154:0/664128510 >> v2:172.21.15.191:3300/0 conn(0x7f24240c3250 0x7f242413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242406ef10 con 0x7f24240a8540 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2414001800 con 0x7f24240a8540 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 >> v1:172.21.15.154:6789/0 conn(0x7f24240a8540 legacy=0x7f2424154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2410003a00 0x7f2410003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f2410004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2410004d90 0x7f2410005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242406fb30 con 0x7f2410004d90 2024-03-14T09:02:03.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242406ec30 con 0x7f2410003a00 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241000bac0 con 0x7f24240c1780 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2410004d90 0x7f2410005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f24292c4700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2410003a00 0x7f2410003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2428ac3700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f2410004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 -- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 msgr2=0x7f2410004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f24240c1780 0x7f2410004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 -- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2410003a00 msgr2=0x7f2410003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2410003a00 0x7f2410003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.885+0000 7f2429ac5700 1 -- 172.21.15.154:0/664128510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100035b0 con 0x7f2410004d90 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f2429ac5700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2410004d90 0x7f2410005210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f241c009fc0 tx=0x7f2410005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea2d988d066bced server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00ec30 con 0x7f2410004d90 2024-03-14T09:02:03.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00f040 con 0x7f2410004d90 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f241bfff700 1 -- 172.21.15.154:0/664128510 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c014480 con 0x7f2410004d90 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 -- 172.21.15.154:0/664128510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2410004d90 msgr2=0x7f2410005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 --2- 172.21.15.154:0/664128510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2410004d90 0x7f2410005210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f241c009fc0 tx=0x7f2410005880 comp rx=0 tx=0).stop 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f2429ac5700 1 -- 172.21.15.154:0/664128510 reap_dead start 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 -- 172.21.15.154:0/664128510 shutdown_connections 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 -- 172.21.15.154:0/664128510 >> 172.21.15.154:0/664128510 conn(0x7f24240c3880 msgr2=0x7f24240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 -- 172.21.15.154:0/664128510 shutdown_connections 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.886+0000 7f242b528700 1 -- 172.21.15.154:0/664128510 wait complete. 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 Processor -- start 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 -- start start 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 0x7f24240c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f242414fd70 0x7f2424153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241e9e50 con 0x7f24241501f0 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24241ea130 con 0x7f242414fd70 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f242b528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24241ea410 con 0x7f2410004d90 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f2429ac5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f2429ac5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57352/0 (socket says 172.21.15.154:57352) 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f2429ac5700 1 -- 172.21.15.154:0/427312400 learned_addr learned my addr 172.21.15.154:0/427312400 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f24292c4700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 0x7f24240c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.888+0000 7f2428ac3700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f242414fd70 0x7f2424153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 -- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 msgr2=0x7f24240c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 0x7f24240c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 -- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f242414fd70 msgr2=0x7f2424153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f242414fd70 0x7f2424153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:03.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 -- 172.21.15.154:0/427312400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24241ea6f0 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f24292c4700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 0x7f24240c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2429ac5700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f241c017040 tx=0x7f241c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c007000 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c003e50 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c014b50 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241ea9d0 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.889+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24240c6380 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.890+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24240697d0 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.890+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f241c023020 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.891+0000 7f2419ffb700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 0x7f240806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.891+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f241c08a1e0 con 0x7f24241501f0 2024-03-14T09:02:03.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.891+0000 7f24292c4700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 0x7f240806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.892+0000 7f24292c4700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 0x7f240806ccf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2414000c00 tx=0x7f2414009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:03.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:03.896+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f241c0557b0 con 0x7f24241501f0 2024-03-14T09:02:04.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.090+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f242406d420 con 0x7f24241501f0 2024-03-14T09:02:04.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.091+0000 7f2419ffb700 1 -- 172.21.15.154:0/427312400 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f241c058dd0 con 0x7f24241501f0 2024-03-14T09:02:04.091 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:04.091 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":31,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:03.727340+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 msgr2=0x7f240806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 0x7f240806ccf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2414000c00 tx=0x7f2414009040 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 msgr2=0x7f2424154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f241c017040 tx=0x7f241c009750 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 shutdown_connections 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f240806a830 0x7f240806ccf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24241501f0 0x7f2424154490 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2410004d90 0x7f24240c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.093+0000 7f242b528700 1 --2- 172.21.15.154:0/427312400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f242414fd70 0x7f2424153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.094+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 >> 172.21.15.154:0/427312400 conn(0x7f24240c3880 msgr2=0x7f24241395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.094+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 shutdown_connections 2024-03-14T09:02:04.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.094+0000 7f242b528700 1 -- 172.21.15.154:0/427312400 wait complete. 2024-03-14T09:02:04.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.227+0000 7f6dd87de700 1 Processor -- start 2024-03-14T09:02:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.227+0000 7f6dd87de700 1 -- start start 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.228+0000 7f6dd87de700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dd00a8540 0x7f6dd013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.228+0000 7f6dd87de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd00c3250 0x7f6dd0154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.228+0000 7f6dd87de700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd013ade0 con 0x7f6dd01550b0 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.228+0000 7f6dd87de700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd006fb30 con 0x7f6dd00a8540 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.228+0000 7f6dd87de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd006ec30 con 0x7f6dd00c3250 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dd00a8540 0x7f6dd013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6dd00a8540 0x7f6dd013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57362/0 (socket says 172.21.15.154:57362) 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 -- 172.21.15.154:0/3300857402 learned_addr learned my addr 172.21.15.154:0/3300857402 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:04.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd5d79700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd00c3250 0x7f6dd0154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 -- 172.21.15.154:0/3300857402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd00c3250 msgr2=0x7f6dd0154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd00c3250 0x7f6dd0154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 -- 172.21.15.154:0/3300857402 >> v1:172.21.15.191:6789/0 conn(0x7f6dd01550b0 legacy=0x7f6dd00c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 -- 172.21.15.154:0/3300857402 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd006ef10 con 0x7f6dd00a8540 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd657a700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00a8540 0x7f6dd013a8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00098f0 tx=0x7f6dc0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cf4f790f989c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd5578700 1 -- 172.21.15.154:0/3300857402 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004230 con 0x7f6dd00a8540 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd5578700 1 -- 172.21.15.154:0/3300857402 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0004390 con 0x7f6dd00a8540 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.229+0000 7f6dd5578700 1 -- 172.21.15.154:0/3300857402 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00036a0 con 0x7f6dd00a8540 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 -- 172.21.15.154:0/3300857402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00a8540 msgr2=0x7f6dd013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00a8540 0x7f6dd013a8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00098f0 tx=0x7f6dc0009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 -- 172.21.15.154:0/3300857402 shutdown_connections 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00a8540 0x7f6dd013a8a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 --2- 172.21.15.154:0/3300857402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd00c3250 0x7f6dd0154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 -- 172.21.15.154:0/3300857402 >> 172.21.15.154:0/3300857402 conn(0x7f6dd00c3880 msgr2=0x7f6dd00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:04.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 -- 172.21.15.154:0/3300857402 shutdown_connections 2024-03-14T09:02:04.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.230+0000 7f6dd87de700 1 -- 172.21.15.154:0/3300857402 wait complete. 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.231+0000 7f6dd87de700 1 Processor -- start 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.231+0000 7f6dd87de700 1 -- start start 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 0x7f6dd01380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 0x7f6dd01eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd006af60 con 0x7f6dd00c3250 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01ea150 con 0x7f6dd00a8540 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01ee320 con 0x7f6dd01550b0 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57372/0 (socket says 172.21.15.154:57372) 2024-03-14T09:02:04.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 -- 172.21.15.154:0/958661980 learned_addr learned my addr 172.21.15.154:0/958661980 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd657a700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 0x7f6dd01380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd6d7b700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 0x7f6dd01eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 -- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 msgr2=0x7f6dd01eab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 0x7f6dd01eab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 -- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 msgr2=0x7f6dd01380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 0x7f6dd01380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 -- 172.21.15.154:0/958661980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc0009550 con 0x7f6dd00c3250 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd657a700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 0x7f6dd01380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd6d7b700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 0x7f6dd01eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd5d79700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c820 tx=0x7f6dcc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc011800 con 0x7f6dd00c3250 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc011960 con 0x7f6dd00c3250 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.232+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd006b170 con 0x7f6dd00c3250 2024-03-14T09:02:04.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.233+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc011d10 con 0x7f6dd00c3250 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.233+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd0138630 con 0x7f6dd00c3250 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.234+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd00697d0 con 0x7f6dd00c3250 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.234+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6dcc018360 con 0x7f6dd00c3250 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.234+0000 7f6dc6ffd700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 0x7f6db406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.235+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dcc088cc0 con 0x7f6dd00c3250 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.235+0000 7f6dd657a700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 0x7f6db406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.235+0000 7f6dd657a700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 0x7f6db406ccf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009c40 tx=0x7f6dc0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.238 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.239+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6dcc054170 con 0x7f6dd00c3250 2024-03-14T09:02:04.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.436+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dd006d420 con 0x7f6dd00c3250 2024-03-14T09:02:04.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.437+0000 7f6dc6ffd700 1 -- 172.21.15.154:0/958661980 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f6dcc057790 con 0x7f6dd00c3250 2024-03-14T09:02:04.437 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:04.437 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":31,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:03.727340+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 msgr2=0x7f6db406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 0x7f6db406ccf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009c40 tx=0x7f6dc0008030 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 msgr2=0x7f6dd01ea5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c820 tx=0x7f6dcc00cbe0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 shutdown_connections 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd01550b0 0x7f6dd01eab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd00a8540 0x7f6dd01380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6db406a830 0x7f6db406ccf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 --2- 172.21.15.154:0/958661980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dd00c3250 0x7f6dd01ea5e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 >> 172.21.15.154:0/958661980 conn(0x7f6dd00c3880 msgr2=0x7f6dd0139b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.439+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 shutdown_connections 2024-03-14T09:02:04.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.440+0000 7f6dd87de700 1 -- 172.21.15.154:0/958661980 wait complete. 2024-03-14T09:02:04.452 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T09:02:04.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.576+0000 7f6b20080700 1 Processor -- start 2024-03-14T09:02:04.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 -- start start 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b181553d0 0x7f6b18158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b180bf9c0 0x7f6b180bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b18154f40 con 0x7f6b180c1d50 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b180c16a0 con 0x7f6b180bf9c0 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.577+0000 7f6b20080700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18158d80 con 0x7f6b181553d0 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b180bf9c0 0x7f6b180bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b180bf9c0 0x7f6b180bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57374/0 (socket says 172.21.15.154:57374) 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 -- 172.21.15.154:0/1054878661 learned_addr learned my addr 172.21.15.154:0/1054878661 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1de1c700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b181553d0 0x7f6b18158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 -- 172.21.15.154:0/1054878661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b181553d0 msgr2=0x7f6b18158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b181553d0 0x7f6b18158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 -- 172.21.15.154:0/1054878661 >> v1:172.21.15.191:6789/0 conn(0x7f6b180c1d50 legacy=0x7f6b180c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 -- 172.21.15.154:0/1054878661 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18154930 con 0x7f6b180bf9c0 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1e61d700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180bf9c0 0x7f6b180bfe20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20c564a47bcdb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1ce1a700 1 -- 172.21.15.154:0/1054878661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000e070 con 0x7f6b180bf9c0 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1ce1a700 1 -- 172.21.15.154:0/1054878661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1000b910 con 0x7f6b180bf9c0 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.578+0000 7f6b1ce1a700 1 -- 172.21.15.154:0/1054878661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000bcc0 con 0x7f6b180bf9c0 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 -- 172.21.15.154:0/1054878661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180bf9c0 msgr2=0x7f6b180bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180bf9c0 0x7f6b180bfe20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 -- 172.21.15.154:0/1054878661 shutdown_connections 2024-03-14T09:02:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b181553d0 0x7f6b18158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 --2- 172.21.15.154:0/1054878661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180bf9c0 0x7f6b180bfe20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 -- 172.21.15.154:0/1054878661 >> 172.21.15.154:0/1054878661 conn(0x7f6b1806a770 msgr2=0x7f6b1806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:04.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 -- 172.21.15.154:0/1054878661 shutdown_connections 2024-03-14T09:02:04.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.579+0000 7f6b20080700 1 -- 172.21.15.154:0/1054878661 wait complete. 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.580+0000 7f6b20080700 1 Processor -- start 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.580+0000 7f6b20080700 1 -- start start 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180bf9c0 0x7f6b181ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b181553d0 0x7f6b1814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1809a6d0 con 0x7f6b180c1d50 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18099a70 con 0x7f6b181553d0 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181ead70 con 0x7f6b180bf9c0 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57386/0 (socket says 172.21.15.154:57386) 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 -- 172.21.15.154:0/355577261 learned_addr learned my addr 172.21.15.154:0/355577261 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1de1c700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180bf9c0 0x7f6b181ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1e61d700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b181553d0 0x7f6b1814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 -- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180bf9c0 msgr2=0x7f6b181ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180bf9c0 0x7f6b181ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 -- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b181553d0 msgr2=0x7f6b1814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b181553d0 0x7f6b1814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 -- 172.21.15.154:0/355577261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10009b30 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b1d61b700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c4e0 tx=0x7f6b0800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080052d0 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08005430 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b180a9eb0 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.581+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080038c0 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.582+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18099c80 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.583+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b180697d0 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.583+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6b08003a20 con 0x7f6b180c1d50 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.583+0000 7f6b0e7fc700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 0x7f6afc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:04.585 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.583+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b08088070 con 0x7f6b180c1d50 2024-03-14T09:02:04.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.584+0000 7f6b1de1c700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 0x7f6afc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:04.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.584+0000 7f6b1de1c700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 0x7f6afc06ccf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009be0 tx=0x7f6b14009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:04.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.588+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6b08053520 con 0x7f6b180c1d50 2024-03-14T09:02:04.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:04.828+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6b18066550 con 0x7f6b180c1d50 2024-03-14T09:02:05.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.739+0000 7f6b0e7fc700 1 -- 172.21.15.154:0/355577261 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6b08056b40 con 0x7f6b180c1d50 2024-03-14T09:02:05.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 msgr2=0x7f6afc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 0x7f6afc06ccf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009be0 tx=0x7f6b14009210 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 msgr2=0x7f6b1806adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c4e0 tx=0x7f6b0800c7f0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 shutdown_connections 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b181553d0 0x7f6b1814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180bf9c0 0x7f6b181ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6afc06a830 0x7f6afc06ccf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 --2- 172.21.15.154:0/355577261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b180c1d50 0x7f6b1806adc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 >> 172.21.15.154:0/355577261 conn(0x7f6b1806a770 msgr2=0x7f6b18072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 shutdown_connections 2024-03-14T09:02:05.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.742+0000 7f6b20080700 1 -- 172.21.15.154:0/355577261 wait complete. 2024-03-14T09:02:05.756 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:02:05.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T09:02:05.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 Processor -- start 2024-03-14T09:02:05.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 -- start start 2024-03-14T09:02:05.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f58b414c5c0 0x7f58b4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b4072710 0x7f58b40c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b4070e10 con 0x7f58b4072290 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40c1bb0 con 0x7f58b4072710 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58bc10e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40c6170 con 0x7f58b414c5c0 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b4072710 0x7f58b40c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b4072710 0x7f58b40c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57394/0 (socket says 172.21.15.154:57394) 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b96a9700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f58b4072290 0x7f58b40c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47000/0 (socket says 172.21.15.154:47000) 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 -- 172.21.15.154:0/189452080 learned_addr learned my addr 172.21.15.154:0/189452080 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b9eaa700 1 --2- 172.21.15.154:0/189452080 >> v2:172.21.15.191:3300/0 conn(0x7f58b414c5c0 0x7f58b4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 -- 172.21.15.154:0/189452080 >> v2:172.21.15.191:3300/0 conn(0x7f58b414c5c0 msgr2=0x7f58b4152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 --2- 172.21.15.154:0/189452080 >> v2:172.21.15.191:3300/0 conn(0x7f58b414c5c0 0x7f58b4152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 -- 172.21.15.154:0/189452080 >> v1:172.21.15.154:6789/0 conn(0x7f58b4072290 legacy=0x7f58b40c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b8ea8700 1 -- 172.21.15.154:0/189452080 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36416190 0 0) 0x7f58b4070e10 con 0x7f58b4072290 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 -- 172.21.15.154:0/189452080 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b414c9a0 con 0x7f58b4072710 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58ba6ab700 1 --2- 172.21.15.154:0/189452080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072710 0x7f58b40c26e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b95bea719d930de server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b8ea8700 1 -- 172.21.15.154:0/189452080 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e070 con 0x7f58b4072710 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b8ea8700 1 -- 172.21.15.154:0/189452080 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00b980 con 0x7f58b4072710 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.888+0000 7f58b8ea8700 1 -- 172.21.15.154:0/189452080 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00bd30 con 0x7f58b4072710 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 -- 172.21.15.154:0/189452080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072710 msgr2=0x7f58b40c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 --2- 172.21.15.154:0/189452080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072710 0x7f58b40c26e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 -- 172.21.15.154:0/189452080 shutdown_connections 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 --2- 172.21.15.154:0/189452080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072710 0x7f58b40c26e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 --2- 172.21.15.154:0/189452080 >> v2:172.21.15.191:3300/0 conn(0x7f58b414c5c0 0x7f58b4152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 -- 172.21.15.154:0/189452080 >> 172.21.15.154:0/189452080 conn(0x7f58b406a770 msgr2=0x7f58b406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 -- 172.21.15.154:0/189452080 shutdown_connections 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.889+0000 7f58bc10e700 1 -- 172.21.15.154:0/189452080 wait complete. 2024-03-14T09:02:05.891 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.890+0000 7f58bc10e700 1 Processor -- start 2024-03-14T09:02:05.892 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.890+0000 7f58bc10e700 1 -- start start 2024-03-14T09:02:05.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 0x7f58b414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58b414c5c0 0x7f58b4148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41eaf40 con 0x7f58b4072290 2024-03-14T09:02:05.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41eb220 con 0x7f58b414c5c0 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58bc10e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41eb500 con 0x7f58b4072710 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57398/0 (socket says 172.21.15.154:57398) 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 -- 172.21.15.154:0/270690366 learned_addr learned my addr 172.21.15.154:0/270690366 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58ba6ab700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58b414c5c0 0x7f58b4148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b96a9700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 0x7f58b414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 -- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 msgr2=0x7f58b414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 0x7f58b414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 -- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58b414c5c0 msgr2=0x7f58b4148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58b414c5c0 0x7f58b4148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 -- 172.21.15.154:0/270690366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009b30 con 0x7f58b4072290 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b96a9700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 0x7f58b414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.891+0000 7f58b9eaa700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:05.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.892+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00125f0 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.892+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0012750 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.892+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b41eb7e0 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.892+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0012b00 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.892+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b41ebda0 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.893+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b41e7b70 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.893+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f58b0012c60 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.893+0000 7f58aa7fc700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 0x7f589806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.893+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58b0088ff0 con 0x7f58b4072290 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.894+0000 7f58b96a9700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 0x7f589806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:05.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.894+0000 7f58b96a9700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 0x7f589806ccf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58b414b9d0 tx=0x7f58a4007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:05.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:05.899+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f58b00544a0 con 0x7f58b4072290 2024-03-14T09:02:06.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.137+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f58b40a7ab0 con 0x7f58b4072290 2024-03-14T09:02:06.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.741+0000 7f58aa7fc700 1 -- 172.21.15.154:0/270690366 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58b0057ac0 con 0x7f58b4072290 2024-03-14T09:02:06.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 msgr2=0x7f589806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 0x7f589806ccf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58b414b9d0 tx=0x7f58a4007d00 comp rx=0 tx=0).stop 2024-03-14T09:02:06.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 msgr2=0x7f58b40bff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000db10 comp rx=0 tx=0).stop 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 shutdown_connections 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58b414c5c0 0x7f58b4148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f589806a830 0x7f589806ccf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b4072290 0x7f58b40bff50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 --2- 172.21.15.154:0/270690366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58b4072710 0x7f58b414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 >> 172.21.15.154:0/270690366 conn(0x7f58b406a770 msgr2=0x7f58b40a6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 shutdown_connections 2024-03-14T09:02:06.743 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.743+0000 7f58bc10e700 1 -- 172.21.15.154:0/270690366 wait complete. 2024-03-14T09:02:06.754 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T09:02:06.755 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.877+0000 7fa92e3b8700 1 Processor -- start 2024-03-14T09:02:06.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.877+0000 7fa92e3b8700 1 -- start start 2024-03-14T09:02:06.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.877+0000 7fa92e3b8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa928151e50 0x7fa9280ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:06.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.877+0000 7fa92e3b8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280c2e30 con 0x7fa9280ace20 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa92e3b8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928154c70 con 0x7fa928158870 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa92e3b8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa928154f50 con 0x7fa928151e50 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa928151e50 0x7fa9280ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa928151e50 0x7fa9280ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57408/0 (socket says 172.21.15.154:57408) 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 -- 172.21.15.154:0/2235708171 learned_addr learned my addr 172.21.15.154:0/2235708171 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa9277fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa9280ace20 0x7fa928158160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47008/0 (socket says 172.21.15.154:47008) 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 -- 172.21.15.154:0/2235708171 >> v1:172.21.15.191:6789/0 conn(0x7fa928158870 legacy=0x7fa92815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 -- 172.21.15.154:0/2235708171 >> v1:172.21.15.154:6789/0 conn(0x7fa9280ace20 legacy=0x7fa928158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 -- 172.21.15.154:0/2235708171 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928155230 con 0x7fa928151e50 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa926ffd700 1 -- 172.21.15.154:0/2235708171 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703947632 0 0) 0x7fa9280c2e30 con 0x7fa9280ace20 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa927fff700 1 --2- 172.21.15.154:0/2235708171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928151e50 0x7fa9280ac8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098f0 tx=0x7fa918009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26827fa7e910208 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa926ffd700 1 -- 172.21.15.154:0/2235708171 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918004230 con 0x7fa928151e50 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa926ffd700 1 -- 172.21.15.154:0/2235708171 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918004390 con 0x7fa928151e50 2024-03-14T09:02:06.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.878+0000 7fa926ffd700 1 -- 172.21.15.154:0/2235708171 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9180036a0 con 0x7fa928151e50 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 -- 172.21.15.154:0/2235708171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928151e50 msgr2=0x7fa9280ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/2235708171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928151e50 0x7fa9280ac8e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098f0 tx=0x7fa918009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 -- 172.21.15.154:0/2235708171 shutdown_connections 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/2235708171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928151e50 0x7fa9280ac8e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 -- 172.21.15.154:0/2235708171 >> 172.21.15.154:0/2235708171 conn(0x7fa9280c3ad0 msgr2=0x7fa9280c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 -- 172.21.15.154:0/2235708171 shutdown_connections 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.879+0000 7fa92e3b8700 1 -- 172.21.15.154:0/2235708171 wait complete. 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.880+0000 7fa92e3b8700 1 Processor -- start 2024-03-14T09:02:06.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.880+0000 7fa92e3b8700 1 -- start start 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 0x7fa928155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 0x7fa9281ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c6e40 con 0x7fa928158870 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928156430 con 0x7fa9280ace20 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92e3b8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281ee620 con 0x7fa928151e50 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57424/0 (socket says 172.21.15.154:57424) 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 -- 172.21.15.154:0/416854408 learned_addr learned my addr 172.21.15.154:0/416854408 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa927fff700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 0x7fa928155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa9277fe700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 0x7fa9281ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 -- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 msgr2=0x7fa9281ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 0x7fa9281ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 -- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 msgr2=0x7fa928155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:06.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 0x7fa928155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 -- 172.21.15.154:0/416854408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009550 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa9277fe700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 0x7fa9281ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa927fff700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 0x7fa928155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa92c955700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cc10 tx=0x7fa92000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.881+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000abb0 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.882+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000ad10 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.882+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920012660 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.882+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9280c68c0 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.882+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9281ed8e0 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.882+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9280c0db0 con 0x7fa928158870 2024-03-14T09:02:06.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.883+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa9200127c0 con 0x7fa928158870 2024-03-14T09:02:06.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.883+0000 7fa924ff9700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 0x7fa91406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:06.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.883+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9200894a0 con 0x7fa928158870 2024-03-14T09:02:06.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.884+0000 7fa927fff700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 0x7fa91406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:06.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.884+0000 7fa927fff700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 0x7fa91406ccf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa918009130 tx=0x7fa918008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:06.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:06.888+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa920054950 con 0x7fa928158870 2024-03-14T09:02:07.098 INFO:tasks.ceph.mds.c.smithi154.stderr:2024-03-14T09:02:07.099+0000 7f1b1a907700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:02:07.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.124+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9280697d0 con 0x7fa928158870 2024-03-14T09:02:07.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.125+0000 7fa924ff9700 1 -- 172.21.15.154:0/416854408 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fa920016090 con 0x7fa928158870 2024-03-14T09:02:07.123 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:07.124 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 msgr2=0x7fa91406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 0x7fa91406ccf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa918009130 tx=0x7fa918008030 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 msgr2=0x7fa92814ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa92000cc10 tx=0x7fa92000a060 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 shutdown_connections 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa91406a830 0x7fa91406ccf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa928158870 0x7fa92814ff70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9280ace20 0x7fa928155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 --2- 172.21.15.154:0/416854408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa928151e50 0x7fa9281ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 >> 172.21.15.154:0/416854408 conn(0x7fa9280c3ad0 msgr2=0x7fa9280c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 shutdown_connections 2024-03-14T09:02:07.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.127+0000 7fa92e3b8700 1 -- 172.21.15.154:0/416854408 wait complete. 2024-03-14T09:02:07.127 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 16 2024-03-14T09:02:07.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:07.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.261+0000 7fbba35d8700 1 Processor -- start 2024-03-14T09:02:07.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.261+0000 7fbba35d8700 1 -- start start 2024-03-14T09:02:07.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba35d8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbb9c14eaf0 0x7fbb9c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba35d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb9c06acb0 0x7fbb9c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba35d8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c0ac8e0 con 0x7fbb9c156330 2024-03-14T09:02:07.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba35d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c06b310 con 0x7fbb9c06acb0 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba35d8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c06de30 con 0x7fbb9c14eaf0 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb9c06acb0 0x7fbb9c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba1b75700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbb9c156330 0x7fbb9c157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47020/0 (socket says 172.21.15.154:47020) 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb9c06acb0 0x7fbb9c155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57436/0 (socket says 172.21.15.154:57436) 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba1b75700 1 -- 172.21.15.154:0/3836689496 learned_addr learned my addr 172.21.15.154:0/3836689496 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba1374700 1 --2- 172.21.15.154:0/3836689496 >> v2:172.21.15.191:3300/0 conn(0x7fbb9c14eaf0 0x7fbb9c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 -- 172.21.15.154:0/3836689496 >> v2:172.21.15.191:3300/0 conn(0x7fbb9c14eaf0 msgr2=0x7fbb9c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 --2- 172.21.15.154:0/3836689496 >> v2:172.21.15.191:3300/0 conn(0x7fbb9c14eaf0 0x7fbb9c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 -- 172.21.15.154:0/3836689496 >> v1:172.21.15.154:6789/0 conn(0x7fbb9c156330 legacy=0x7fbb9c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 -- 172.21.15.154:0/3836689496 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0acb30 con 0x7fbb9c06acb0 2024-03-14T09:02:07.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbb93fff700 1 -- 172.21.15.154:0/3836689496 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005147896 0 0) 0x7fbb9c0ac8e0 con 0x7fbb9c156330 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbba0b73700 1 --2- 172.21.15.154:0/3836689496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c155df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c070da0 tx=0x7fbb98009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f20d8f94e9a853 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.262+0000 7fbb93fff700 1 -- 172.21.15.154:0/3836689496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011830 con 0x7fbb9c06acb0 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbb93fff700 1 -- 172.21.15.154:0/3836689496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98011990 con 0x7fbb9c06acb0 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbb93fff700 1 -- 172.21.15.154:0/3836689496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011d20 con 0x7fbb9c06acb0 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 -- 172.21.15.154:0/3836689496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c06acb0 msgr2=0x7fbb9c155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3836689496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c155df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c070da0 tx=0x7fbb98009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 -- 172.21.15.154:0/3836689496 shutdown_connections 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3836689496 >> v2:172.21.15.191:3300/0 conn(0x7fbb9c14eaf0 0x7fbb9c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3836689496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c155df0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 -- 172.21.15.154:0/3836689496 >> 172.21.15.154:0/3836689496 conn(0x7fbb9c0c3880 msgr2=0x7fbb9c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 -- 172.21.15.154:0/3836689496 shutdown_connections 2024-03-14T09:02:07.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.263+0000 7fbba35d8700 1 -- 172.21.15.154:0/3836689496 wait complete. 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.264+0000 7fbba35d8700 1 Processor -- start 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.264+0000 7fbba35d8700 1 -- start start 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb9c14eaf0 0x7fbb9c0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c14fd70 con 0x7fbb9c156330 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c14f8e0 con 0x7fbb9c06acb0 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba35d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c151ce0 con 0x7fbb9c14eaf0 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57440/0 (socket says 172.21.15.154:57440) 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 -- 172.21.15.154:0/3645771169 learned_addr learned my addr 172.21.15.154:0/3645771169 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1374700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba0b73700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb9c14eaf0 0x7fbb9c0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 -- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb9c14eaf0 msgr2=0x7fbb9c0c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb9c14eaf0 0x7fbb9c0c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 -- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 msgr2=0x7fbb9c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 -- 172.21.15.154:0/3645771169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980094a0 con 0x7fbb9c156330 2024-03-14T09:02:07.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1374700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.265+0000 7fbba1b75700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb94002230 tx=0x7fbb9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.266+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9400ac20 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.266+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9400ada0 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.266+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c151fc0 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.266+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94007310 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.266+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c14f290 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.267+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c14fa50 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.267+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbb94007470 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.267+0000 7fbb91ffb700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 0x7fbb8006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.268+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb94088c30 con 0x7fbb9c156330 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.268+0000 7fbba1374700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 0x7fbb8006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.268+0000 7fbba1374700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 0x7fbb8006ccf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009c30 tx=0x7fbb8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.272+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbb94054050 con 0x7fbb9c156330 2024-03-14T09:02:07.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.470+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb9c0697d0 con 0x7fbb9c156330 2024-03-14T09:02:07.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.471+0000 7fbb91ffb700 1 -- 172.21.15.154:0/3645771169 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16149 (secure 0 0 0) 0x7fbb94018020 con 0x7fbb9c156330 2024-03-14T09:02:07.470 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:07.470 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:04.729742+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:07.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 msgr2=0x7fbb8006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 0x7fbb8006ccf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009c30 tx=0x7fbb8c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 msgr2=0x7fbb9c0c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb94002230 tx=0x7fbb9400a060 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 shutdown_connections 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbb8006a830 0x7fbb8006ccf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c156330 0x7fbb9c0c2a00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb9c06acb0 0x7fbb9c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 --2- 172.21.15.154:0/3645771169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb9c14eaf0 0x7fbb9c0c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 >> 172.21.15.154:0/3645771169 conn(0x7fbb9c0c3880 msgr2=0x7fbb9c14c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 shutdown_connections 2024-03-14T09:02:07.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.473+0000 7fbba35d8700 1 -- 172.21.15.154:0/3645771169 wait complete. 2024-03-14T09:02:07.485 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:02:07.485 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:07.606 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.608+0000 7f095fa01700 1 Processor -- start 2024-03-14T09:02:07.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.608+0000 7f095fa01700 1 -- start start 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095fa01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09580a8540 0x7f095813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095fa01700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09580c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095fa01700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095813ade0 con 0x7f09580c3250 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095fa01700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095806fb30 con 0x7f09580a8540 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095fa01700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095806ec30 con 0x7f09580c1780 2024-03-14T09:02:07.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09580a8540 0x7f095813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09580a8540 0x7f095813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57446/0 (socket says 172.21.15.154:57446) 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 -- 172.21.15.154:0/733322493 learned_addr learned my addr 172.21.15.154:0/733322493 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095df9e700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09580c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 -- 172.21.15.154:0/733322493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 msgr2=0x7f09580c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09580c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 -- 172.21.15.154:0/733322493 >> v1:172.21.15.191:6789/0 conn(0x7f09580c3250 legacy=0x7f0958154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 -- 172.21.15.154:0/733322493 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095806ef10 con 0x7f09580a8540 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.609+0000 7f095d79d700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580a8540 0x7f095813a8a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09540098f0 tx=0x7f0954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218d0c55e015f6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f094ffff700 1 -- 172.21.15.154:0/733322493 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400b8a0 con 0x7f09580a8540 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f094ffff700 1 -- 172.21.15.154:0/733322493 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095400ba00 con 0x7f09580a8540 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f094ffff700 1 -- 172.21.15.154:0/733322493 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400bd90 con 0x7f09580a8540 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 -- 172.21.15.154:0/733322493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580a8540 msgr2=0x7f095813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580a8540 0x7f095813a8a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09540098f0 tx=0x7f0954009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 -- 172.21.15.154:0/733322493 shutdown_connections 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09580c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 --2- 172.21.15.154:0/733322493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580a8540 0x7f095813a8a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 -- 172.21.15.154:0/733322493 >> 172.21.15.154:0/733322493 conn(0x7f09580c3880 msgr2=0x7f09580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 -- 172.21.15.154:0/733322493 shutdown_connections 2024-03-14T09:02:07.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.610+0000 7f095fa01700 1 -- 172.21.15.154:0/733322493 wait complete. 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 Processor -- start 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 -- start start 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09580a8540 0x7f09580c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09581537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958139180 con 0x7f09580c3250 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958154220 con 0x7f09580a8540 2024-03-14T09:02:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095fa01700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0958152a70 con 0x7f09580c1780 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095df9e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095df9e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57452/0 (socket says 172.21.15.154:57452) 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095df9e700 1 -- 172.21.15.154:0/3006782972 learned_addr learned my addr 172.21.15.154:0/3006782972 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095cf9c700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09581537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.612+0000 7f095d79d700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09580a8540 0x7f09580c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 -- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 msgr2=0x7f09581537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09581537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 -- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09580a8540 msgr2=0x7f09580c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09580a8540 0x7f09580c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 -- 172.21.15.154:0/3006782972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009550 con 0x7f09580c3250 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095df9e700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0950007e90 tx=0x7f095000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000ecd0 con 0x7f09580c3250 2024-03-14T09:02:07.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000ee50 con 0x7f09580c3250 2024-03-14T09:02:07.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09581eafe0 con 0x7f09580c3250 2024-03-14T09:02:07.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950014660 con 0x7f09580c3250 2024-03-14T09:02:07.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.613+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958152fa0 con 0x7f09580c3250 2024-03-14T09:02:07.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.614+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09580697d0 con 0x7f09580c3250 2024-03-14T09:02:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.615+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f09500147c0 con 0x7f09580c3250 2024-03-14T09:02:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.616+0000 7f094dffb700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 0x7f093c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.616+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0950089d20 con 0x7f09580c3250 2024-03-14T09:02:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.616+0000 7f095d79d700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 0x7f093c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.617+0000 7f095d79d700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 0x7f093c06ccf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0954014040 tx=0x7f0954008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.620+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f09500551b0 con 0x7f09580c3250 2024-03-14T09:02:07.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.811+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0958155900 con 0x7f093c06a830 2024-03-14T09:02:07.817 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.818+0000 7f094dffb700 1 -- 172.21.15.154:0/3006782972 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253808 (secure 0 0 0) 0x7f095400bb70 con 0x7f093c06a830 2024-03-14T09:02:07.821 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 msgr2=0x7f093c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 0x7f093c06ccf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0954014040 tx=0x7f0954008030 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 msgr2=0x7f0958153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0950007e90 tx=0x7f095000e3c0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 shutdown_connections 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f093c06a830 0x7f093c06ccf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09580c3250 0x7f0958153ce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09580a8540 0x7f09580c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 --2- 172.21.15.154:0/3006782972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09580c1780 0x7f09581537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 >> 172.21.15.154:0/3006782972 conn(0x7f09580c3880 msgr2=0x7f095806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 shutdown_connections 2024-03-14T09:02:07.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.821+0000 7f095fa01700 1 -- 172.21.15.154:0/3006782972 wait complete. 2024-03-14T09:02:07.825 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:07.832 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-03-14T09:02:07.529105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":3727360,"data_stored":2117972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.216357"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318584+0000","last_change":"2024-03-14T09:02:02.318584+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318584+0000","last_undegraded":"2024-03-14T09:02:02.318584+0000","last_fullsized":"2024-03-14T09:02:02.318584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316530+0000","last_change":"2024-03-14T09:02:02.316530+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316530+0000","last_undegraded":"2024-03-14T09:02:02.316530+0000","last_fullsized":"2024-03-14T09:02:02.316530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318325+0000","last_change":"2024-03-14T09:02:02.318325+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318325+0000","last_undegraded":"2024-03-14T09:02:02.318325+0000","last_fullsized":"2024-03-14T09:02:02.318325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316429+0000","last_change":"2024-03-14T09:02:02.316429+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316429+0000","last_undegraded":"2024-03-14T09:02:02.316429+0000","last_fullsized":"2024-03-14T09:02:02.316429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314864+0000","last_change":"2024-03-14T09:02:02.314864+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314864+0000","last_undegraded":"2024-03-14T09:02:02.314864+0000","last_fullsized":"2024-03-14T09:02:02.314864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314789+0000","last_change":"2024-03-14T09:02:02.314789+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314789+0000","last_undegraded":"2024-03-14T09:02:02.314789+0000","last_fullsized":"2024-03-14T09:02:02.314789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315066+0000","last_change":"2024-03-14T09:02:02.315066+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315066+0000","last_undegraded":"2024-03-14T09:02:02.315066+0000","last_fullsized":"2024-03-14T09:02:02.315066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314316+0000","last_change":"2024-03-14T09:02:02.314316+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314316+0000","last_undegraded":"2024-03-14T09:02:02.314316+0000","last_fullsized":"2024-03-14T09:02:02.314316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316096+0000","last_change":"2024-03-14T09:02:02.316096+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316096+0000","last_undegraded":"2024-03-14T09:02:02.316096+0000","last_fullsized":"2024-03-14T09:02:02.316096+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317947+0000","last_change":"2024-03-14T09:02:02.317947+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317947+0000","last_undegraded":"2024-03-14T09:02:02.317947+0000","last_fullsized":"2024-03-14T09:02:02.317947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314264+0000","last_change":"2024-03-14T09:02:02.314264+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314264+0000","last_undegraded":"2024-03-14T09:02:02.314264+0000","last_fullsized":"2024-03-14T09:02:02.314264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314831+0000","last_change":"2024-03-14T09:02:02.314831+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314831+0000","last_undegraded":"2024-03-14T09:02:02.314831+0000","last_fullsized":"2024-03-14T09:02:02.314831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318177+0000","last_change":"2024-03-14T09:02:02.318177+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318177+0000","last_undegraded":"2024-03-14T09:02:02.318177+0000","last_fullsized":"2024-03-14T09:02:02.318177+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314253+0000","last_change":"2024-03-14T09:02:02.314253+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314253+0000","last_undegraded":"2024-03-14T09:02:02.314253+0000","last_fullsized":"2024-03-14T09:02:02.314253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317486+0000","last_change":"2024-03-14T09:02:02.317486+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317486+0000","last_undegraded":"2024-03-14T09:02:02.317486+0000","last_fullsized":"2024-03-14T09:02:02.317486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315703+0000","last_change":"2024-03-14T09:02:02.315703+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315703+0000","last_undegraded":"2024-03-14T09:02:02.315703+0000","last_fullsized":"2024-03-14T09:02:02.315703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315047+0000","last_change":"2024-03-14T09:02:02.315047+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315047+0000","last_undegraded":"2024-03-14T09:02:02.315047+0000","last_fullsized":"2024-03-14T09:02:02.315047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315410+0000","last_change":"2024-03-14T09:02:02.315410+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315410+0000","last_undegraded":"2024-03-14T09:02:02.315410+0000","last_fullsized":"2024-03-14T09:02:02.315410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316059+0000","last_change":"2024-03-14T09:02:02.316059+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316059+0000","last_undegraded":"2024-03-14T09:02:02.316059+0000","last_fullsized":"2024-03-14T09:02:02.316059+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318974+0000","last_change":"2024-03-14T09:02:02.318974+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318974+0000","last_undegraded":"2024-03-14T09:02:02.318974+0000","last_fullsized":"2024-03-14T09:02:02.318974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315628+0000","last_change":"2024-03-14T09:02:02.315628+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315628+0000","last_undegraded":"2024-03-14T09:02:02.315628+0000","last_fullsized":"2024-03-14T09:02:02.315628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317807+0000","last_change":"2024-03-14T09:02:02.317807+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317807+0000","last_undegraded":"2024-03-14T09:02:02.317807+0000","last_fullsized":"2024-03-14T09:02:02.317807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317028+0000","last_change":"2024-03-14T09:02:02.317028+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317028+0000","last_undegraded":"2024-03-14T09:02:02.317028+0000","last_fullsized":"2024-03-14T09:02:02.317028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315541+0000","last_change":"2024-03-14T09:02:02.315541+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315541+0000","last_undegraded":"2024-03-14T09:02:02.315541+0000","last_fullsized":"2024-03-14T09:02:02.315541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316921+0000","last_change":"2024-03-14T09:02:02.316921+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316921+0000","last_undegraded":"2024-03-14T09:02:02.316921+0000","last_fullsized":"2024-03-14T09:02:02.316921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315501+0000","last_change":"2024-03-14T09:02:02.315501+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315501+0000","last_undegraded":"2024-03-14T09:02:02.315501+0000","last_fullsized":"2024-03-14T09:02:02.315501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318647+0000","last_change":"2024-03-14T09:02:02.318647+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318647+0000","last_undegraded":"2024-03-14T09:02:02.318647+0000","last_fullsized":"2024-03-14T09:02:02.318647+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314837+0000","last_change":"2024-03-14T09:02:02.314837+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314837+0000","last_undegraded":"2024-03-14T09:02:02.314837+0000","last_fullsized":"2024-03-14T09:02:02.314837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314833+0000","last_change":"2024-03-14T09:02:02.314833+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314833+0000","last_undegraded":"2024-03-14T09:02:02.314833+0000","last_fullsized":"2024-03-14T09:02:02.314833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318564+0000","last_change":"2024-03-14T09:02:02.318564+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318564+0000","last_undegraded":"2024-03-14T09:02:02.318564+0000","last_fullsized":"2024-03-14T09:02:02.318564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-14T09:01:58.051810+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:58.051810+0000","last_peered":"2024-03-14T09:01:58.051810+0000","last_clean":"2024-03-14T09:01:58.051810+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:58.051810+0000","last_undegraded":"2024-03-14T09:01:58.051810+0000","last_fullsized":"2024-03-14T09:01:58.051810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":710431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":704614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:07.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:07.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.995+0000 7f56e902b700 1 Processor -- start 2024-03-14T09:02:07.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.995+0000 7f56e902b700 1 -- start start 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e902b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e4158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e902b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56e40c1d50 0x7f56e40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e902b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e4154f40 con 0x7f56e40bf9c0 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e902b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e4158d80 con 0x7f56e40c1d50 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e902b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4154930 con 0x7f56e41553d0 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56e40c1d50 0x7f56e40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e359e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f56e40bf9c0 0x7f56e40c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47028/0 (socket says 172.21.15.154:47028) 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56e40c1d50 0x7f56e40c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57464/0 (socket says 172.21.15.154:57464) 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e359e700 1 -- 172.21.15.154:0/1690236560 learned_addr learned my addr 172.21.15.154:0/1690236560 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e2d9d700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:07.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 -- 172.21.15.154:0/1690236560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 msgr2=0x7f56e4158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e4158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 -- 172.21.15.154:0/1690236560 >> v1:172.21.15.154:6789/0 conn(0x7f56e40bf9c0 legacy=0x7f56e40c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 -- 172.21.15.154:0/1690236560 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e40c57e0 con 0x7f56e40c1d50 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e1d9b700 1 -- 172.21.15.154:0/1690236560 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817261159 0 0) 0x7f56e4154f40 con 0x7f56e40bf9c0 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e259c700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e40c0f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009840 tx=0x7f56d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9be817cad8c462 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e1d9b700 1 -- 172.21.15.154:0/1690236560 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011830 con 0x7f56e40c1d50 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e1d9b700 1 -- 172.21.15.154:0/1690236560 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d4011990 con 0x7f56e40c1d50 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.996+0000 7f56e1d9b700 1 -- 172.21.15.154:0/1690236560 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4011d20 con 0x7f56e40c1d50 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 -- 172.21.15.154:0/1690236560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 msgr2=0x7f56e40c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e40c0f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009840 tx=0x7f56d4009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 -- 172.21.15.154:0/1690236560 shutdown_connections 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e4158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 --2- 172.21.15.154:0/1690236560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e40c0f90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 -- 172.21.15.154:0/1690236560 >> 172.21.15.154:0/1690236560 conn(0x7f56e406a770 msgr2=0x7f56e406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:07.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 -- 172.21.15.154:0/1690236560 shutdown_connections 2024-03-14T09:02:07.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.997+0000 7f56e902b700 1 -- 172.21.15.154:0/1690236560 wait complete. 2024-03-14T09:02:07.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.998+0000 7f56e902b700 1 Processor -- start 2024-03-14T09:02:07.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.998+0000 7f56e902b700 1 -- start start 2024-03-14T09:02:08.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56e40bf9c0 0x7f56e4071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41eaef0 con 0x7f56e40c1d50 2024-03-14T09:02:08.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40abea0 con 0x7f56e40bf9c0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e902b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e414f340 con 0x7f56e41553d0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57470/0 (socket says 172.21.15.154:57470) 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e259c700 1 -- 172.21.15.154:0/4112168226 learned_addr learned my addr 172.21.15.154:0/4112168226 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e2d9d700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56e40bf9c0 0x7f56e4071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:07.999+0000 7f56e359e700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 -- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 msgr2=0x7f56e414ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e414ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 -- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56e40bf9c0 msgr2=0x7f56e4071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56e40bf9c0 0x7f56e4071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 -- 172.21.15.154:0/4112168226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40094a0 con 0x7f56e40c1d50 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e259c700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56e40740c0 tx=0x7f56d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40038a0 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d401f070 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e4099e90 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4003c20 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.000+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4072050 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.001+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e40697d0 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.001+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f56d401a430 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.002+0000 7f56cb7fe700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 0x7f56d006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.002+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56d4021070 con 0x7f56e40c1d50 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.002+0000 7f56e2d9d700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 0x7f56d006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.002+0000 7f56e2d9d700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 0x7f56d006cd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009930 tx=0x7f56d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.006+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f56d4055bc0 con 0x7f56e40c1d50 2024-03-14T09:02:08.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.194+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56e41557b0 con 0x7f56d006a860 2024-03-14T09:02:08.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.201+0000 7f56cb7fe700 1 -- 172.21.15.154:0/4112168226 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253808 (secure 0 0 0) 0x7f56d8009e00 con 0x7f56d006a860 2024-03-14T09:02:08.204 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:08.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 msgr2=0x7f56d006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 0x7f56d006cd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009930 tx=0x7f56d8008040 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 msgr2=0x7f56e414e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56e40740c0 tx=0x7f56d4004750 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 shutdown_connections 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56e41553d0 0x7f56e414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56e40bf9c0 0x7f56e4071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56d006a860 0x7f56d006cd20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 --2- 172.21.15.154:0/4112168226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56e40c1d50 0x7f56e414e8c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 >> 172.21.15.154:0/4112168226 conn(0x7f56e406a770 msgr2=0x7f56e4153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 shutdown_connections 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.204+0000 7f56e902b700 1 -- 172.21.15.154:0/4112168226 wait complete. 2024-03-14T09:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:08.216 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-03-14T09:02:07.529105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":3727360,"data_stored":2117972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.216357"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318584+0000","last_change":"2024-03-14T09:02:02.318584+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318584+0000","last_undegraded":"2024-03-14T09:02:02.318584+0000","last_fullsized":"2024-03-14T09:02:02.318584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316530+0000","last_change":"2024-03-14T09:02:02.316530+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316530+0000","last_undegraded":"2024-03-14T09:02:02.316530+0000","last_fullsized":"2024-03-14T09:02:02.316530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318325+0000","last_change":"2024-03-14T09:02:02.318325+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318325+0000","last_undegraded":"2024-03-14T09:02:02.318325+0000","last_fullsized":"2024-03-14T09:02:02.318325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316429+0000","last_change":"2024-03-14T09:02:02.316429+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316429+0000","last_undegraded":"2024-03-14T09:02:02.316429+0000","last_fullsized":"2024-03-14T09:02:02.316429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314864+0000","last_change":"2024-03-14T09:02:02.314864+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314864+0000","last_undegraded":"2024-03-14T09:02:02.314864+0000","last_fullsized":"2024-03-14T09:02:02.314864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314789+0000","last_change":"2024-03-14T09:02:02.314789+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314789+0000","last_undegraded":"2024-03-14T09:02:02.314789+0000","last_fullsized":"2024-03-14T09:02:02.314789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315066+0000","last_change":"2024-03-14T09:02:02.315066+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315066+0000","last_undegraded":"2024-03-14T09:02:02.315066+0000","last_fullsized":"2024-03-14T09:02:02.315066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314316+0000","last_change":"2024-03-14T09:02:02.314316+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314316+0000","last_undegraded":"2024-03-14T09:02:02.314316+0000","last_fullsized":"2024-03-14T09:02:02.314316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316096+0000","last_change":"2024-03-14T09:02:02.316096+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316096+0000","last_undegraded":"2024-03-14T09:02:02.316096+0000","last_fullsized":"2024-03-14T09:02:02.316096+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317947+0000","last_change":"2024-03-14T09:02:02.317947+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317947+0000","last_undegraded":"2024-03-14T09:02:02.317947+0000","last_fullsized":"2024-03-14T09:02:02.317947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314264+0000","last_change":"2024-03-14T09:02:02.314264+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314264+0000","last_undegraded":"2024-03-14T09:02:02.314264+0000","last_fullsized":"2024-03-14T09:02:02.314264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314831+0000","last_change":"2024-03-14T09:02:02.314831+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314831+0000","last_undegraded":"2024-03-14T09:02:02.314831+0000","last_fullsized":"2024-03-14T09:02:02.314831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318177+0000","last_change":"2024-03-14T09:02:02.318177+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318177+0000","last_undegraded":"2024-03-14T09:02:02.318177+0000","last_fullsized":"2024-03-14T09:02:02.318177+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314253+0000","last_change":"2024-03-14T09:02:02.314253+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314253+0000","last_undegraded":"2024-03-14T09:02:02.314253+0000","last_fullsized":"2024-03-14T09:02:02.314253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317486+0000","last_change":"2024-03-14T09:02:02.317486+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317486+0000","last_undegraded":"2024-03-14T09:02:02.317486+0000","last_fullsized":"2024-03-14T09:02:02.317486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315703+0000","last_change":"2024-03-14T09:02:02.315703+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315703+0000","last_undegraded":"2024-03-14T09:02:02.315703+0000","last_fullsized":"2024-03-14T09:02:02.315703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315047+0000","last_change":"2024-03-14T09:02:02.315047+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315047+0000","last_undegraded":"2024-03-14T09:02:02.315047+0000","last_fullsized":"2024-03-14T09:02:02.315047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315410+0000","last_change":"2024-03-14T09:02:02.315410+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315410+0000","last_undegraded":"2024-03-14T09:02:02.315410+0000","last_fullsized":"2024-03-14T09:02:02.315410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316059+0000","last_change":"2024-03-14T09:02:02.316059+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316059+0000","last_undegraded":"2024-03-14T09:02:02.316059+0000","last_fullsized":"2024-03-14T09:02:02.316059+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318974+0000","last_change":"2024-03-14T09:02:02.318974+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318974+0000","last_undegraded":"2024-03-14T09:02:02.318974+0000","last_fullsized":"2024-03-14T09:02:02.318974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315628+0000","last_change":"2024-03-14T09:02:02.315628+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315628+0000","last_undegraded":"2024-03-14T09:02:02.315628+0000","last_fullsized":"2024-03-14T09:02:02.315628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317807+0000","last_change":"2024-03-14T09:02:02.317807+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317807+0000","last_undegraded":"2024-03-14T09:02:02.317807+0000","last_fullsized":"2024-03-14T09:02:02.317807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317028+0000","last_change":"2024-03-14T09:02:02.317028+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317028+0000","last_undegraded":"2024-03-14T09:02:02.317028+0000","last_fullsized":"2024-03-14T09:02:02.317028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315541+0000","last_change":"2024-03-14T09:02:02.315541+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315541+0000","last_undegraded":"2024-03-14T09:02:02.315541+0000","last_fullsized":"2024-03-14T09:02:02.315541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316921+0000","last_change":"2024-03-14T09:02:02.316921+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316921+0000","last_undegraded":"2024-03-14T09:02:02.316921+0000","last_fullsized":"2024-03-14T09:02:02.316921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315501+0000","last_change":"2024-03-14T09:02:02.315501+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315501+0000","last_undegraded":"2024-03-14T09:02:02.315501+0000","last_fullsized":"2024-03-14T09:02:02.315501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318647+0000","last_change":"2024-03-14T09:02:02.318647+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318647+0000","last_undegraded":"2024-03-14T09:02:02.318647+0000","last_fullsized":"2024-03-14T09:02:02.318647+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314837+0000","last_change":"2024-03-14T09:02:02.314837+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314837+0000","last_undegraded":"2024-03-14T09:02:02.314837+0000","last_fullsized":"2024-03-14T09:02:02.314837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314833+0000","last_change":"2024-03-14T09:02:02.314833+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314833+0000","last_undegraded":"2024-03-14T09:02:02.314833+0000","last_fullsized":"2024-03-14T09:02:02.314833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318564+0000","last_change":"2024-03-14T09:02:02.318564+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318564+0000","last_undegraded":"2024-03-14T09:02:02.318564+0000","last_fullsized":"2024-03-14T09:02:02.318564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-14T09:01:58.051810+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:58.051810+0000","last_peered":"2024-03-14T09:01:58.051810+0000","last_clean":"2024-03-14T09:01:58.051810+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:58.051810+0000","last_undegraded":"2024-03-14T09:01:58.051810+0000","last_fullsized":"2024-03-14T09:01:58.051810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":710431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":704614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:08.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:08.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.379+0000 7f5decb0c700 1 Processor -- start 2024-03-14T09:02:08.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.379+0000 7f5decb0c700 1 -- start start 2024-03-14T09:02:08.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.380+0000 7f5decb0c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5de813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.380+0000 7f5decb0c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5de80c1780 0x7f5de80c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.380+0000 7f5decb0c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de813ade0 con 0x7f5de80c3250 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.380+0000 7f5decb0c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de806fb30 con 0x7f5de80c1780 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.380+0000 7f5decb0c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de806ec30 con 0x7f5de80a8540 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5de80c3250 0x7f5de8154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47030/0 (socket says 172.21.15.154:47030) 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de5d9b700 1 -- 172.21.15.154:0/371706049 learned_addr learned my addr 172.21.15.154:0/371706049 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/371706049 >> v2:172.21.15.191:3300/0 conn(0x7f5de80c1780 0x7f5de80c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880848059 0 0) 0x7f5dd0001110 con 0x7f5de80c3250 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de659c700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5de813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd40035b0 con 0x7f5de80c3250 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 706177524 0 0) 0x7f5dd40035b0 con 0x7f5de80c3250 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 msgr2=0x7f5de813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5de813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 >> v2:172.21.15.191:3300/0 conn(0x7f5de80c1780 msgr2=0x7f5de80c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:08.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 --2- 172.21.15.154:0/371706049 >> v2:172.21.15.191:3300/0 conn(0x7f5de80c1780 0x7f5de80c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de806ef10 con 0x7f5de80c3250 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f5dd0001800 con 0x7f5de80c3250 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 >> v1:172.21.15.154:6789/0 conn(0x7f5de80c3250 legacy=0x7f5de8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd4003a00 0x7f5dd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5dd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5dd4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de806fb30 con 0x7f5dd4004d90 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de806ec30 con 0x7f5dd4003a00 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd400ba90 con 0x7f5de80a8540 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5dd4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de659c700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd4003a00 0x7f5dd4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.381+0000 7f5de5d9b700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5dd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 -- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 msgr2=0x7f5dd4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de80a8540 0x7f5dd4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 -- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd4003a00 msgr2=0x7f5dd4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dd4003a00 0x7f5dd4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 -- 172.21.15.154:0/371706049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40035b0 con 0x7f5dd4004d90 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5dd4005210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009fc0 tx=0x7f5dd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6650ee46fd803d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de000ec30 con 0x7f5dd4004d90 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de000f040 con 0x7f5dd4004d90 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de559a700 1 -- 172.21.15.154:0/371706049 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0014480 con 0x7f5dd4004d90 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5decb0c700 1 -- 172.21.15.154:0/371706049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 msgr2=0x7f5dd4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5decb0c700 1 --2- 172.21.15.154:0/371706049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5dd4005210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009fc0 tx=0x7f5dd4005880 comp rx=0 tx=0).stop 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.382+0000 7f5de6d9d700 1 -- 172.21.15.154:0/371706049 reap_dead start 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.383+0000 7f5decb0c700 1 -- 172.21.15.154:0/371706049 shutdown_connections 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.383+0000 7f5decb0c700 1 -- 172.21.15.154:0/371706049 >> 172.21.15.154:0/371706049 conn(0x7f5de80c3880 msgr2=0x7f5de80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.383+0000 7f5decb0c700 1 -- 172.21.15.154:0/371706049 shutdown_connections 2024-03-14T09:02:08.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.383+0000 7f5decb0c700 1 -- 172.21.15.154:0/371706049 wait complete. 2024-03-14T09:02:08.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.384+0000 7f5decb0c700 1 Processor -- start 2024-03-14T09:02:08.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.384+0000 7f5decb0c700 1 -- start start 2024-03-14T09:02:08.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.384+0000 7f5decb0c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.384+0000 7f5decb0c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 0x7f5de8151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 0x7f5de81e4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de81e7b40 con 0x7f5dd4004d90 2024-03-14T09:02:08.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de8150dc0 con 0x7f5de81e75a0 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80a9e50 con 0x7f5de81e3ea0 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57474/0 (socket says 172.21.15.154:57474) 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 -- 172.21.15.154:0/345492522 learned_addr learned my addr 172.21.15.154:0/345492522 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de5d9b700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 0x7f5de8151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 0x7f5de81e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 -- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 msgr2=0x7f5de81e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 0x7f5de81e4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 -- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 msgr2=0x7f5de8151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 0x7f5de8151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 -- 172.21.15.154:0/345492522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8151140 con 0x7f5dd4004d90 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de6d9d700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 0x7f5de81e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de5d9b700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 0x7f5de8151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:08.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5de659c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80119d0 tx=0x7f5dd8011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800f8e0 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800fa40 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de81513e0 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800fdd0 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.385+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de8152310 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.386+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de80697d0 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.387+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5dd8014540 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.387+0000 7f5ddeffd700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 0x7f5dc806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.387+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dd808afe0 con 0x7f5dd4004d90 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.387+0000 7f5de5d9b700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 0x7f5dc806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.388+0000 7f5de5d9b700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 0x7f5dc806ccf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0000c00 tx=0x7f5dd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.391+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5dd8056490 con 0x7f5dd4004d90 2024-03-14T09:02:08.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.581+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5de8155900 con 0x7f5dc806a830 2024-03-14T09:02:08.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.588+0000 7f5ddeffd700 1 -- 172.21.15.154:0/345492522 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253808 (secure 0 0 0) 0x7f5dd0007030 con 0x7f5dc806a830 2024-03-14T09:02:08.591 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 msgr2=0x7f5dc806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 0x7f5dc806ccf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0000c00 tx=0x7f5dd0009040 comp rx=0 tx=0).stop 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 msgr2=0x7f5de80acaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80119d0 tx=0x7f5dd8011d90 comp rx=0 tx=0).stop 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 shutdown_connections 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5de81e3ea0 0x7f5de81e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5dc806a830 0x7f5dc806ccf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd4004d90 0x7f5de80acaf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 --2- 172.21.15.154:0/345492522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de81e75a0 0x7f5de8151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 >> 172.21.15.154:0/345492522 conn(0x7f5de80c3880 msgr2=0x7f5de80c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 shutdown_connections 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.591+0000 7f5decb0c700 1 -- 172.21.15.154:0/345492522 wait complete. 2024-03-14T09:02:08.594 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:08.603 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-03-14T09:02:07.529105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":3727360,"data_stored":2117972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.216357"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318584+0000","last_change":"2024-03-14T09:02:02.318584+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318584+0000","last_undegraded":"2024-03-14T09:02:02.318584+0000","last_fullsized":"2024-03-14T09:02:02.318584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316530+0000","last_change":"2024-03-14T09:02:02.316530+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316530+0000","last_undegraded":"2024-03-14T09:02:02.316530+0000","last_fullsized":"2024-03-14T09:02:02.316530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318325+0000","last_change":"2024-03-14T09:02:02.318325+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318325+0000","last_undegraded":"2024-03-14T09:02:02.318325+0000","last_fullsized":"2024-03-14T09:02:02.318325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316429+0000","last_change":"2024-03-14T09:02:02.316429+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316429+0000","last_undegraded":"2024-03-14T09:02:02.316429+0000","last_fullsized":"2024-03-14T09:02:02.316429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314864+0000","last_change":"2024-03-14T09:02:02.314864+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314864+0000","last_undegraded":"2024-03-14T09:02:02.314864+0000","last_fullsized":"2024-03-14T09:02:02.314864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314789+0000","last_change":"2024-03-14T09:02:02.314789+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314789+0000","last_undegraded":"2024-03-14T09:02:02.314789+0000","last_fullsized":"2024-03-14T09:02:02.314789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315066+0000","last_change":"2024-03-14T09:02:02.315066+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315066+0000","last_undegraded":"2024-03-14T09:02:02.315066+0000","last_fullsized":"2024-03-14T09:02:02.315066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314316+0000","last_change":"2024-03-14T09:02:02.314316+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314316+0000","last_undegraded":"2024-03-14T09:02:02.314316+0000","last_fullsized":"2024-03-14T09:02:02.314316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316096+0000","last_change":"2024-03-14T09:02:02.316096+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316096+0000","last_undegraded":"2024-03-14T09:02:02.316096+0000","last_fullsized":"2024-03-14T09:02:02.316096+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317947+0000","last_change":"2024-03-14T09:02:02.317947+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317947+0000","last_undegraded":"2024-03-14T09:02:02.317947+0000","last_fullsized":"2024-03-14T09:02:02.317947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314264+0000","last_change":"2024-03-14T09:02:02.314264+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314264+0000","last_undegraded":"2024-03-14T09:02:02.314264+0000","last_fullsized":"2024-03-14T09:02:02.314264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314831+0000","last_change":"2024-03-14T09:02:02.314831+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314831+0000","last_undegraded":"2024-03-14T09:02:02.314831+0000","last_fullsized":"2024-03-14T09:02:02.314831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318177+0000","last_change":"2024-03-14T09:02:02.318177+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318177+0000","last_undegraded":"2024-03-14T09:02:02.318177+0000","last_fullsized":"2024-03-14T09:02:02.318177+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314253+0000","last_change":"2024-03-14T09:02:02.314253+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314253+0000","last_undegraded":"2024-03-14T09:02:02.314253+0000","last_fullsized":"2024-03-14T09:02:02.314253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317486+0000","last_change":"2024-03-14T09:02:02.317486+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317486+0000","last_undegraded":"2024-03-14T09:02:02.317486+0000","last_fullsized":"2024-03-14T09:02:02.317486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315703+0000","last_change":"2024-03-14T09:02:02.315703+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315703+0000","last_undegraded":"2024-03-14T09:02:02.315703+0000","last_fullsized":"2024-03-14T09:02:02.315703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315047+0000","last_change":"2024-03-14T09:02:02.315047+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315047+0000","last_undegraded":"2024-03-14T09:02:02.315047+0000","last_fullsized":"2024-03-14T09:02:02.315047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315410+0000","last_change":"2024-03-14T09:02:02.315410+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315410+0000","last_undegraded":"2024-03-14T09:02:02.315410+0000","last_fullsized":"2024-03-14T09:02:02.315410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316059+0000","last_change":"2024-03-14T09:02:02.316059+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316059+0000","last_undegraded":"2024-03-14T09:02:02.316059+0000","last_fullsized":"2024-03-14T09:02:02.316059+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318974+0000","last_change":"2024-03-14T09:02:02.318974+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318974+0000","last_undegraded":"2024-03-14T09:02:02.318974+0000","last_fullsized":"2024-03-14T09:02:02.318974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315628+0000","last_change":"2024-03-14T09:02:02.315628+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315628+0000","last_undegraded":"2024-03-14T09:02:02.315628+0000","last_fullsized":"2024-03-14T09:02:02.315628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317807+0000","last_change":"2024-03-14T09:02:02.317807+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317807+0000","last_undegraded":"2024-03-14T09:02:02.317807+0000","last_fullsized":"2024-03-14T09:02:02.317807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317028+0000","last_change":"2024-03-14T09:02:02.317028+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317028+0000","last_undegraded":"2024-03-14T09:02:02.317028+0000","last_fullsized":"2024-03-14T09:02:02.317028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315541+0000","last_change":"2024-03-14T09:02:02.315541+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315541+0000","last_undegraded":"2024-03-14T09:02:02.315541+0000","last_fullsized":"2024-03-14T09:02:02.315541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316921+0000","last_change":"2024-03-14T09:02:02.316921+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316921+0000","last_undegraded":"2024-03-14T09:02:02.316921+0000","last_fullsized":"2024-03-14T09:02:02.316921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315501+0000","last_change":"2024-03-14T09:02:02.315501+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315501+0000","last_undegraded":"2024-03-14T09:02:02.315501+0000","last_fullsized":"2024-03-14T09:02:02.315501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318647+0000","last_change":"2024-03-14T09:02:02.318647+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318647+0000","last_undegraded":"2024-03-14T09:02:02.318647+0000","last_fullsized":"2024-03-14T09:02:02.318647+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314837+0000","last_change":"2024-03-14T09:02:02.314837+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314837+0000","last_undegraded":"2024-03-14T09:02:02.314837+0000","last_fullsized":"2024-03-14T09:02:02.314837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314833+0000","last_change":"2024-03-14T09:02:02.314833+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314833+0000","last_undegraded":"2024-03-14T09:02:02.314833+0000","last_fullsized":"2024-03-14T09:02:02.314833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318564+0000","last_change":"2024-03-14T09:02:02.318564+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318564+0000","last_undegraded":"2024-03-14T09:02:02.318564+0000","last_fullsized":"2024-03-14T09:02:02.318564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-14T09:01:58.051810+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:58.051810+0000","last_peered":"2024-03-14T09:01:58.051810+0000","last_clean":"2024-03-14T09:01:58.051810+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:58.051810+0000","last_undegraded":"2024-03-14T09:01:58.051810+0000","last_fullsized":"2024-03-14T09:01:58.051810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":710431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":704614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:08.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-14T09:02:08.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-14T09:02:08.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-14T09:02:08.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318584+0000', 'last_change': '2024-03-14T09:02:02.318584+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318584+0000', 'last_undegraded': '2024-03-14T09:02:02.318584+0000', 'last_fullsized': '2024-03-14T09:02:02.318584+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.316530+0000', 'last_change': '2024-03-14T09:02:02.316530+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.316530+0000', 'last_undegraded': '2024-03-14T09:02:02.316530+0000', 'last_fullsized': '2024-03-14T09:02:02.316530+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314247+0000', 'last_change': '2024-03-14T09:02:02.314247+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314247+0000', 'last_undegraded': '2024-03-14T09:02:02.314247+0000', 'last_fullsized': '2024-03-14T09:02:02.314247+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318325+0000', 'last_change': '2024-03-14T09:02:02.318325+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318325+0000', 'last_undegraded': '2024-03-14T09:02:02.318325+0000', 'last_fullsized': '2024-03-14T09:02:02.318325+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.316429+0000', 'last_change': '2024-03-14T09:02:02.316429+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.316429+0000', 'last_undegraded': '2024-03-14T09:02:02.316429+0000', 'last_fullsized': '2024-03-14T09:02:02.316429+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314864+0000', 'last_change': '2024-03-14T09:02:02.314864+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314864+0000', 'last_undegraded': '2024-03-14T09:02:02.314864+0000', 'last_fullsized': '2024-03-14T09:02:02.314864+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314789+0000', 'last_change': '2024-03-14T09:02:02.314789+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314789+0000', 'last_undegraded': '2024-03-14T09:02:02.314789+0000', 'last_fullsized': '2024-03-14T09:02:02.314789+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315066+0000', 'last_change': '2024-03-14T09:02:02.315066+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315066+0000', 'last_undegraded': '2024-03-14T09:02:02.315066+0000', 'last_fullsized': '2024-03-14T09:02:02.315066+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-14T09:02:08.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314316+0000', 'last_change': '2024-03-14T09:02:02.314316+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314316+0000', 'last_undegraded': '2024-03-14T09:02:02.314316+0000', 'last_fullsized': '2024-03-14T09:02:02.314316+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.316096+0000', 'last_change': '2024-03-14T09:02:02.316096+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.316096+0000', 'last_undegraded': '2024-03-14T09:02:02.316096+0000', 'last_fullsized': '2024-03-14T09:02:02.316096+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2024-03-14T09:02:08.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.317947+0000', 'last_change': '2024-03-14T09:02:02.317947+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.317947+0000', 'last_undegraded': '2024-03-14T09:02:02.317947+0000', 'last_fullsized': '2024-03-14T09:02:02.317947+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314264+0000', 'last_change': '2024-03-14T09:02:02.314264+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314264+0000', 'last_undegraded': '2024-03-14T09:02:02.314264+0000', 'last_fullsized': '2024-03-14T09:02:02.314264+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314831+0000', 'last_change': '2024-03-14T09:02:02.314831+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314831+0000', 'last_undegraded': '2024-03-14T09:02:02.314831+0000', 'last_fullsized': '2024-03-14T09:02:02.314831+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-14T09:02:08.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318177+0000', 'last_change': '2024-03-14T09:02:02.318177+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318177+0000', 'last_undegraded': '2024-03-14T09:02:02.318177+0000', 'last_fullsized': '2024-03-14T09:02:02.318177+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314247+0000', 'last_change': '2024-03-14T09:02:02.314247+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314247+0000', 'last_undegraded': '2024-03-14T09:02:02.314247+0000', 'last_fullsized': '2024-03-14T09:02:02.314247+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314253+0000', 'last_change': '2024-03-14T09:02:02.314253+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314253+0000', 'last_undegraded': '2024-03-14T09:02:02.314253+0000', 'last_fullsized': '2024-03-14T09:02:02.314253+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-14T09:02:08.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.317486+0000', 'last_change': '2024-03-14T09:02:02.317486+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.317486+0000', 'last_undegraded': '2024-03-14T09:02:02.317486+0000', 'last_fullsized': '2024-03-14T09:02:02.317486+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315703+0000', 'last_change': '2024-03-14T09:02:02.315703+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315703+0000', 'last_undegraded': '2024-03-14T09:02:02.315703+0000', 'last_fullsized': '2024-03-14T09:02:02.315703+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315047+0000', 'last_change': '2024-03-14T09:02:02.315047+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315047+0000', 'last_undegraded': '2024-03-14T09:02:02.315047+0000', 'last_fullsized': '2024-03-14T09:02:02.315047+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315410+0000', 'last_change': '2024-03-14T09:02:02.315410+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315410+0000', 'last_undegraded': '2024-03-14T09:02:02.315410+0000', 'last_fullsized': '2024-03-14T09:02:02.315410+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.316059+0000', 'last_change': '2024-03-14T09:02:02.316059+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.316059+0000', 'last_undegraded': '2024-03-14T09:02:02.316059+0000', 'last_fullsized': '2024-03-14T09:02:02.316059+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318974+0000', 'last_change': '2024-03-14T09:02:02.318974+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318974+0000', 'last_undegraded': '2024-03-14T09:02:02.318974+0000', 'last_fullsized': '2024-03-14T09:02:02.318974+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315628+0000', 'last_change': '2024-03-14T09:02:02.315628+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315628+0000', 'last_undegraded': '2024-03-14T09:02:02.315628+0000', 'last_fullsized': '2024-03-14T09:02:02.315628+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.317807+0000', 'last_change': '2024-03-14T09:02:02.317807+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.317807+0000', 'last_undegraded': '2024-03-14T09:02:02.317807+0000', 'last_fullsized': '2024-03-14T09:02:02.317807+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.317028+0000', 'last_change': '2024-03-14T09:02:02.317028+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.317028+0000', 'last_undegraded': '2024-03-14T09:02:02.317028+0000', 'last_fullsized': '2024-03-14T09:02:02.317028+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:02:08.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315541+0000', 'last_change': '2024-03-14T09:02:02.315541+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315541+0000', 'last_undegraded': '2024-03-14T09:02:02.315541+0000', 'last_fullsized': '2024-03-14T09:02:02.315541+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.316921+0000', 'last_change': '2024-03-14T09:02:02.316921+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.316921+0000', 'last_undegraded': '2024-03-14T09:02:02.316921+0000', 'last_fullsized': '2024-03-14T09:02:02.316921+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2024-03-14T09:02:08.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.315501+0000', 'last_change': '2024-03-14T09:02:02.315501+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.315501+0000', 'last_undegraded': '2024-03-14T09:02:02.315501+0000', 'last_fullsized': '2024-03-14T09:02:02.315501+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318647+0000', 'last_change': '2024-03-14T09:02:02.318647+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318647+0000', 'last_undegraded': '2024-03-14T09:02:02.318647+0000', 'last_fullsized': '2024-03-14T09:02:02.318647+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314837+0000', 'last_change': '2024-03-14T09:02:02.314837+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314837+0000', 'last_undegraded': '2024-03-14T09:02:02.314837+0000', 'last_fullsized': '2024-03-14T09:02:02.314837+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-14T09:02:08.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.314833+0000', 'last_change': '2024-03-14T09:02:02.314833+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.314833+0000', 'last_undegraded': '2024-03-14T09:02:02.314833+0000', 'last_fullsized': '2024-03-14T09:02:02.314833+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-14T09:02:03.311957+0000', 'last_change': '2024-03-14T09:02:03.311957+0000', 'last_active': '2024-03-14T09:02:03.311957+0000', 'last_peered': '2024-03-14T09:02:03.311957+0000', 'last_clean': '2024-03-14T09:02:03.311957+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:03.311957+0000', 'last_undegraded': '2024-03-14T09:02:03.311957+0000', 'last_fullsized': '2024-03-14T09:02:03.311957+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:03.311957+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2024-03-14T09:02:08.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:02:02.318564+0000', 'last_change': '2024-03-14T09:02:02.318564+0000', 'last_active': '2024-03-14T09:02:02.306938+0000', 'last_peered': '2024-03-14T09:02:02.306938+0000', 'last_clean': '2024-03-14T09:02:02.306938+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:02:02.318564+0000', 'last_undegraded': '2024-03-14T09:02:02.318564+0000', 'last_fullsized': '2024-03-14T09:02:02.318564+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'last_clean_scrub_stamp': '2024-03-14T09:02:02.306938+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:02:08.625 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-14T09:02:08.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.764+0000 7f6d730b0700 1 Processor -- start 2024-03-14T09:02:08.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.764+0000 7f6d730b0700 1 -- start start 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d730b0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c0a8540 0x7f6d6c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d730b0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d6c0c3250 0x7f6d6c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d730b0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c13ade0 con 0x7f6d6c1550b0 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d730b0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c06fb30 con 0x7f6d6c0c3250 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d730b0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c06ec30 con 0x7f6d6c0a8540 2024-03-14T09:02:08.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d6c0c3250 0x7f6d6c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d7164d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6d6c1550b0 0x7f6d6c0c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47040/0 (socket says 172.21.15.154:47040) 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d7164d700 1 -- 172.21.15.154:0/1099312988 learned_addr learned my addr 172.21.15.154:0/1099312988 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d70e4c700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c0a8540 0x7f6d6c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6bfff700 1 -- 172.21.15.154:0/1099312988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c0a8540 msgr2=0x7f6d6c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6bfff700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c0a8540 0x7f6d6c13a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6bfff700 1 -- 172.21.15.154:0/1099312988 >> v1:172.21.15.154:6789/0 conn(0x7f6d6c1550b0 legacy=0x7f6d6c0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6bfff700 1 -- 172.21.15.154:0/1099312988 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c06ef10 con 0x7f6d6c0c3250 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.765+0000 7f6d6b7fe700 1 -- 172.21.15.154:0/1099312988 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608854472 0 0) 0x7f6d64001490 con 0x7f6d6c1550b0 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d6bfff700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c154b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009840 tx=0x7f6d58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d6f46123314543 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d6b7fe700 1 -- 172.21.15.154:0/1099312988 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58011830 con 0x7f6d6c0c3250 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d6b7fe700 1 -- 172.21.15.154:0/1099312988 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58011990 con 0x7f6d6c0c3250 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d6b7fe700 1 -- 172.21.15.154:0/1099312988 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58011d20 con 0x7f6d6c0c3250 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 -- 172.21.15.154:0/1099312988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0c3250 msgr2=0x7f6d6c154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c154b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009840 tx=0x7f6d58009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 -- 172.21.15.154:0/1099312988 shutdown_connections 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c0a8540 0x7f6d6c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 --2- 172.21.15.154:0/1099312988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c154b70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 -- 172.21.15.154:0/1099312988 >> 172.21.15.154:0/1099312988 conn(0x7f6d6c0c3880 msgr2=0x7f6d6c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 -- 172.21.15.154:0/1099312988 shutdown_connections 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.766+0000 7f6d730b0700 1 -- 172.21.15.154:0/1099312988 wait complete. 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.767+0000 7f6d730b0700 1 Processor -- start 2024-03-14T09:02:08.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 -- start start 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c1550b0 0x7f6d6c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c153b90 con 0x7f6d6c0a8540 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c1eaa40 con 0x7f6d6c0c3250 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d730b0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c1ead20 con 0x7f6d6c1550b0 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d70e4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d70e4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57482/0 (socket says 172.21.15.154:57482) 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.768+0000 7f6d70e4c700 1 -- 172.21.15.154:0/3441434155 learned_addr learned my addr 172.21.15.154:0/3441434155 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d7164d700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c1550b0 0x7f6d6c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d6bfff700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 -- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c1550b0 msgr2=0x7f6d6c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c1550b0 0x7f6d6c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 -- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d6c0c3250 msgr2=0x7f6d6c0c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c0c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 -- 172.21.15.154:0/3441434155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580094a0 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d70e4c700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c7f0 tx=0x7f6d6000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60004220 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60004380 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c1eb000 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d600036a0 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.769+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c0c5e30 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.770+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c0697d0 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.771+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6d60003800 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.771+0000 7f6d697fa700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 0x7f6d5006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.771+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d60088c10 con 0x7f6d6c0a8540 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.772+0000 7f6d6bfff700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 0x7f6d5006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:08.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.772+0000 7f6d6bfff700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 0x7f6d5006ccf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6d580090b0 tx=0x7f6d58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:08.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:08.775+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6d60054110 con 0x7f6d6c0a8540 2024-03-14T09:02:09.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.048+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6d6c153e70 con 0x7f6d6c0a8540 2024-03-14T09:02:09.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.049+0000 7f6d697fa700 1 -- 172.21.15.154:0/3441434155 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3302 (secure 0 0 0) 0x7f6d6001ed00 con 0x7f6d6c0a8540 2024-03-14T09:02:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:09.049 INFO:teuthology.orchestra.run.smithi154.stdout:{"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":67,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1710406872,"num_in_osds":8,"osd_in_since":1710406864,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225120256,"bytes_avail":772868993024,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4956}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-03-14T09:01:53.525940+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 msgr2=0x7f6d5006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 0x7f6d5006ccf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6d580090b0 tx=0x7f6d58008030 comp rx=0 tx=0).stop 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 msgr2=0x7f6d6c1ea3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c7f0 tx=0x7f6d6000cb00 comp rx=0 tx=0).stop 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 shutdown_connections 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d6c1550b0 0x7f6d6c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d5006a830 0x7f6d5006ccf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d6c0a8540 0x7f6d6c1ea3f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 --2- 172.21.15.154:0/3441434155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d6c0c3250 0x7f6d6c0c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 >> 172.21.15.154:0/3441434155 conn(0x7f6d6c0c3880 msgr2=0x7f6d6c139ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:09.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 shutdown_connections 2024-03-14T09:02:09.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.051+0000 7f6d730b0700 1 -- 172.21.15.154:0/3441434155 wait complete. 2024-03-14T09:02:09.063 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-14T09:02:09.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:09.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.184+0000 7fb74c957700 1 Processor -- start 2024-03-14T09:02:09.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.184+0000 7fb74c957700 1 -- start start 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.185+0000 7fb74c957700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.185+0000 7fb74c957700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.185+0000 7fb74c957700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7480ac8e0 con 0x7fb74814eaf0 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.185+0000 7fb74c957700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74806b310 con 0x7fb748155df0 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.185+0000 7fb74c957700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74806de30 con 0x7fb74815d270 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb745d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb745d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:43358/0 (socket says 172.21.15.154:43358) 2024-03-14T09:02:09.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:43814/0 (socket says 172.21.15.154:43814) 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb74659c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fb74814eaf0 0x7fb74806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:51184/0 (socket says 172.21.15.154:51184) 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb745d9b700 1 -- 172.21.15.154:0/2889349683 learned_addr learned my addr 172.21.15.154:0/2889349683 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 -- 172.21.15.154:0/2889349683 >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 msgr2=0x7fb748156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 --2- 172.21.15.154:0/2889349683 >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 -- 172.21.15.154:0/2889349683 >> v1:172.21.15.191:6789/0 conn(0x7fb74814eaf0 legacy=0x7fb74806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 -- 172.21.15.154:0/2889349683 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480acb30 con 0x7fb74815d270 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb745d9b700 1 --2- 172.21.15.154:0/2889349683 >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb746d9d700 1 --2- 172.21.15.154:0/2889349683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb740009d90 tx=0x7fb74000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1759c6963092c49c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb74559a700 1 -- 172.21.15.154:0/2889349683 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000e070 con 0x7fb74815d270 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb74559a700 1 -- 172.21.15.154:0/2889349683 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74000bb20 con 0x7fb74815d270 2024-03-14T09:02:09.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.186+0000 7fb74559a700 1 -- 172.21.15.154:0/2889349683 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740012430 con 0x7fb74815d270 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 -- 172.21.15.154:0/2889349683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 msgr2=0x7fb74815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 --2- 172.21.15.154:0/2889349683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb740009d90 tx=0x7fb74000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 -- 172.21.15.154:0/2889349683 shutdown_connections 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 --2- 172.21.15.154:0/2889349683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb74815d270 0x7fb74815f660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 --2- 172.21.15.154:0/2889349683 >> v2:172.21.15.191:3300/0 conn(0x7fb748155df0 0x7fb748156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 -- 172.21.15.154:0/2889349683 >> 172.21.15.154:0/2889349683 conn(0x7fb7480c3880 msgr2=0x7fb7480c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:09.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 -- 172.21.15.154:0/2889349683 shutdown_connections 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.187+0000 7fb74c957700 1 -- 172.21.15.154:0/2889349683 wait complete. 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.188+0000 7fb74c957700 1 Processor -- start 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.188+0000 7fb74c957700 1 -- start start 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb74814eaf0 0x7fb7481447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 0x7fb74814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74814fe80 con 0x7fb74815d270 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb748144cf0 con 0x7fb74814eaf0 2024-03-14T09:02:09.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74c957700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74814f590 con 0x7fb748155df0 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb746d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57494/0 (socket says 172.21.15.154:57494) 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb746d9d700 1 -- 172.21.15.154:0/2299730700 learned_addr learned my addr 172.21.15.154:0/2299730700 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb745d9b700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 0x7fb74814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.189+0000 7fb74659c700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb74814eaf0 0x7fb7481447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 -- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 msgr2=0x7fb74814ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 0x7fb74814ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 -- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb74814eaf0 msgr2=0x7fb7481447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb74814eaf0 0x7fb7481447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 -- 172.21.15.154:0/2299730700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c009550 con 0x7fb74815d270 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb745d9b700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 0x7fb74814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb746d9d700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb740014040 tx=0x7fb740004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740012430 con 0x7fb74815d270 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740020070 con 0x7fb74815d270 2024-03-14T09:02:09.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740009a70 con 0x7fb74815d270 2024-03-14T09:02:09.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740003ea0 con 0x7fb74815d270 2024-03-14T09:02:09.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.190+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74814f830 con 0x7fb74815d270 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.191+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74814fff0 con 0x7fb74815d270 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.191+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb740004280 con 0x7fb74815d270 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.192+0000 7fb736ffd700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 0x7fb72806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.192+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb740022070 con 0x7fb74815d270 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.192+0000 7fb74659c700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 0x7fb72806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:09.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.192+0000 7fb74659c700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 0x7fb72806ccf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb738000f60 tx=0x7fb738007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:09.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.196+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb740055e60 con 0x7fb74815d270 2024-03-14T09:02:09.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.392+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb74806e080 con 0x7fb72806a830 2024-03-14T09:02:09.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.399+0000 7fb736ffd700 1 -- 172.21.15.154:0/2299730700 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253808 (secure 0 0 0) 0x7fb738004870 con 0x7fb72806a830 2024-03-14T09:02:09.402 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 msgr2=0x7fb72806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 0x7fb72806ccf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb738000f60 tx=0x7fb738007d30 comp rx=0 tx=0).stop 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 msgr2=0x7fb74814a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb740014040 tx=0x7fb740004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 shutdown_connections 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb72806a830 0x7fb72806ccf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb74815d270 0x7fb74814a190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb74814eaf0 0x7fb7481447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 --2- 172.21.15.154:0/2299730700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb748155df0 0x7fb74814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 >> 172.21.15.154:0/2299730700 conn(0x7fb7480c3880 msgr2=0x7fb748071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 shutdown_connections 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:09.403+0000 7fb74c957700 1 -- 172.21.15.154:0/2299730700 wait complete. 2024-03-14T09:02:09.405 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:09.415 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-03-14T09:02:07.529105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219844,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754876,"statfs":{"total":773094113280,"available":772868993024,"internally_reserved":0,"allocated":3727360,"data_stored":2117972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.216357"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318584+0000","last_change":"2024-03-14T09:02:02.318584+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318584+0000","last_undegraded":"2024-03-14T09:02:02.318584+0000","last_fullsized":"2024-03-14T09:02:02.318584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316530+0000","last_change":"2024-03-14T09:02:02.316530+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316530+0000","last_undegraded":"2024-03-14T09:02:02.316530+0000","last_fullsized":"2024-03-14T09:02:02.316530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318325+0000","last_change":"2024-03-14T09:02:02.318325+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318325+0000","last_undegraded":"2024-03-14T09:02:02.318325+0000","last_fullsized":"2024-03-14T09:02:02.318325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316429+0000","last_change":"2024-03-14T09:02:02.316429+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316429+0000","last_undegraded":"2024-03-14T09:02:02.316429+0000","last_fullsized":"2024-03-14T09:02:02.316429+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314864+0000","last_change":"2024-03-14T09:02:02.314864+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314864+0000","last_undegraded":"2024-03-14T09:02:02.314864+0000","last_fullsized":"2024-03-14T09:02:02.314864+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314789+0000","last_change":"2024-03-14T09:02:02.314789+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314789+0000","last_undegraded":"2024-03-14T09:02:02.314789+0000","last_fullsized":"2024-03-14T09:02:02.314789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315066+0000","last_change":"2024-03-14T09:02:02.315066+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315066+0000","last_undegraded":"2024-03-14T09:02:02.315066+0000","last_fullsized":"2024-03-14T09:02:02.315066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314316+0000","last_change":"2024-03-14T09:02:02.314316+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314316+0000","last_undegraded":"2024-03-14T09:02:02.314316+0000","last_fullsized":"2024-03-14T09:02:02.314316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316096+0000","last_change":"2024-03-14T09:02:02.316096+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316096+0000","last_undegraded":"2024-03-14T09:02:02.316096+0000","last_fullsized":"2024-03-14T09:02:02.316096+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317947+0000","last_change":"2024-03-14T09:02:02.317947+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317947+0000","last_undegraded":"2024-03-14T09:02:02.317947+0000","last_fullsized":"2024-03-14T09:02:02.317947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314264+0000","last_change":"2024-03-14T09:02:02.314264+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314264+0000","last_undegraded":"2024-03-14T09:02:02.314264+0000","last_fullsized":"2024-03-14T09:02:02.314264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314831+0000","last_change":"2024-03-14T09:02:02.314831+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314831+0000","last_undegraded":"2024-03-14T09:02:02.314831+0000","last_fullsized":"2024-03-14T09:02:02.314831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318177+0000","last_change":"2024-03-14T09:02:02.318177+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318177+0000","last_undegraded":"2024-03-14T09:02:02.318177+0000","last_fullsized":"2024-03-14T09:02:02.318177+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314247+0000","last_change":"2024-03-14T09:02:02.314247+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314247+0000","last_undegraded":"2024-03-14T09:02:02.314247+0000","last_fullsized":"2024-03-14T09:02:02.314247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314253+0000","last_change":"2024-03-14T09:02:02.314253+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314253+0000","last_undegraded":"2024-03-14T09:02:02.314253+0000","last_fullsized":"2024-03-14T09:02:02.314253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317486+0000","last_change":"2024-03-14T09:02:02.317486+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317486+0000","last_undegraded":"2024-03-14T09:02:02.317486+0000","last_fullsized":"2024-03-14T09:02:02.317486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315703+0000","last_change":"2024-03-14T09:02:02.315703+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315703+0000","last_undegraded":"2024-03-14T09:02:02.315703+0000","last_fullsized":"2024-03-14T09:02:02.315703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315047+0000","last_change":"2024-03-14T09:02:02.315047+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315047+0000","last_undegraded":"2024-03-14T09:02:02.315047+0000","last_fullsized":"2024-03-14T09:02:02.315047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315410+0000","last_change":"2024-03-14T09:02:02.315410+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315410+0000","last_undegraded":"2024-03-14T09:02:02.315410+0000","last_fullsized":"2024-03-14T09:02:02.315410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316059+0000","last_change":"2024-03-14T09:02:02.316059+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316059+0000","last_undegraded":"2024-03-14T09:02:02.316059+0000","last_fullsized":"2024-03-14T09:02:02.316059+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318974+0000","last_change":"2024-03-14T09:02:02.318974+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318974+0000","last_undegraded":"2024-03-14T09:02:02.318974+0000","last_fullsized":"2024-03-14T09:02:02.318974+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315628+0000","last_change":"2024-03-14T09:02:02.315628+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315628+0000","last_undegraded":"2024-03-14T09:02:02.315628+0000","last_fullsized":"2024-03-14T09:02:02.315628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317807+0000","last_change":"2024-03-14T09:02:02.317807+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317807+0000","last_undegraded":"2024-03-14T09:02:02.317807+0000","last_fullsized":"2024-03-14T09:02:02.317807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.317028+0000","last_change":"2024-03-14T09:02:02.317028+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.317028+0000","last_undegraded":"2024-03-14T09:02:02.317028+0000","last_fullsized":"2024-03-14T09:02:02.317028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315541+0000","last_change":"2024-03-14T09:02:02.315541+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315541+0000","last_undegraded":"2024-03-14T09:02:02.315541+0000","last_fullsized":"2024-03-14T09:02:02.315541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.316921+0000","last_change":"2024-03-14T09:02:02.316921+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.316921+0000","last_undegraded":"2024-03-14T09:02:02.316921+0000","last_fullsized":"2024-03-14T09:02:02.316921+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.315501+0000","last_change":"2024-03-14T09:02:02.315501+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.315501+0000","last_undegraded":"2024-03-14T09:02:02.315501+0000","last_fullsized":"2024-03-14T09:02:02.315501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318647+0000","last_change":"2024-03-14T09:02:02.318647+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318647+0000","last_undegraded":"2024-03-14T09:02:02.318647+0000","last_fullsized":"2024-03-14T09:02:02.318647+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314837+0000","last_change":"2024-03-14T09:02:02.314837+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314837+0000","last_undegraded":"2024-03-14T09:02:02.314837+0000","last_fullsized":"2024-03-14T09:02:02.314837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.314833+0000","last_change":"2024-03-14T09:02:02.314833+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.314833+0000","last_undegraded":"2024-03-14T09:02:02.314833+0000","last_fullsized":"2024-03-14T09:02:02.314833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-14T09:02:03.311957+0000","last_change":"2024-03-14T09:02:03.311957+0000","last_active":"2024-03-14T09:02:03.311957+0000","last_peered":"2024-03-14T09:02:03.311957+0000","last_clean":"2024-03-14T09:02:03.311957+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:03.311957+0000","last_undegraded":"2024-03-14T09:02:03.311957+0000","last_fullsized":"2024-03-14T09:02:03.311957+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-03-14T09:02:02.318564+0000","last_change":"2024-03-14T09:02:02.318564+0000","last_active":"2024-03-14T09:02:02.306938+0000","last_peered":"2024-03-14T09:02:02.306938+0000","last_clean":"2024-03-14T09:02:02.306938+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:02:02.318564+0000","last_undegraded":"2024-03-14T09:02:02.318564+0000","last_fullsized":"2024-03-14T09:02:02.318564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-14T09:01:58.051810+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:01:58.051810+0000","last_peered":"2024-03-14T09:01:58.051810+0000","last_clean":"2024-03-14T09:01:58.051810+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:01:58.051810+0000","last_undegraded":"2024-03-14T09:01:58.051810+0000","last_fullsized":"2024-03-14T09:01:58.051810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":710431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":704614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":311296,"data_stored":114246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:12.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:12.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 Processor -- start 2024-03-14T09:02:12.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 -- start start 2024-03-14T09:02:12.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1cd006c670 0x7f1cd014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd00a84d0 0x7f1cd00a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1cd006d5f0 0x7f1cd006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00a9890 con 0x7f1cd006d5f0 2024-03-14T09:02:12.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00ab2f0 con 0x7f1cd006c670 2024-03-14T09:02:12.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.557+0000 7f1cd7103700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01500b0 con 0x7f1cd00a84d0 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1cd006d5f0 0x7f1cd006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1cd006d5f0 0x7f1cd006da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58232/0 (socket says 172.21.15.154:58232) 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 -- 172.21.15.154:0/3684823457 learned_addr learned my addr 172.21.15.154:0/3684823457 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 -- 172.21.15.154:0/3684823457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd00a84d0 msgr2=0x7f1cd00a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 --2- 172.21.15.154:0/3684823457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd00a84d0 0x7f1cd00a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 -- 172.21.15.154:0/3684823457 >> v2:172.21.15.191:3300/0 conn(0x7f1cd006c670 msgr2=0x7f1cd014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 --2- 172.21.15.154:0/3684823457 >> v2:172.21.15.191:3300/0 conn(0x7f1cd006c670 0x7f1cd014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 -- 172.21.15.154:0/3684823457 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00aa3b0 con 0x7f1cd006d5f0 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1cd56a0700 1 --2- 172.21.15.154:0/3684823457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006d5f0 0x7f1cd006da80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0072550 tx=0x7f1cc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8608ba3935fc1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1ccf7fe700 1 -- 172.21.15.154:0/3684823457 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800e070 con 0x7f1cd006d5f0 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1ccf7fe700 1 -- 172.21.15.154:0/3684823457 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800b9c0 con 0x7f1cd006d5f0 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.558+0000 7f1ccf7fe700 1 -- 172.21.15.154:0/3684823457 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800bd70 con 0x7f1cd006d5f0 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 -- 172.21.15.154:0/3684823457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006d5f0 msgr2=0x7f1cd006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 --2- 172.21.15.154:0/3684823457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006d5f0 0x7f1cd006da80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0072550 tx=0x7f1cc800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 -- 172.21.15.154:0/3684823457 shutdown_connections 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 --2- 172.21.15.154:0/3684823457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006d5f0 0x7f1cd006da80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 --2- 172.21.15.154:0/3684823457 >> v2:172.21.15.191:3300/0 conn(0x7f1cd006c670 0x7f1cd014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 --2- 172.21.15.154:0/3684823457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd00a84d0 0x7f1cd00a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 -- 172.21.15.154:0/3684823457 >> 172.21.15.154:0/3684823457 conn(0x7f1cd00c5e70 msgr2=0x7f1cd00c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 -- 172.21.15.154:0/3684823457 shutdown_connections 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.559+0000 7f1cd7103700 1 -- 172.21.15.154:0/3684823457 wait complete. 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.560+0000 7f1cd7103700 1 Processor -- start 2024-03-14T09:02:12.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.560+0000 7f1cd7103700 1 -- start start 2024-03-14T09:02:12.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd006d5f0 0x7f1cd00c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cd00a84d0 0x7f1cd01dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0153560 con 0x7f1cd006c670 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00bfb90 con 0x7f1cd00a84d0 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd7103700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01df7d0 con 0x7f1cd006d5f0 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58244/0 (socket says 172.21.15.154:58244) 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 -- 172.21.15.154:0/1735335287 learned_addr learned my addr 172.21.15.154:0/1735335287 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1ccffff700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd006d5f0 0x7f1cd00c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd56a0700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cd00a84d0 0x7f1cd01dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 -- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd006d5f0 msgr2=0x7f1cd00c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd006d5f0 0x7f1cd00c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 -- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cd00a84d0 msgr2=0x7f1cd01dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cd00a84d0 0x7f1cd01dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.561+0000 7f1cd4e9f700 1 -- 172.21.15.154:0/1735335287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8009b30 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1cd4e9f700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b530 tx=0x7f1cc400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4011830 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc4011990 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd01e8c40 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4011d40 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.562+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0153280 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.563+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd01e8a10 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.563+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1cc401a400 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.564+0000 7f1ccd7fa700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 0x7f1cb406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.564+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cc4088bb0 con 0x7f1cd006c670 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.564+0000 7f1ccffff700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 0x7f1cb406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.564+0000 7f1ccffff700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 0x7f1cb406ccf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009970 tx=0x7f1cbc009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.568+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1cd01e8a10 con 0x7f1cd006c670 2024-03-14T09:02:12.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.759+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cd00aa950 con 0x7f1cb406a830 2024-03-14T09:02:12.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.767+0000 7f1ccd7fa700 1 -- 172.21.15.154:0/1735335287 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264747 (secure 0 0 0) 0x7f1cbc0095c0 con 0x7f1cb406a830 2024-03-14T09:02:12.770 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 msgr2=0x7f1cb406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 0x7f1cb406ccf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009970 tx=0x7f1cbc009110 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 msgr2=0x7f1cd01514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b530 tx=0x7f1cc400b840 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 shutdown_connections 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cd00a84d0 0x7f1cd01dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.770+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1cb406a830 0x7f1cb406ccf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.771+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd006c670 0x7f1cd01514d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.771+0000 7f1cd7103700 1 --2- 172.21.15.154:0/1735335287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1cd006d5f0 0x7f1cd00c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.771+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 >> 172.21.15.154:0/1735335287 conn(0x7f1cd00c5e70 msgr2=0x7f1cd014e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.771+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 shutdown_connections 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.771+0000 7f1cd7103700 1 -- 172.21.15.154:0/1735335287 wait complete. 2024-03-14T09:02:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-03-14T09:02:11.530009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4136960,"data_stored":2300700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001893"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:05.435130+0000","last_change":"2024-03-14T09:02:03.755819+0000","last_active":"2024-03-14T09:02:05.435130+0000","last_peered":"2024-03-14T09:02:05.435130+0000","last_clean":"2024-03-14T09:02:05.435130+0000","last_became_active":"2024-03-14T09:02:03.750027+0000","last_became_peered":"2024-03-14T09:02:03.750027+0000","last_unstale":"2024-03-14T09:02:05.435130+0000","last_undegraded":"2024-03-14T09:02:05.435130+0000","last_fullsized":"2024-03-14T09:02:05.435130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249199+0000","last_change":"2024-03-14T09:02:03.757259+0000","last_active":"2024-03-14T09:02:06.249199+0000","last_peered":"2024-03-14T09:02:06.249199+0000","last_clean":"2024-03-14T09:02:06.249199+0000","last_became_active":"2024-03-14T09:02:03.757050+0000","last_became_peered":"2024-03-14T09:02:03.757050+0000","last_unstale":"2024-03-14T09:02:06.249199+0000","last_undegraded":"2024-03-14T09:02:06.249199+0000","last_fullsized":"2024-03-14T09:02:06.249199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742825+0000","last_change":"2024-03-14T09:02:03.741721+0000","last_active":"2024-03-14T09:02:04.742825+0000","last_peered":"2024-03-14T09:02:04.742825+0000","last_clean":"2024-03-14T09:02:04.742825+0000","last_became_active":"2024-03-14T09:02:03.741350+0000","last_became_peered":"2024-03-14T09:02:03.741350+0000","last_unstale":"2024-03-14T09:02:04.742825+0000","last_undegraded":"2024-03-14T09:02:04.742825+0000","last_fullsized":"2024-03-14T09:02:04.742825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.214734+0000","last_change":"2024-03-14T09:02:03.747476+0000","last_active":"2024-03-14T09:02:06.214734+0000","last_peered":"2024-03-14T09:02:06.214734+0000","last_clean":"2024-03-14T09:02:06.214734+0000","last_became_active":"2024-03-14T09:02:03.747104+0000","last_became_peered":"2024-03-14T09:02:03.747104+0000","last_unstale":"2024-03-14T09:02:06.214734+0000","last_undegraded":"2024-03-14T09:02:06.214734+0000","last_fullsized":"2024-03-14T09:02:06.214734+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738738+0000","last_change":"2024-03-14T09:02:03.751416+0000","last_active":"2024-03-14T09:02:04.738738+0000","last_peered":"2024-03-14T09:02:04.738738+0000","last_clean":"2024-03-14T09:02:04.738738+0000","last_became_active":"2024-03-14T09:02:03.751119+0000","last_became_peered":"2024-03-14T09:02:03.751119+0000","last_unstale":"2024-03-14T09:02:04.738738+0000","last_undegraded":"2024-03-14T09:02:04.738738+0000","last_fullsized":"2024-03-14T09:02:04.738738+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:06.397586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249142+0000","last_change":"2024-03-14T09:02:03.747083+0000","last_active":"2024-03-14T09:02:06.249142+0000","last_peered":"2024-03-14T09:02:06.249142+0000","last_clean":"2024-03-14T09:02:06.249142+0000","last_became_active":"2024-03-14T09:02:03.743458+0000","last_became_peered":"2024-03-14T09:02:03.743458+0000","last_unstale":"2024-03-14T09:02:06.249142+0000","last_undegraded":"2024-03-14T09:02:06.249142+0000","last_fullsized":"2024-03-14T09:02:06.249142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682396+0000","last_change":"2024-03-14T09:02:03.751248+0000","last_active":"2024-03-14T09:02:06.682396+0000","last_peered":"2024-03-14T09:02:06.682396+0000","last_clean":"2024-03-14T09:02:06.682396+0000","last_became_active":"2024-03-14T09:02:03.750977+0000","last_became_peered":"2024-03-14T09:02:03.750977+0000","last_unstale":"2024-03-14T09:02:06.682396+0000","last_undegraded":"2024-03-14T09:02:06.682396+0000","last_fullsized":"2024-03-14T09:02:06.682396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682319+0000","last_change":"2024-03-14T09:02:03.753172+0000","last_active":"2024-03-14T09:02:06.682319+0000","last_peered":"2024-03-14T09:02:06.682319+0000","last_clean":"2024-03-14T09:02:06.682319+0000","last_became_active":"2024-03-14T09:02:03.752882+0000","last_became_peered":"2024-03-14T09:02:03.752882+0000","last_unstale":"2024-03-14T09:02:06.682319+0000","last_undegraded":"2024-03-14T09:02:06.682319+0000","last_fullsized":"2024-03-14T09:02:06.682319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.744084+0000","last_change":"2024-03-14T09:02:03.741974+0000","last_active":"2024-03-14T09:02:04.744084+0000","last_peered":"2024-03-14T09:02:04.744084+0000","last_clean":"2024-03-14T09:02:04.744084+0000","last_became_active":"2024-03-14T09:02:03.741657+0000","last_became_peered":"2024-03-14T09:02:03.741657+0000","last_unstale":"2024-03-14T09:02:04.744084+0000","last_undegraded":"2024-03-14T09:02:04.744084+0000","last_fullsized":"2024-03-14T09:02:04.744084+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.215002+0000","last_change":"2024-03-14T09:02:03.757773+0000","last_active":"2024-03-14T09:02:06.215002+0000","last_peered":"2024-03-14T09:02:06.215002+0000","last_clean":"2024-03-14T09:02:06.215002+0000","last_became_active":"2024-03-14T09:02:03.757426+0000","last_became_peered":"2024-03-14T09:02:03.757426+0000","last_unstale":"2024-03-14T09:02:06.215002+0000","last_undegraded":"2024-03-14T09:02:06.215002+0000","last_fullsized":"2024-03-14T09:02:06.215002+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682618+0000","last_change":"2024-03-14T09:02:03.758860+0000","last_active":"2024-03-14T09:02:06.682618+0000","last_peered":"2024-03-14T09:02:06.682618+0000","last_clean":"2024-03-14T09:02:06.682618+0000","last_became_active":"2024-03-14T09:02:03.758547+0000","last_became_peered":"2024-03-14T09:02:03.758547+0000","last_unstale":"2024-03-14T09:02:06.682618+0000","last_undegraded":"2024-03-14T09:02:06.682618+0000","last_fullsized":"2024-03-14T09:02:06.682618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742943+0000","last_change":"2024-03-14T09:02:03.750480+0000","last_active":"2024-03-14T09:02:04.742943+0000","last_peered":"2024-03-14T09:02:04.742943+0000","last_clean":"2024-03-14T09:02:04.742943+0000","last_became_active":"2024-03-14T09:02:03.750198+0000","last_became_peered":"2024-03-14T09:02:03.750198+0000","last_unstale":"2024-03-14T09:02:04.742943+0000","last_undegraded":"2024-03-14T09:02:04.742943+0000","last_fullsized":"2024-03-14T09:02:04.742943+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738899+0000","last_change":"2024-03-14T09:02:03.751100+0000","last_active":"2024-03-14T09:02:04.738899+0000","last_peered":"2024-03-14T09:02:04.738899+0000","last_clean":"2024-03-14T09:02:04.738899+0000","last_became_active":"2024-03-14T09:02:03.750866+0000","last_became_peered":"2024-03-14T09:02:03.750866+0000","last_unstale":"2024-03-14T09:02:04.738899+0000","last_undegraded":"2024-03-14T09:02:04.738899+0000","last_fullsized":"2024-03-14T09:02:04.738899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:07.275479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249105+0000","last_change":"2024-03-14T09:02:03.752408+0000","last_active":"2024-03-14T09:02:06.249105+0000","last_peered":"2024-03-14T09:02:06.249105+0000","last_clean":"2024-03-14T09:02:06.249105+0000","last_became_active":"2024-03-14T09:02:03.752136+0000","last_became_peered":"2024-03-14T09:02:03.752136+0000","last_unstale":"2024-03-14T09:02:06.249105+0000","last_undegraded":"2024-03-14T09:02:06.249105+0000","last_fullsized":"2024-03-14T09:02:06.249105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742597+0000","last_change":"2024-03-14T09:02:03.751964+0000","last_active":"2024-03-14T09:02:04.742597+0000","last_peered":"2024-03-14T09:02:04.742597+0000","last_clean":"2024-03-14T09:02:04.742597+0000","last_became_active":"2024-03-14T09:02:03.751703+0000","last_became_peered":"2024-03-14T09:02:03.751703+0000","last_unstale":"2024-03-14T09:02:04.742597+0000","last_undegraded":"2024-03-14T09:02:04.742597+0000","last_fullsized":"2024-03-14T09:02:04.742597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682603+0000","last_change":"2024-03-14T09:02:03.752051+0000","last_active":"2024-03-14T09:02:06.682603+0000","last_peered":"2024-03-14T09:02:06.682603+0000","last_clean":"2024-03-14T09:02:06.682603+0000","last_became_active":"2024-03-14T09:02:03.751771+0000","last_became_peered":"2024-03-14T09:02:03.751771+0000","last_unstale":"2024-03-14T09:02:06.682603+0000","last_undegraded":"2024-03-14T09:02:06.682603+0000","last_fullsized":"2024-03-14T09:02:06.682603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682836+0000","last_change":"2024-03-14T09:02:03.756745+0000","last_active":"2024-03-14T09:02:06.682836+0000","last_peered":"2024-03-14T09:02:06.682836+0000","last_clean":"2024-03-14T09:02:06.682836+0000","last_became_active":"2024-03-14T09:02:03.756499+0000","last_became_peered":"2024-03-14T09:02:03.756499+0000","last_unstale":"2024-03-14T09:02:06.682836+0000","last_undegraded":"2024-03-14T09:02:06.682836+0000","last_fullsized":"2024-03-14T09:02:06.682836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249194+0000","last_change":"2024-03-14T09:02:03.760516+0000","last_active":"2024-03-14T09:02:06.249194+0000","last_peered":"2024-03-14T09:02:06.249194+0000","last_clean":"2024-03-14T09:02:06.249194+0000","last_became_active":"2024-03-14T09:02:03.760292+0000","last_became_peered":"2024-03-14T09:02:03.760292+0000","last_unstale":"2024-03-14T09:02:06.249194+0000","last_undegraded":"2024-03-14T09:02:06.249194+0000","last_fullsized":"2024-03-14T09:02:06.249194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:42:45.106158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.248976+0000","last_change":"2024-03-14T09:02:03.765489+0000","last_active":"2024-03-14T09:02:06.248976+0000","last_peered":"2024-03-14T09:02:06.248976+0000","last_clean":"2024-03-14T09:02:06.248976+0000","last_became_active":"2024-03-14T09:02:03.765228+0000","last_became_peered":"2024-03-14T09:02:03.765228+0000","last_unstale":"2024-03-14T09:02:06.248976+0000","last_undegraded":"2024-03-14T09:02:06.248976+0000","last_fullsized":"2024-03-14T09:02:06.248976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:39:59.816967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738943+0000","last_change":"2024-03-14T09:02:03.757453+0000","last_active":"2024-03-14T09:02:04.738943+0000","last_peered":"2024-03-14T09:02:04.738943+0000","last_clean":"2024-03-14T09:02:04.738943+0000","last_became_active":"2024-03-14T09:02:03.757182+0000","last_became_peered":"2024-03-14T09:02:03.757182+0000","last_unstale":"2024-03-14T09:02:04.738943+0000","last_undegraded":"2024-03-14T09:02:04.738943+0000","last_fullsized":"2024-03-14T09:02:04.738943+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:09.988215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.743697+0000","last_change":"2024-03-14T09:02:03.743740+0000","last_active":"2024-03-14T09:02:04.743697+0000","last_peered":"2024-03-14T09:02:04.743697+0000","last_clean":"2024-03-14T09:02:04.743697+0000","last_became_active":"2024-03-14T09:02:03.742714+0000","last_became_peered":"2024-03-14T09:02:03.742714+0000","last_unstale":"2024-03-14T09:02:04.743697+0000","last_undegraded":"2024-03-14T09:02:04.743697+0000","last_fullsized":"2024-03-14T09:02:04.743697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.743665+0000","last_change":"2024-03-14T09:02:03.749275+0000","last_active":"2024-03-14T09:02:04.743665+0000","last_peered":"2024-03-14T09:02:04.743665+0000","last_clean":"2024-03-14T09:02:04.743665+0000","last_became_active":"2024-03-14T09:02:03.748874+0000","last_became_peered":"2024-03-14T09:02:03.748874+0000","last_unstale":"2024-03-14T09:02:04.743665+0000","last_undegraded":"2024-03-14T09:02:04.743665+0000","last_fullsized":"2024-03-14T09:02:04.743665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.255641+0000","last_change":"2024-03-14T09:02:03.741584+0000","last_active":"2024-03-14T09:02:06.255641+0000","last_peered":"2024-03-14T09:02:06.255641+0000","last_clean":"2024-03-14T09:02:06.255641+0000","last_became_active":"2024-03-14T09:02:03.741367+0000","last_became_peered":"2024-03-14T09:02:03.741367+0000","last_unstale":"2024-03-14T09:02:06.255641+0000","last_undegraded":"2024-03-14T09:02:06.255641+0000","last_fullsized":"2024-03-14T09:02:06.255641+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742337+0000","last_change":"2024-03-14T09:02:03.757604+0000","last_active":"2024-03-14T09:02:04.742337+0000","last_peered":"2024-03-14T09:02:04.742337+0000","last_clean":"2024-03-14T09:02:04.742337+0000","last_became_active":"2024-03-14T09:02:03.757387+0000","last_became_peered":"2024-03-14T09:02:03.757387+0000","last_unstale":"2024-03-14T09:02:04.742337+0000","last_undegraded":"2024-03-14T09:02:04.742337+0000","last_fullsized":"2024-03-14T09:02:04.742337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.215013+0000","last_change":"2024-03-14T09:02:03.746056+0000","last_active":"2024-03-14T09:02:06.215013+0000","last_peered":"2024-03-14T09:02:06.215013+0000","last_clean":"2024-03-14T09:02:06.215013+0000","last_became_active":"2024-03-14T09:02:03.745200+0000","last_became_peered":"2024-03-14T09:02:03.745200+0000","last_unstale":"2024-03-14T09:02:06.215013+0000","last_undegraded":"2024-03-14T09:02:06.215013+0000","last_fullsized":"2024-03-14T09:02:06.215013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249249+0000","last_change":"2024-03-14T09:02:03.753751+0000","last_active":"2024-03-14T09:02:06.249249+0000","last_peered":"2024-03-14T09:02:06.249249+0000","last_clean":"2024-03-14T09:02:06.249249+0000","last_became_active":"2024-03-14T09:02:03.753485+0000","last_became_peered":"2024-03-14T09:02:03.753485+0000","last_unstale":"2024-03-14T09:02:06.249249+0000","last_undegraded":"2024-03-14T09:02:06.249249+0000","last_fullsized":"2024-03-14T09:02:06.249249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:54.950201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.215057+0000","last_change":"2024-03-14T09:02:03.757244+0000","last_active":"2024-03-14T09:02:06.215057+0000","last_peered":"2024-03-14T09:02:06.215057+0000","last_clean":"2024-03-14T09:02:06.215057+0000","last_became_active":"2024-03-14T09:02:03.753248+0000","last_became_peered":"2024-03-14T09:02:03.753248+0000","last_unstale":"2024-03-14T09:02:06.215057+0000","last_undegraded":"2024-03-14T09:02:06.215057+0000","last_fullsized":"2024-03-14T09:02:06.215057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742894+0000","last_change":"2024-03-14T09:02:03.737708+0000","last_active":"2024-03-14T09:02:04.742894+0000","last_peered":"2024-03-14T09:02:04.742894+0000","last_clean":"2024-03-14T09:02:04.742894+0000","last_became_active":"2024-03-14T09:02:03.737433+0000","last_became_peered":"2024-03-14T09:02:03.737433+0000","last_unstale":"2024-03-14T09:02:04.742894+0000","last_undegraded":"2024-03-14T09:02:04.742894+0000","last_fullsized":"2024-03-14T09:02:04.742894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.743700+0000","last_change":"2024-03-14T09:02:03.751534+0000","last_active":"2024-03-14T09:02:04.743700+0000","last_peered":"2024-03-14T09:02:04.743700+0000","last_clean":"2024-03-14T09:02:04.743700+0000","last_became_active":"2024-03-14T09:02:03.751265+0000","last_became_peered":"2024-03-14T09:02:03.751265+0000","last_unstale":"2024-03-14T09:02:04.743700+0000","last_undegraded":"2024-03-14T09:02:04.743700+0000","last_fullsized":"2024-03-14T09:02:04.743700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682421+0000","last_change":"2024-03-14T09:02:03.754829+0000","last_active":"2024-03-14T09:02:06.682421+0000","last_peered":"2024-03-14T09:02:06.682421+0000","last_clean":"2024-03-14T09:02:06.682421+0000","last_became_active":"2024-03-14T09:02:03.754553+0000","last_became_peered":"2024-03-14T09:02:03.754553+0000","last_unstale":"2024-03-14T09:02:06.682421+0000","last_undegraded":"2024-03-14T09:02:06.682421+0000","last_fullsized":"2024-03-14T09:02:06.682421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:05.435118+0000","last_change":"2024-03-14T09:02:03.754647+0000","last_active":"2024-03-14T09:02:05.435118+0000","last_peered":"2024-03-14T09:02:05.435118+0000","last_clean":"2024-03-14T09:02:05.435118+0000","last_became_active":"2024-03-14T09:02:03.750909+0000","last_became_peered":"2024-03-14T09:02:03.750909+0000","last_unstale":"2024-03-14T09:02:05.435118+0000","last_undegraded":"2024-03-14T09:02:05.435118+0000","last_fullsized":"2024-03-14T09:02:05.435118+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682776+0000","last_change":"2024-03-14T09:02:03.747713+0000","last_active":"2024-03-14T09:02:06.682776+0000","last_peered":"2024-03-14T09:02:06.682776+0000","last_clean":"2024-03-14T09:02:06.682776+0000","last_became_active":"2024-03-14T09:02:03.747113+0000","last_became_peered":"2024-03-14T09:02:03.747113+0000","last_unstale":"2024-03-14T09:02:06.682776+0000","last_undegraded":"2024-03-14T09:02:06.682776+0000","last_fullsized":"2024-03-14T09:02:06.682776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.255749+0000","last_change":"2024-03-14T09:02:03.744443+0000","last_active":"2024-03-14T09:02:06.255749+0000","last_peered":"2024-03-14T09:02:06.255749+0000","last_clean":"2024-03-14T09:02:06.255749+0000","last_became_active":"2024-03-14T09:02:03.744149+0000","last_became_peered":"2024-03-14T09:02:03.744149+0000","last_unstale":"2024-03-14T09:02:06.255749+0000","last_undegraded":"2024-03-14T09:02:06.255749+0000","last_fullsized":"2024-03-14T09:02:06.255749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.756131+0000","last_change":"2024-03-14T09:02:03.331160+0000","last_active":"2024-03-14T09:02:03.756131+0000","last_peered":"2024-03-14T09:02:03.756131+0000","last_clean":"2024-03-14T09:02:03.756131+0000","last_became_active":"2024-03-14T09:02:03.330643+0000","last_became_peered":"2024-03-14T09:02:03.330643+0000","last_unstale":"2024-03-14T09:02:03.756131+0000","last_undegraded":"2024-03-14T09:02:03.756131+0000","last_fullsized":"2024-03-14T09:02:03.756131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:05.435194+0000","last_change":"2024-03-14T09:02:03.750291+0000","last_active":"2024-03-14T09:02:05.435194+0000","last_peered":"2024-03-14T09:02:05.435194+0000","last_clean":"2024-03-14T09:02:05.435194+0000","last_became_active":"2024-03-14T09:02:03.749738+0000","last_became_peered":"2024-03-14T09:02:03.749738+0000","last_unstale":"2024-03-14T09:02:05.435194+0000","last_undegraded":"2024-03-14T09:02:05.435194+0000","last_fullsized":"2024-03-14T09:02:05.435194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.750138+0000","last_change":"2024-03-14T09:02:03.335358+0000","last_active":"2024-03-14T09:02:03.750138+0000","last_peered":"2024-03-14T09:02:03.750138+0000","last_clean":"2024-03-14T09:02:03.750138+0000","last_became_active":"2024-03-14T09:02:03.335031+0000","last_became_peered":"2024-03-14T09:02:03.335031+0000","last_unstale":"2024-03-14T09:02:03.750138+0000","last_undegraded":"2024-03-14T09:02:03.750138+0000","last_fullsized":"2024-03-14T09:02:03.750138+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.743884+0000","last_change":"2024-03-14T09:02:03.750990+0000","last_active":"2024-03-14T09:02:04.743884+0000","last_peered":"2024-03-14T09:02:04.743884+0000","last_clean":"2024-03-14T09:02:04.743884+0000","last_became_active":"2024-03-14T09:02:03.750742+0000","last_became_peered":"2024-03-14T09:02:03.750742+0000","last_unstale":"2024-03-14T09:02:04.743884+0000","last_undegraded":"2024-03-14T09:02:04.743884+0000","last_fullsized":"2024-03-14T09:02:04.743884+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.765348+0000","last_change":"2024-03-14T09:02:03.331945+0000","last_active":"2024-03-14T09:02:03.765348+0000","last_peered":"2024-03-14T09:02:03.765348+0000","last_clean":"2024-03-14T09:02:03.765348+0000","last_became_active":"2024-03-14T09:02:03.331686+0000","last_became_peered":"2024-03-14T09:02:03.331686+0000","last_unstale":"2024-03-14T09:02:03.765348+0000","last_undegraded":"2024-03-14T09:02:03.765348+0000","last_fullsized":"2024-03-14T09:02:03.765348+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.215016+0000","last_change":"2024-03-14T09:02:03.757204+0000","last_active":"2024-03-14T09:02:06.215016+0000","last_peered":"2024-03-14T09:02:06.215016+0000","last_clean":"2024-03-14T09:02:06.215016+0000","last_became_active":"2024-03-14T09:02:03.752933+0000","last_became_peered":"2024-03-14T09:02:03.752933+0000","last_unstale":"2024-03-14T09:02:06.215016+0000","last_undegraded":"2024-03-14T09:02:06.215016+0000","last_fullsized":"2024-03-14T09:02:06.215016+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.759971+0000","last_change":"2024-03-14T09:02:03.331128+0000","last_active":"2024-03-14T09:02:03.759971+0000","last_peered":"2024-03-14T09:02:03.759971+0000","last_clean":"2024-03-14T09:02:03.759971+0000","last_became_active":"2024-03-14T09:02:03.330898+0000","last_became_peered":"2024-03-14T09:02:03.330898+0000","last_unstale":"2024-03-14T09:02:03.759971+0000","last_undegraded":"2024-03-14T09:02:03.759971+0000","last_fullsized":"2024-03-14T09:02:03.759971+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682506+0000","last_change":"2024-03-14T09:02:03.747325+0000","last_active":"2024-03-14T09:02:06.682506+0000","last_peered":"2024-03-14T09:02:06.682506+0000","last_clean":"2024-03-14T09:02:06.682506+0000","last_became_active":"2024-03-14T09:02:03.747029+0000","last_became_peered":"2024-03-14T09:02:03.747029+0000","last_unstale":"2024-03-14T09:02:06.682506+0000","last_undegraded":"2024-03-14T09:02:06.682506+0000","last_fullsized":"2024-03-14T09:02:06.682506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.756879+0000","last_change":"2024-03-14T09:02:03.331030+0000","last_active":"2024-03-14T09:02:03.756879+0000","last_peered":"2024-03-14T09:02:03.756879+0000","last_clean":"2024-03-14T09:02:03.756879+0000","last_became_active":"2024-03-14T09:02:03.330772+0000","last_became_peered":"2024-03-14T09:02:03.330772+0000","last_unstale":"2024-03-14T09:02:03.756879+0000","last_undegraded":"2024-03-14T09:02:03.756879+0000","last_fullsized":"2024-03-14T09:02:03.756879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742665+0000","last_change":"2024-03-14T09:02:03.738226+0000","last_active":"2024-03-14T09:02:04.742665+0000","last_peered":"2024-03-14T09:02:04.742665+0000","last_clean":"2024-03-14T09:02:04.742665+0000","last_became_active":"2024-03-14T09:02:03.737960+0000","last_became_peered":"2024-03-14T09:02:03.737960+0000","last_unstale":"2024-03-14T09:02:04.742665+0000","last_undegraded":"2024-03-14T09:02:04.742665+0000","last_fullsized":"2024-03-14T09:02:04.742665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.756995+0000","last_change":"2024-03-14T09:02:03.330899+0000","last_active":"2024-03-14T09:02:03.756995+0000","last_peered":"2024-03-14T09:02:03.756995+0000","last_clean":"2024-03-14T09:02:03.756995+0000","last_became_active":"2024-03-14T09:02:03.330649+0000","last_became_peered":"2024-03-14T09:02:03.330649+0000","last_unstale":"2024-03-14T09:02:03.756995+0000","last_undegraded":"2024-03-14T09:02:03.756995+0000","last_fullsized":"2024-03-14T09:02:03.756995+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742839+0000","last_change":"2024-03-14T09:02:03.743458+0000","last_active":"2024-03-14T09:02:04.742839+0000","last_peered":"2024-03-14T09:02:04.742839+0000","last_clean":"2024-03-14T09:02:04.742839+0000","last_became_active":"2024-03-14T09:02:03.742694+0000","last_became_peered":"2024-03-14T09:02:03.742694+0000","last_unstale":"2024-03-14T09:02:04.742839+0000","last_undegraded":"2024-03-14T09:02:04.742839+0000","last_fullsized":"2024-03-14T09:02:04.742839+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.741214+0000","last_change":"2024-03-14T09:02:03.327627+0000","last_active":"2024-03-14T09:02:03.741214+0000","last_peered":"2024-03-14T09:02:03.741214+0000","last_clean":"2024-03-14T09:02:03.741214+0000","last_became_active":"2024-03-14T09:02:03.327348+0000","last_became_peered":"2024-03-14T09:02:03.327348+0000","last_unstale":"2024-03-14T09:02:03.741214+0000","last_undegraded":"2024-03-14T09:02:03.741214+0000","last_fullsized":"2024-03-14T09:02:03.741214+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.214630+0000","last_change":"2024-03-14T09:02:03.740713+0000","last_active":"2024-03-14T09:02:06.214630+0000","last_peered":"2024-03-14T09:02:06.214630+0000","last_clean":"2024-03-14T09:02:06.214630+0000","last_became_active":"2024-03-14T09:02:03.740458+0000","last_became_peered":"2024-03-14T09:02:03.740458+0000","last_unstale":"2024-03-14T09:02:06.214630+0000","last_undegraded":"2024-03-14T09:02:06.214630+0000","last_fullsized":"2024-03-14T09:02:06.214630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.733188+0000","last_change":"2024-03-14T09:02:03.330919+0000","last_active":"2024-03-14T09:02:03.733188+0000","last_peered":"2024-03-14T09:02:03.733188+0000","last_clean":"2024-03-14T09:02:03.733188+0000","last_became_active":"2024-03-14T09:02:03.330613+0000","last_became_peered":"2024-03-14T09:02:03.330613+0000","last_unstale":"2024-03-14T09:02:03.733188+0000","last_undegraded":"2024-03-14T09:02:03.733188+0000","last_fullsized":"2024-03-14T09:02:03.733188+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742976+0000","last_change":"2024-03-14T09:02:03.757103+0000","last_active":"2024-03-14T09:02:04.742976+0000","last_peered":"2024-03-14T09:02:04.742976+0000","last_clean":"2024-03-14T09:02:04.742976+0000","last_became_active":"2024-03-14T09:02:03.756572+0000","last_became_peered":"2024-03-14T09:02:03.756572+0000","last_unstale":"2024-03-14T09:02:04.742976+0000","last_undegraded":"2024-03-14T09:02:04.742976+0000","last_fullsized":"2024-03-14T09:02:04.742976+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.741087+0000","last_change":"2024-03-14T09:02:03.335707+0000","last_active":"2024-03-14T09:02:03.741087+0000","last_peered":"2024-03-14T09:02:03.741087+0000","last_clean":"2024-03-14T09:02:03.741087+0000","last_became_active":"2024-03-14T09:02:03.335448+0000","last_became_peered":"2024-03-14T09:02:03.335448+0000","last_unstale":"2024-03-14T09:02:03.741087+0000","last_undegraded":"2024-03-14T09:02:03.741087+0000","last_fullsized":"2024-03-14T09:02:03.741087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249078+0000","last_change":"2024-03-14T09:02:03.758670+0000","last_active":"2024-03-14T09:02:06.249078+0000","last_peered":"2024-03-14T09:02:06.249078+0000","last_clean":"2024-03-14T09:02:06.249078+0000","last_became_active":"2024-03-14T09:02:03.758426+0000","last_became_peered":"2024-03-14T09:02:03.758426+0000","last_unstale":"2024-03-14T09:02:06.249078+0000","last_undegraded":"2024-03-14T09:02:06.249078+0000","last_fullsized":"2024-03-14T09:02:06.249078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:37:44.347728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.736579+0000","last_change":"2024-03-14T09:02:03.331300+0000","last_active":"2024-03-14T09:02:03.736579+0000","last_peered":"2024-03-14T09:02:03.736579+0000","last_clean":"2024-03-14T09:02:03.736579+0000","last_became_active":"2024-03-14T09:02:03.331033+0000","last_became_peered":"2024-03-14T09:02:03.331033+0000","last_unstale":"2024-03-14T09:02:03.736579+0000","last_undegraded":"2024-03-14T09:02:03.736579+0000","last_fullsized":"2024-03-14T09:02:03.736579+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249240+0000","last_change":"2024-03-14T09:02:03.748711+0000","last_active":"2024-03-14T09:02:06.249240+0000","last_peered":"2024-03-14T09:02:06.249240+0000","last_clean":"2024-03-14T09:02:06.249240+0000","last_became_active":"2024-03-14T09:02:03.748483+0000","last_became_peered":"2024-03-14T09:02:03.748483+0000","last_unstale":"2024-03-14T09:02:06.249240+0000","last_undegraded":"2024-03-14T09:02:06.249240+0000","last_fullsized":"2024-03-14T09:02:06.249240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:43.215729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.755592+0000","last_change":"2024-03-14T09:02:03.335856+0000","last_active":"2024-03-14T09:02:03.755592+0000","last_peered":"2024-03-14T09:02:03.755592+0000","last_clean":"2024-03-14T09:02:03.755592+0000","last_became_active":"2024-03-14T09:02:03.335559+0000","last_became_peered":"2024-03-14T09:02:03.335559+0000","last_unstale":"2024-03-14T09:02:03.755592+0000","last_undegraded":"2024-03-14T09:02:03.755592+0000","last_fullsized":"2024-03-14T09:02:03.755592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742955+0000","last_change":"2024-03-14T09:02:03.741618+0000","last_active":"2024-03-14T09:02:04.742955+0000","last_peered":"2024-03-14T09:02:04.742955+0000","last_clean":"2024-03-14T09:02:04.742955+0000","last_became_active":"2024-03-14T09:02:03.740972+0000","last_became_peered":"2024-03-14T09:02:03.740972+0000","last_unstale":"2024-03-14T09:02:04.742955+0000","last_undegraded":"2024-03-14T09:02:04.742955+0000","last_fullsized":"2024-03-14T09:02:04.742955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.773843+0000","last_change":"2024-03-14T09:02:03.333899+0000","last_active":"2024-03-14T09:02:03.773843+0000","last_peered":"2024-03-14T09:02:03.773843+0000","last_clean":"2024-03-14T09:02:03.773843+0000","last_became_active":"2024-03-14T09:02:03.333685+0000","last_became_peered":"2024-03-14T09:02:03.333685+0000","last_unstale":"2024-03-14T09:02:03.773843+0000","last_undegraded":"2024-03-14T09:02:03.773843+0000","last_fullsized":"2024-03-14T09:02:03.773843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.214626+0000","last_change":"2024-03-14T09:02:03.740610+0000","last_active":"2024-03-14T09:02:06.214626+0000","last_peered":"2024-03-14T09:02:06.214626+0000","last_clean":"2024-03-14T09:02:06.214626+0000","last_became_active":"2024-03-14T09:02:03.740348+0000","last_became_peered":"2024-03-14T09:02:03.740348+0000","last_unstale":"2024-03-14T09:02:06.214626+0000","last_undegraded":"2024-03-14T09:02:06.214626+0000","last_fullsized":"2024-03-14T09:02:06.214626+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.757078+0000","last_change":"2024-03-14T09:02:03.329164+0000","last_active":"2024-03-14T09:02:03.757078+0000","last_peered":"2024-03-14T09:02:03.757078+0000","last_clean":"2024-03-14T09:02:03.757078+0000","last_became_active":"2024-03-14T09:02:03.328895+0000","last_became_peered":"2024-03-14T09:02:03.328895+0000","last_unstale":"2024-03-14T09:02:03.757078+0000","last_undegraded":"2024-03-14T09:02:03.757078+0000","last_fullsized":"2024-03-14T09:02:03.757078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.249078+0000","last_change":"2024-03-14T09:02:03.751429+0000","last_active":"2024-03-14T09:02:06.249078+0000","last_peered":"2024-03-14T09:02:06.249078+0000","last_clean":"2024-03-14T09:02:06.249078+0000","last_became_active":"2024-03-14T09:02:03.751167+0000","last_became_peered":"2024-03-14T09:02:03.751167+0000","last_unstale":"2024-03-14T09:02:06.249078+0000","last_undegraded":"2024-03-14T09:02:06.249078+0000","last_fullsized":"2024-03-14T09:02:06.249078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.731933+0000","last_change":"2024-03-14T09:02:03.329056+0000","last_active":"2024-03-14T09:02:03.731933+0000","last_peered":"2024-03-14T09:02:03.731933+0000","last_clean":"2024-03-14T09:02:03.731933+0000","last_became_active":"2024-03-14T09:02:03.328772+0000","last_became_peered":"2024-03-14T09:02:03.328772+0000","last_unstale":"2024-03-14T09:02:03.731933+0000","last_undegraded":"2024-03-14T09:02:03.731933+0000","last_fullsized":"2024-03-14T09:02:03.731933+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682783+0000","last_change":"2024-03-14T09:02:03.747948+0000","last_active":"2024-03-14T09:02:06.682783+0000","last_peered":"2024-03-14T09:02:06.682783+0000","last_clean":"2024-03-14T09:02:06.682783+0000","last_became_active":"2024-03-14T09:02:03.747471+0000","last_became_peered":"2024-03-14T09:02:03.747471+0000","last_unstale":"2024-03-14T09:02:06.682783+0000","last_undegraded":"2024-03-14T09:02:06.682783+0000","last_fullsized":"2024-03-14T09:02:06.682783+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.741318+0000","last_change":"2024-03-14T09:02:03.335208+0000","last_active":"2024-03-14T09:02:03.741318+0000","last_peered":"2024-03-14T09:02:03.741318+0000","last_clean":"2024-03-14T09:02:03.741318+0000","last_became_active":"2024-03-14T09:02:03.334929+0000","last_became_peered":"2024-03-14T09:02:03.334929+0000","last_unstale":"2024-03-14T09:02:03.741318+0000","last_undegraded":"2024-03-14T09:02:03.741318+0000","last_fullsized":"2024-03-14T09:02:03.741318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:05.434989+0000","last_change":"2024-03-14T09:02:03.749453+0000","last_active":"2024-03-14T09:02:05.434989+0000","last_peered":"2024-03-14T09:02:05.434989+0000","last_clean":"2024-03-14T09:02:05.434989+0000","last_became_active":"2024-03-14T09:02:03.749186+0000","last_became_peered":"2024-03-14T09:02:03.749186+0000","last_unstale":"2024-03-14T09:02:05.434989+0000","last_undegraded":"2024-03-14T09:02:05.434989+0000","last_fullsized":"2024-03-14T09:02:05.434989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.766528+0000","last_change":"2024-03-14T09:02:03.330619+0000","last_active":"2024-03-14T09:02:03.766528+0000","last_peered":"2024-03-14T09:02:03.766528+0000","last_clean":"2024-03-14T09:02:03.766528+0000","last_became_active":"2024-03-14T09:02:03.330237+0000","last_became_peered":"2024-03-14T09:02:03.330237+0000","last_unstale":"2024-03-14T09:02:03.766528+0000","last_undegraded":"2024-03-14T09:02:03.766528+0000","last_fullsized":"2024-03-14T09:02:03.766528+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.743908+0000","last_change":"2024-03-14T09:02:03.743189+0000","last_active":"2024-03-14T09:02:04.743908+0000","last_peered":"2024-03-14T09:02:04.743908+0000","last_clean":"2024-03-14T09:02:04.743908+0000","last_became_active":"2024-03-14T09:02:03.742865+0000","last_became_peered":"2024-03-14T09:02:03.742865+0000","last_unstale":"2024-03-14T09:02:04.743908+0000","last_undegraded":"2024-03-14T09:02:04.743908+0000","last_fullsized":"2024-03-14T09:02:04.743908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.753118+0000","last_change":"2024-03-14T09:02:03.328451+0000","last_active":"2024-03-14T09:02:03.753118+0000","last_peered":"2024-03-14T09:02:03.753118+0000","last_clean":"2024-03-14T09:02:03.753118+0000","last_became_active":"2024-03-14T09:02:03.328184+0000","last_became_peered":"2024-03-14T09:02:03.328184+0000","last_unstale":"2024-03-14T09:02:03.753118+0000","last_undegraded":"2024-03-14T09:02:03.753118+0000","last_fullsized":"2024-03-14T09:02:03.753118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742899+0000","last_change":"2024-03-14T09:02:03.746864+0000","last_active":"2024-03-14T09:02:04.742899+0000","last_peered":"2024-03-14T09:02:04.742899+0000","last_clean":"2024-03-14T09:02:04.742899+0000","last_became_active":"2024-03-14T09:02:03.746405+0000","last_became_peered":"2024-03-14T09:02:03.746405+0000","last_unstale":"2024-03-14T09:02:04.742899+0000","last_undegraded":"2024-03-14T09:02:04.742899+0000","last_fullsized":"2024-03-14T09:02:04.742899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.755339+0000","last_change":"2024-03-14T09:02:03.330088+0000","last_active":"2024-03-14T09:02:03.755339+0000","last_peered":"2024-03-14T09:02:03.755339+0000","last_clean":"2024-03-14T09:02:03.755339+0000","last_became_active":"2024-03-14T09:02:03.329788+0000","last_became_peered":"2024-03-14T09:02:03.329788+0000","last_unstale":"2024-03-14T09:02:03.755339+0000","last_undegraded":"2024-03-14T09:02:03.755339+0000","last_fullsized":"2024-03-14T09:02:03.755339+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.214840+0000","last_change":"2024-03-14T09:02:03.746425+0000","last_active":"2024-03-14T09:02:06.214840+0000","last_peered":"2024-03-14T09:02:06.214840+0000","last_clean":"2024-03-14T09:02:06.214840+0000","last_became_active":"2024-03-14T09:02:03.746141+0000","last_became_peered":"2024-03-14T09:02:03.746141+0000","last_unstale":"2024-03-14T09:02:06.214840+0000","last_undegraded":"2024-03-14T09:02:06.214840+0000","last_fullsized":"2024-03-14T09:02:06.214840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.756925+0000","last_change":"2024-03-14T09:02:03.333813+0000","last_active":"2024-03-14T09:02:03.756925+0000","last_peered":"2024-03-14T09:02:03.756925+0000","last_clean":"2024-03-14T09:02:03.756925+0000","last_became_active":"2024-03-14T09:02:03.333439+0000","last_became_peered":"2024-03-14T09:02:03.333439+0000","last_unstale":"2024-03-14T09:02:03.756925+0000","last_undegraded":"2024-03-14T09:02:03.756925+0000","last_fullsized":"2024-03-14T09:02:03.756925+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.739074+0000","last_change":"2024-03-14T09:02:03.750997+0000","last_active":"2024-03-14T09:02:04.739074+0000","last_peered":"2024-03-14T09:02:04.739074+0000","last_clean":"2024-03-14T09:02:04.739074+0000","last_became_active":"2024-03-14T09:02:03.750760+0000","last_became_peered":"2024-03-14T09:02:03.750760+0000","last_unstale":"2024-03-14T09:02:04.739074+0000","last_undegraded":"2024-03-14T09:02:04.739074+0000","last_fullsized":"2024-03-14T09:02:04.739074+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:48.882749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.736775+0000","last_change":"2024-03-14T09:02:03.333934+0000","last_active":"2024-03-14T09:02:03.736775+0000","last_peered":"2024-03-14T09:02:03.736775+0000","last_clean":"2024-03-14T09:02:03.736775+0000","last_became_active":"2024-03-14T09:02:03.332400+0000","last_became_peered":"2024-03-14T09:02:03.332400+0000","last_unstale":"2024-03-14T09:02:03.736775+0000","last_undegraded":"2024-03-14T09:02:03.736775+0000","last_fullsized":"2024-03-14T09:02:03.736775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738762+0000","last_change":"2024-03-14T09:02:03.750636+0000","last_active":"2024-03-14T09:02:04.738762+0000","last_peered":"2024-03-14T09:02:04.738762+0000","last_clean":"2024-03-14T09:02:04.738762+0000","last_became_active":"2024-03-14T09:02:03.750345+0000","last_became_peered":"2024-03-14T09:02:03.750345+0000","last_unstale":"2024-03-14T09:02:04.738762+0000","last_undegraded":"2024-03-14T09:02:04.738762+0000","last_fullsized":"2024-03-14T09:02:04.738762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:21.995078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.760188+0000","last_change":"2024-03-14T09:02:03.332262+0000","last_active":"2024-03-14T09:02:03.760188+0000","last_peered":"2024-03-14T09:02:03.760188+0000","last_clean":"2024-03-14T09:02:03.760188+0000","last_became_active":"2024-03-14T09:02:03.331880+0000","last_became_peered":"2024-03-14T09:02:03.331880+0000","last_unstale":"2024-03-14T09:02:03.760188+0000","last_undegraded":"2024-03-14T09:02:03.760188+0000","last_fullsized":"2024-03-14T09:02:03.760188+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738838+0000","last_change":"2024-03-14T09:02:03.754570+0000","last_active":"2024-03-14T09:02:04.738838+0000","last_peered":"2024-03-14T09:02:04.738838+0000","last_clean":"2024-03-14T09:02:04.738838+0000","last_became_active":"2024-03-14T09:02:03.752638+0000","last_became_peered":"2024-03-14T09:02:03.752638+0000","last_unstale":"2024-03-14T09:02:04.738838+0000","last_undegraded":"2024-03-14T09:02:04.738838+0000","last_fullsized":"2024-03-14T09:02:04.738838+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:51.536705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.739433+0000","last_change":"2024-03-14T09:02:03.331651+0000","last_active":"2024-03-14T09:02:03.739433+0000","last_peered":"2024-03-14T09:02:03.739433+0000","last_clean":"2024-03-14T09:02:03.739433+0000","last_became_active":"2024-03-14T09:02:03.331386+0000","last_became_peered":"2024-03-14T09:02:03.331386+0000","last_unstale":"2024-03-14T09:02:03.739433+0000","last_undegraded":"2024-03-14T09:02:03.739433+0000","last_fullsized":"2024-03-14T09:02:03.739433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.738677+0000","last_change":"2024-03-14T09:02:03.752238+0000","last_active":"2024-03-14T09:02:04.738677+0000","last_peered":"2024-03-14T09:02:04.738677+0000","last_clean":"2024-03-14T09:02:04.738677+0000","last_became_active":"2024-03-14T09:02:03.751848+0000","last_became_peered":"2024-03-14T09:02:03.751848+0000","last_unstale":"2024-03-14T09:02:04.738677+0000","last_undegraded":"2024-03-14T09:02:04.738677+0000","last_fullsized":"2024-03-14T09:02:04.738677+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:12:11.855082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.736615+0000","last_change":"2024-03-14T09:02:03.331842+0000","last_active":"2024-03-14T09:02:03.736615+0000","last_peered":"2024-03-14T09:02:03.736615+0000","last_clean":"2024-03-14T09:02:03.736615+0000","last_became_active":"2024-03-14T09:02:03.331505+0000","last_became_peered":"2024-03-14T09:02:03.331505+0000","last_unstale":"2024-03-14T09:02:03.736615+0000","last_undegraded":"2024-03-14T09:02:03.736615+0000","last_fullsized":"2024-03-14T09:02:03.736615+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682472+0000","last_change":"2024-03-14T09:02:03.751604+0000","last_active":"2024-03-14T09:02:06.682472+0000","last_peered":"2024-03-14T09:02:06.682472+0000","last_clean":"2024-03-14T09:02:06.682472+0000","last_became_active":"2024-03-14T09:02:03.751176+0000","last_became_peered":"2024-03-14T09:02:03.751176+0000","last_unstale":"2024-03-14T09:02:06.682472+0000","last_undegraded":"2024-03-14T09:02:06.682472+0000","last_fullsized":"2024-03-14T09:02:06.682472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.734873+0000","last_change":"2024-03-14T09:02:03.336025+0000","last_active":"2024-03-14T09:02:03.734873+0000","last_peered":"2024-03-14T09:02:03.734873+0000","last_clean":"2024-03-14T09:02:03.734873+0000","last_became_active":"2024-03-14T09:02:03.335598+0000","last_became_peered":"2024-03-14T09:02:03.335598+0000","last_unstale":"2024-03-14T09:02:03.734873+0000","last_undegraded":"2024-03-14T09:02:03.734873+0000","last_fullsized":"2024-03-14T09:02:03.734873+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.739097+0000","last_change":"2024-03-14T09:02:03.751539+0000","last_active":"2024-03-14T09:02:04.739097+0000","last_peered":"2024-03-14T09:02:04.739097+0000","last_clean":"2024-03-14T09:02:04.739097+0000","last_became_active":"2024-03-14T09:02:03.751299+0000","last_became_peered":"2024-03-14T09:02:03.751299+0000","last_unstale":"2024-03-14T09:02:04.739097+0000","last_undegraded":"2024-03-14T09:02:04.739097+0000","last_fullsized":"2024-03-14T09:02:04.739097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:38.365353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.751476+0000","last_change":"2024-03-14T09:02:03.324982+0000","last_active":"2024-03-14T09:02:03.751476+0000","last_peered":"2024-03-14T09:02:03.751476+0000","last_clean":"2024-03-14T09:02:03.751476+0000","last_became_active":"2024-03-14T09:02:03.324739+0000","last_became_peered":"2024-03-14T09:02:03.324739+0000","last_unstale":"2024-03-14T09:02:03.751476+0000","last_undegraded":"2024-03-14T09:02:03.751476+0000","last_fullsized":"2024-03-14T09:02:03.751476+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742822+0000","last_change":"2024-03-14T09:02:03.750602+0000","last_active":"2024-03-14T09:02:04.742822+0000","last_peered":"2024-03-14T09:02:04.742822+0000","last_clean":"2024-03-14T09:02:04.742822+0000","last_became_active":"2024-03-14T09:02:03.750193+0000","last_became_peered":"2024-03-14T09:02:03.750193+0000","last_unstale":"2024-03-14T09:02:04.742822+0000","last_undegraded":"2024-03-14T09:02:04.742822+0000","last_fullsized":"2024-03-14T09:02:04.742822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.255897+0000","last_change":"2024-03-14T09:02:03.331634+0000","last_active":"2024-03-14T09:02:06.255897+0000","last_peered":"2024-03-14T09:02:06.255897+0000","last_clean":"2024-03-14T09:02:06.255897+0000","last_became_active":"2024-03-14T09:02:03.331213+0000","last_became_peered":"2024-03-14T09:02:03.331213+0000","last_unstale":"2024-03-14T09:02:06.255897+0000","last_undegraded":"2024-03-14T09:02:06.255897+0000","last_fullsized":"2024-03-14T09:02:06.255897+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.255731+0000","last_change":"2024-03-14T09:02:03.745980+0000","last_active":"2024-03-14T09:02:06.255731+0000","last_peered":"2024-03-14T09:02:06.255731+0000","last_clean":"2024-03-14T09:02:06.255731+0000","last_became_active":"2024-03-14T09:02:03.745710+0000","last_became_peered":"2024-03-14T09:02:03.745710+0000","last_unstale":"2024-03-14T09:02:06.255731+0000","last_undegraded":"2024-03-14T09:02:06.255731+0000","last_fullsized":"2024-03-14T09:02:06.255731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.733661+0000","last_change":"2024-03-14T09:02:03.329704+0000","last_active":"2024-03-14T09:02:03.733661+0000","last_peered":"2024-03-14T09:02:03.733661+0000","last_clean":"2024-03-14T09:02:03.733661+0000","last_became_active":"2024-03-14T09:02:03.328905+0000","last_became_peered":"2024-03-14T09:02:03.328905+0000","last_unstale":"2024-03-14T09:02:03.733661+0000","last_undegraded":"2024-03-14T09:02:03.733661+0000","last_fullsized":"2024-03-14T09:02:03.733661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:05.435140+0000","last_change":"2024-03-14T09:02:03.750999+0000","last_active":"2024-03-14T09:02:05.435140+0000","last_peered":"2024-03-14T09:02:05.435140+0000","last_clean":"2024-03-14T09:02:05.435140+0000","last_became_active":"2024-03-14T09:02:03.750615+0000","last_became_peered":"2024-03-14T09:02:03.750615+0000","last_unstale":"2024-03-14T09:02:05.435140+0000","last_undegraded":"2024-03-14T09:02:05.435140+0000","last_fullsized":"2024-03-14T09:02:05.435140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.735382+0000","last_change":"2024-03-14T09:02:03.332108+0000","last_active":"2024-03-14T09:02:03.735382+0000","last_peered":"2024-03-14T09:02:03.735382+0000","last_clean":"2024-03-14T09:02:03.735382+0000","last_became_active":"2024-03-14T09:02:03.331581+0000","last_became_peered":"2024-03-14T09:02:03.331581+0000","last_unstale":"2024-03-14T09:02:03.735382+0000","last_undegraded":"2024-03-14T09:02:03.735382+0000","last_fullsized":"2024-03-14T09:02:03.735382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.255854+0000","last_change":"2024-03-14T09:02:03.750281+0000","last_active":"2024-03-14T09:02:06.255854+0000","last_peered":"2024-03-14T09:02:06.255854+0000","last_clean":"2024-03-14T09:02:06.255854+0000","last_became_active":"2024-03-14T09:02:03.750044+0000","last_became_peered":"2024-03-14T09:02:03.750044+0000","last_unstale":"2024-03-14T09:02:06.255854+0000","last_undegraded":"2024-03-14T09:02:06.255854+0000","last_fullsized":"2024-03-14T09:02:06.255854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.731845+0000","last_change":"2024-03-14T09:02:03.323982+0000","last_active":"2024-03-14T09:02:03.731845+0000","last_peered":"2024-03-14T09:02:03.731845+0000","last_clean":"2024-03-14T09:02:03.731845+0000","last_became_active":"2024-03-14T09:02:03.323752+0000","last_became_peered":"2024-03-14T09:02:03.323752+0000","last_unstale":"2024-03-14T09:02:03.731845+0000","last_undegraded":"2024-03-14T09:02:03.731845+0000","last_fullsized":"2024-03-14T09:02:03.731845+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.248818+0000","last_change":"2024-03-14T09:02:03.754008+0000","last_active":"2024-03-14T09:02:06.248818+0000","last_peered":"2024-03-14T09:02:06.248818+0000","last_clean":"2024-03-14T09:02:06.248818+0000","last_became_active":"2024-03-14T09:02:03.753707+0000","last_became_peered":"2024-03-14T09:02:03.753707+0000","last_unstale":"2024-03-14T09:02:06.248818+0000","last_undegraded":"2024-03-14T09:02:06.248818+0000","last_fullsized":"2024-03-14T09:02:06.248818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:23.291407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.736693+0000","last_change":"2024-03-14T09:02:03.332151+0000","last_active":"2024-03-14T09:02:03.736693+0000","last_peered":"2024-03-14T09:02:03.736693+0000","last_clean":"2024-03-14T09:02:03.736693+0000","last_became_active":"2024-03-14T09:02:03.331768+0000","last_became_peered":"2024-03-14T09:02:03.331768+0000","last_unstale":"2024-03-14T09:02:03.736693+0000","last_undegraded":"2024-03-14T09:02:03.736693+0000","last_fullsized":"2024-03-14T09:02:03.736693+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:04.742703+0000","last_change":"2024-03-14T09:02:03.749871+0000","last_active":"2024-03-14T09:02:04.742703+0000","last_peered":"2024-03-14T09:02:04.742703+0000","last_clean":"2024-03-14T09:02:04.742703+0000","last_became_active":"2024-03-14T09:02:03.748885+0000","last_became_peered":"2024-03-14T09:02:03.748885+0000","last_unstale":"2024-03-14T09:02:04.742703+0000","last_undegraded":"2024-03-14T09:02:04.742703+0000","last_fullsized":"2024-03-14T09:02:04.742703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.741639+0000","last_change":"2024-03-14T09:02:03.335789+0000","last_active":"2024-03-14T09:02:03.741639+0000","last_peered":"2024-03-14T09:02:03.741639+0000","last_clean":"2024-03-14T09:02:03.741639+0000","last_became_active":"2024-03-14T09:02:03.335514+0000","last_became_peered":"2024-03-14T09:02:03.335514+0000","last_unstale":"2024-03-14T09:02:03.741639+0000","last_undegraded":"2024-03-14T09:02:03.741639+0000","last_fullsized":"2024-03-14T09:02:03.741639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-14T09:02:06.682627+0000","last_change":"2024-03-14T09:02:03.758083+0000","last_active":"2024-03-14T09:02:06.682627+0000","last_peered":"2024-03-14T09:02:06.682627+0000","last_clean":"2024-03-14T09:02:06.682627+0000","last_became_active":"2024-03-14T09:02:03.757724+0000","last_became_peered":"2024-03-14T09:02:03.757724+0000","last_unstale":"2024-03-14T09:02:06.682627+0000","last_undegraded":"2024-03-14T09:02:06.682627+0000","last_fullsized":"2024-03-14T09:02:06.682627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.739180+0000","last_change":"2024-03-14T09:02:03.333342+0000","last_active":"2024-03-14T09:02:03.739180+0000","last_peered":"2024-03-14T09:02:03.739180+0000","last_clean":"2024-03-14T09:02:03.739180+0000","last_became_active":"2024-03-14T09:02:03.333078+0000","last_became_peered":"2024-03-14T09:02:03.333078+0000","last_unstale":"2024-03-14T09:02:03.739180+0000","last_undegraded":"2024-03-14T09:02:03.739180+0000","last_fullsized":"2024-03-14T09:02:03.739180+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-14T09:02:03.738519+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:02:03.738519+0000","last_peered":"2024-03-14T09:02:03.738519+0000","last_clean":"2024-03-14T09:02:03.738519+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:02:03.738519+0000","last_undegraded":"2024-03-14T09:02:03.738519+0000","last_fullsized":"2024-03-14T09:02:03.738519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":729863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":140991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":729773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":139475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":140991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":139509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:12.787 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:02:12.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T09:02:12.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.951+0000 7f1b94abc700 1 Processor -- start 2024-03-14T09:02:12.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.951+0000 7f1b94abc700 1 -- start start 2024-03-14T09:02:12.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b9013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900c3250 0x7f1b90154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1b901550b0 0x7f1b900c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9013ade0 con 0x7f1b900c3250 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9006fb30 con 0x7f1b901550b0 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b94abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9006ec30 con 0x7f1b900a8540 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b8dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900c3250 0x7f1b90154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b8dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900c3250 0x7f1b90154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58248/0 (socket says 172.21.15.154:58248) 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b8dd9b700 1 -- 172.21.15.154:0/2347345592 learned_addr learned my addr 172.21.15.154:0/2347345592 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b8ed9d700 1 --2- 172.21.15.154:0/2347345592 >> v2:172.21.15.191:3300/0 conn(0x7f1b901550b0 0x7f1b900c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.952+0000 7f1b8e59c700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b9013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 -- 172.21.15.154:0/2347345592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 msgr2=0x7f1b9013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b9013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 -- 172.21.15.154:0/2347345592 >> v2:172.21.15.191:3300/0 conn(0x7f1b901550b0 msgr2=0x7f1b900c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 --2- 172.21.15.154:0/2347345592 >> v2:172.21.15.191:3300/0 conn(0x7f1b901550b0 0x7f1b900c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 -- 172.21.15.154:0/2347345592 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9006ef10 con 0x7f1b900c3250 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8dd9b700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900c3250 0x7f1b90154b70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b900c1440 tx=0x7f1b80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a885e6813bec82 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8d59a700 1 -- 172.21.15.154:0/2347345592 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011830 con 0x7f1b900c3250 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8d59a700 1 -- 172.21.15.154:0/2347345592 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80011990 con 0x7f1b900c3250 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b8d59a700 1 -- 172.21.15.154:0/2347345592 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011d20 con 0x7f1b900c3250 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b94abc700 1 -- 172.21.15.154:0/2347345592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900c3250 msgr2=0x7f1b90154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.953+0000 7f1b94abc700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900c3250 0x7f1b90154b70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b900c1440 tx=0x7f1b80009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 -- 172.21.15.154:0/2347345592 shutdown_connections 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 --2- 172.21.15.154:0/2347345592 >> v2:172.21.15.191:3300/0 conn(0x7f1b901550b0 0x7f1b900c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b9013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 --2- 172.21.15.154:0/2347345592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900c3250 0x7f1b90154b70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 -- 172.21.15.154:0/2347345592 >> 172.21.15.154:0/2347345592 conn(0x7f1b900c3880 msgr2=0x7f1b900c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 -- 172.21.15.154:0/2347345592 shutdown_connections 2024-03-14T09:02:12.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.954+0000 7f1b94abc700 1 -- 172.21.15.154:0/2347345592 wait complete. 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.955+0000 7f1b94abc700 1 Processor -- start 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.955+0000 7f1b94abc700 1 -- start start 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.955+0000 7f1b94abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b901deff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b900c3250 0x7f1b901edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b901ed860 con 0x7f1b901550b0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9014e7f0 con 0x7f1b900c3250 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90150210 con 0x7f1b900a8540 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58250/0 (socket says 172.21.15.154:58250) 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 -- 172.21.15.154:0/1648787487 learned_addr learned my addr 172.21.15.154:0/1648787487 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 -- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 msgr2=0x7f1b901deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b901deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 -- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b900c3250 msgr2=0x7f1b901edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8dd9b700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b900c3250 0x7f1b901edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b900c3250 0x7f1b901edc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 -- 172.21.15.154:0/1648787487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80009550 con 0x7f1b901550b0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b8ed9d700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800efc0 tx=0x7f1b8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88009ba0 con 0x7f1b901550b0 2024-03-14T09:02:12.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800f040 con 0x7f1b901550b0 2024-03-14T09:02:12.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88014480 con 0x7f1b901550b0 2024-03-14T09:02:12.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9014e510 con 0x7f1b901550b0 2024-03-14T09:02:12.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.956+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b90153b80 con 0x7f1b901550b0 2024-03-14T09:02:12.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.957+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b901ed9a0 con 0x7f1b901550b0 2024-03-14T09:02:12.959 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.958+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1b880145e0 con 0x7f1b901550b0 2024-03-14T09:02:12.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.959+0000 7f1b7effd700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 0x7f1b7006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:12.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.959+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b88089a70 con 0x7f1b901550b0 2024-03-14T09:02:12.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.959+0000 7f1b8e59c700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 0x7f1b7006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:12.960 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.960+0000 7f1b8e59c700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 0x7f1b7006ccf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b84006b20 tx=0x7f1b84006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:12.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:12.963+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1b88055000 con 0x7f1b901550b0 2024-03-14T09:02:13.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.237+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1b901ed9a0 con 0x7f1b901550b0 2024-03-14T09:02:13.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.240+0000 7f1b7effd700 1 -- 172.21.15.154:0/1648787487 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1b901ed9a0 con 0x7f1b901550b0 2024-03-14T09:02:13.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 msgr2=0x7f1b7006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 0x7f1b7006ccf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1b84006b20 tx=0x7f1b84006ab0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 msgr2=0x7f1b901ee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800efc0 tx=0x7f1b8800c3a0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 shutdown_connections 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b7006a830 0x7f1b7006ccf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901550b0 0x7f1b901ee1c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b900a8540 0x7f1b901deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 --2- 172.21.15.154:0/1648787487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b900c3250 0x7f1b901edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.242+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 >> 172.21.15.154:0/1648787487 conn(0x7f1b900c3880 msgr2=0x7f1b900c4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.243+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 shutdown_connections 2024-03-14T09:02:13.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.243+0000 7f1b94abc700 1 -- 172.21.15.154:0/1648787487 wait complete. 2024-03-14T09:02:13.244 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2024-03-14T09:02:13.255 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:13.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.376+0000 7fdb14641700 1 Processor -- start 2024-03-14T09:02:13.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.376+0000 7fdb14641700 1 -- start start 2024-03-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb14641700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 0x7fdb0c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb14641700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb14641700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c13ade0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb14641700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c06fb30 con 0x7fdb0c0c1780 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb14641700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c06ec30 con 0x7fdb0c0c3250 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb12bde700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb12bde700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58266/0 (socket says 172.21.15.154:58266) 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.377+0000 7fdb12bde700 1 -- 172.21.15.154:0/1863325093 learned_addr learned my addr 172.21.15.154:0/1863325093 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb11bdc700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 0x7fdb0c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb12bde700 1 -- 172.21.15.154:0/1863325093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 msgr2=0x7fdb0c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb12bde700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 0x7fdb0c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb12bde700 1 -- 172.21.15.154:0/1863325093 >> v1:172.21.15.191:6789/0 conn(0x7fdb0c0a8540 legacy=0x7fdb0c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb12bde700 1 -- 172.21.15.154:0/1863325093 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c06ef10 con 0x7fdb0c0c1780 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb11bdc700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 0x7fdb0c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb12bde700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d150 tx=0x7fdb0400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e399de23886ec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb113db700 1 -- 172.21.15.154:0/1863325093 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04010070 con 0x7fdb0c0c1780 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb113db700 1 -- 172.21.15.154:0/1863325093 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0400ddc0 con 0x7fdb0c0c1780 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.378+0000 7fdb113db700 1 -- 172.21.15.154:0/1863325093 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400b310 con 0x7fdb0c0c1780 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 -- 172.21.15.154:0/1863325093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0c1780 msgr2=0x7fdb0c0c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d150 tx=0x7fdb0400d460 comp rx=0 tx=0).stop 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 -- 172.21.15.154:0/1863325093 shutdown_connections 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0c1780 0x7fdb0c0c1c10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 --2- 172.21.15.154:0/1863325093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c3250 0x7fdb0c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 -- 172.21.15.154:0/1863325093 >> 172.21.15.154:0/1863325093 conn(0x7fdb0c0c3880 msgr2=0x7fdb0c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 -- 172.21.15.154:0/1863325093 shutdown_connections 2024-03-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.379+0000 7fdb14641700 1 -- 172.21.15.154:0/1863325093 wait complete. 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.380+0000 7fdb14641700 1 Processor -- start 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.380+0000 7fdb14641700 1 -- start start 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c1780 0x7fdb0c0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 0x7fdb0c136b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c150ca0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1342f0 con 0x7fdb0c0c3250 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c136320 con 0x7fdb0c0c1780 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58280/0 (socket says 172.21.15.154:58280) 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 -- 172.21.15.154:0/4259618182 learned_addr learned my addr 172.21.15.154:0/4259618182 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb11bdc700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c1780 0x7fdb0c0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb12bde700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 0x7fdb0c136b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 -- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c1780 msgr2=0x7fdb0c0abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c1780 0x7fdb0c0abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 -- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 msgr2=0x7fdb0c136b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 0x7fdb0c136b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 -- 172.21.15.154:0/4259618182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0400bd40 con 0x7fdb0c0a8540 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb12bde700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 0x7fdb0c136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb123dd700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080052d0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb08005430 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c135da0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080038c0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.381+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1e6bd0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.382+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c0697d0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.383+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdb08003a20 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.383+0000 7fdb02ffd700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 0x7fdaf006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.383+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb08087fa0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.383+0000 7fdb11bdc700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 0x7fdaf006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.384+0000 7fdb11bdc700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 0x7fdaf006ccf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006f10 tx=0x7fdafc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:13.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.388+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdb080bb4f0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.629+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb0c13ade0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.629+0000 7fdb02ffd700 1 -- 172.21.15.154:0/4259618182 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4743 (secure 0 0 0) 0x7fdb0c13ade0 con 0x7fdb0c0a8540 2024-03-14T09:02:13.629 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:13.629 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-14T09:02:13.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 msgr2=0x7fdaf006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 0x7fdaf006ccf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006f10 tx=0x7fdafc008040 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 msgr2=0x7fdb0c133ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c760 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 shutdown_connections 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0c3250 0x7fdb0c136b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdaf006a830 0x7fdaf006ccf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb0c0a8540 0x7fdb0c133ca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 --2- 172.21.15.154:0/4259618182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdb0c0c1780 0x7fdb0c0abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 >> 172.21.15.154:0/4259618182 conn(0x7fdb0c0c3880 msgr2=0x7fdb0c0c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 shutdown_connections 2024-03-14T09:02:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:13.631+0000 7fdb14641700 1 -- 172.21.15.154:0/4259618182 wait complete. 2024-03-14T09:02:13.633 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 16 2024-03-14T09:02:14.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T09:02:03.727658+0000', 'modified': '2024-03-14T09:02:06.738355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4956}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4956': {'gid': 4956, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6835/3981292790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6833', 'nonce': 3981292790}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 3981292790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T09:02:14.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T09:02:14.047 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-14T09:02:14.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.170+0000 7f04c552e700 1 Processor -- start 2024-03-14T09:02:14.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.170+0000 7f04c552e700 1 -- start start 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04c552e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04c552e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 0x7f04c015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04c552e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00ac8e0 con 0x7f04c014eaf0 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04c552e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c006b310 con 0x7f04c015d270 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04c552e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c006de30 con 0x7f04c0155df0 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04bf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 0x7f04c015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04bf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 0x7f04c015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47900/0 (socket says 172.21.15.154:47900) 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:33812/0 (socket says 172.21.15.154:33812) 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04beffd700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f04c014eaf0 0x7f04c006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:42588/0 (socket says 172.21.15.154:42588) 2024-03-14T09:02:14.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.171+0000 7f04bf7fe700 1 -- 172.21.15.154:0/3600559283 learned_addr learned my addr 172.21.15.154:0/3600559283 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04be7fc700 1 -- 172.21.15.154:0/3600559283 >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 msgr2=0x7f04c015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04be7fc700 1 --2- 172.21.15.154:0/3600559283 >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 0x7f04c015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04be7fc700 1 -- 172.21.15.154:0/3600559283 >> v1:172.21.15.191:6789/0 conn(0x7f04c014eaf0 legacy=0x7f04c006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04be7fc700 1 -- 172.21.15.154:0/3600559283 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00acb30 con 0x7f04c0155df0 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04be7fc700 1 --2- 172.21.15.154:0/3600559283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009840 tx=0x7f04b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52829a622ec2218f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04bdffb700 1 -- 172.21.15.154:0/3600559283 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011830 con 0x7f04c0155df0 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04bdffb700 1 -- 172.21.15.154:0/3600559283 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0011990 con 0x7f04c0155df0 2024-03-14T09:02:14.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.172+0000 7f04bdffb700 1 -- 172.21.15.154:0/3600559283 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011d20 con 0x7f04c0155df0 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 -- 172.21.15.154:0/3600559283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 msgr2=0x7f04c0156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 --2- 172.21.15.154:0/3600559283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009840 tx=0x7f04b0009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 -- 172.21.15.154:0/3600559283 shutdown_connections 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 --2- 172.21.15.154:0/3600559283 >> v2:172.21.15.191:3300/0 conn(0x7f04c015d270 0x7f04c015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 --2- 172.21.15.154:0/3600559283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c0155df0 0x7f04c0156250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 -- 172.21.15.154:0/3600559283 >> 172.21.15.154:0/3600559283 conn(0x7f04c00c3880 msgr2=0x7f04c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 -- 172.21.15.154:0/3600559283 shutdown_connections 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.173+0000 7f04c552e700 1 -- 172.21.15.154:0/3600559283 wait complete. 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.174+0000 7f04c552e700 1 Processor -- start 2024-03-14T09:02:14.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.174+0000 7f04c552e700 1 -- start start 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c014eaf0 0x7f04c0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c015d270 0x7f04c014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0144b70 con 0x7f04c0155df0 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c01519c0 con 0x7f04c015d270 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04c552e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0151690 con 0x7f04c014eaf0 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58296/0 (socket says 172.21.15.154:58296) 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 -- 172.21.15.154:0/2294994827 learned_addr learned my addr 172.21.15.154:0/2294994827 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04beffd700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c014eaf0 0x7f04c0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c015d270 0x7f04c014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 -- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c014eaf0 msgr2=0x7f04c0151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c014eaf0 0x7f04c0151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 -- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c015d270 msgr2=0x7f04c014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c015d270 0x7f04c014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.175+0000 7f04be7fc700 1 -- 172.21.15.154:0/2294994827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00094a0 con 0x7f04c0155df0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f04b00090b0 tx=0x7f04b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04a77fe700 1 -- 172.21.15.154:0/2294994827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0004400 con 0x7f04c0155df0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04a77fe700 1 -- 172.21.15.154:0/2294994827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b001f070 con 0x7f04c0155df0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c015bd40 con 0x7f04c0155df0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04a77fe700 1 -- 172.21.15.154:0/2294994827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00038e0 con 0x7f04c0155df0 2024-03-14T09:02:14.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.176+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c015cb30 con 0x7f04c0155df0 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.177+0000 7f04a77fe700 1 -- 172.21.15.154:0/2294994827 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f04b0021030 con 0x7f04c0155df0 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.178+0000 7f04a77fe700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 0x7f04ac06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.178+0000 7f04a77fe700 1 -- 172.21.15.154:0/2294994827 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b0089e90 con 0x7f04c0155df0 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.178+0000 7f04beffd700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 0x7f04ac06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.178+0000 7f04beffd700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 0x7f04ac06ccf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04b4000f60 tx=0x7f04b4007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.179+0000 7f04d16e7b80 1 Processor -- start 2024-03-14T09:02:14.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 -- start start 2024-03-14T09:02:14.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc45103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4510920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc444a940 con 0x560cc452a180 2024-03-14T09:02:14.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4448300 con 0x560cc44f0fe0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cc4443bf0 con 0x560cc44f13c0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04bf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58306/0 (socket says 172.21.15.154:58306) 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.180+0000 7f04bf7fe700 1 -- 172.21.15.154:0/3465814297 learned_addr learned my addr 172.21.15.154:0/3465814297 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04be7fc700 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4510920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04beffd700 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc45103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 -- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 msgr2=0x560cc4510920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4510920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 -- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 msgr2=0x560cc45103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc45103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 -- 172.21.15.154:0/3465814297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc42c5360 con 0x560cc452a180 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f04b8011420 tx=0x7f04b8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caca9833ca75b9e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04a4ff9700 1 -- 172.21.15.154:0/3465814297 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800e040 con 0x560cc452a180 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04a4ff9700 1 -- 172.21.15.154:0/3465814297 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800f040 con 0x560cc452a180 2024-03-14T09:02:14.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.181+0000 7f04a4ff9700 1 -- 172.21.15.154:0/3465814297 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8015650 con 0x560cc452a180 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- 172.21.15.154:0/3465814297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 msgr2=0x560cc452a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f04b8011420 tx=0x7f04b8009810 comp rx=0 tx=0).stop 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- 172.21.15.154:0/3465814297 shutdown_connections 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc452a180 0x560cc452a600 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc45103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/3465814297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4510920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- 172.21.15.154:0/3465814297 >> 172.21.15.154:0/3465814297 conn(0x560cc44671d0 msgr2=0x560cc44efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- 172.21.15.154:0/3465814297 shutdown_connections 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- 172.21.15.154:0/3465814297 wait complete. 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.182+0000 7f04d16e7b80 1 -- start start 2024-03-14T09:02:14.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.183+0000 7f04d16e7b80 1 Processor -- start 2024-03-14T09:02:14.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc44f0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc45316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc43dfd30 con 0x560cc44f13c0 2024-03-14T09:02:14.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc43df9d0 con 0x560cc44f0fe0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cc42c7650 con 0x560cc452a180 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 10 client.0 _ll_register_callbacks cb 0x560cc43e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58312/0 (socket says 172.21.15.154:58312) 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 -- 172.21.15.154:0/2236876816 learned_addr learned my addr 172.21.15.154:0/2236876816 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04beffd700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc44f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc45316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 -- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 msgr2=0x560cc45316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc45316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 -- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 msgr2=0x560cc44f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc44f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 -- 172.21.15.154:0/2236876816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b8009e10 con 0x560cc44f13c0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc45316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f04b0088470 tx=0x7f04b00588e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7f464a5bc5323e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.189 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f049dffb700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00593e0 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b0059680 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.184+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc44e4450 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b0058f80 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 10 client.5175 fetch_fsmap learned FSMap version 16 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x560cc4271b10 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560cc44e4450 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f04b0058470 con 0x560cc44f13c0 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 10 client.5175 fetch_fsmap finished waiting for FSMap version 16 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 10 client.5175 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 20 client.5175 populate_metadata read hostname 'smithi154' 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc4541630 0x560cc4541a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 4 client.5175 mds_command: new command op to 4956 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-14T09:02:14.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 --> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc443c6b0 con 0x560cc4541630 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04beffd700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc4541630 0x560cc4541a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f049dffb700 10 client.5175 ms_handle_connect on v2:172.21.15.154:6833/3981292790 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.185+0000 7f04beffd700 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc4541630 0x560cc4541a10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2c9e4f57e8d31c server_cookie=eec289dd90cbf992 in_seq=0 out_seq=0 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f049dffb700 1 -- 172.21.15.154:0/2236876816 <== mds.0 v2:172.21.15.154:6833/3981292790 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560cc443c6b0 con 0x560cc4541630 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f049dffb700 10 client.5175 handle_command_reply: tid=0 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 client.5175 shutdown 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 msgr2=0x560cc452e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f04b0088470 tx=0x7f04b00588e0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 shutdown_connections 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc4541630 0x560cc4541a10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc45316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f0fe0 0x560cc44f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2236876816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f13c0 0x560cc452e010 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 >> 172.21.15.154:0/2236876816 conn(0x560cc44671d0 msgr2=0x560cc452a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f049dffb700 0 client.5175 ms_handle_reset on v2:172.21.15.154:6833/3981292790 2024-03-14T09:02:14.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 shutdown_connections 2024-03-14T09:02:14.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2236876816 wait complete. 2024-03-14T09:02:14.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.186+0000 7f04d16e7b80 20 client.5175 trim_cache size 0 max 16384 2024-03-14T09:02:14.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.203+0000 7f04d16e7b80 1 Processor -- start 2024-03-14T09:02:14.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 -- start start 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 0x560cc44f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc44f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc450d4d0 con 0x560cc44f0fe0 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc452f630 con 0x560cc44f13c0 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cc452af90 con 0x560cc452a180 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58322/0 (socket says 172.21.15.154:58322) 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 -- 172.21.15.154:0/2142037824 learned_addr learned my addr 172.21.15.154:0/2142037824 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 0x560cc44f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc44f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 -- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 msgr2=0x560cc44f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc44f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 -- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 msgr2=0x560cc44f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 0x560cc44f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.204+0000 7f04beffd700 1 -- 172.21.15.154:0/2142037824 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc4409e50 con 0x560cc44f0fe0 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04be7fc700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 0x560cc44f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc44f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04beffd700 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001610 tx=0x7f04b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565f3c7eb37dd8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04a57fa700 1 -- 172.21.15.154:0/2142037824 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4027070 con 0x560cc44f0fe0 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04a57fa700 1 -- 172.21.15.154:0/2142037824 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4001c00 con 0x560cc44f0fe0 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04a57fa700 1 -- 172.21.15.154:0/2142037824 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40135f0 con 0x560cc44f0fe0 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2142037824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 msgr2=0x560cc44eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001610 tx=0x7f04b4013000 comp rx=0 tx=0).stop 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2142037824 shutdown_connections 2024-03-14T09:02:14.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc452a180 0x560cc44f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc44eb2c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/2142037824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc44f13c0 0x560cc44f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2142037824 >> 172.21.15.154:0/2142037824 conn(0x560cc44671d0 msgr2=0x560cc44e83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2142037824 shutdown_connections 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.205+0000 7f04d16e7b80 1 -- 172.21.15.154:0/2142037824 wait complete. 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.206+0000 7f04d16e7b80 1 -- start start 2024-03-14T09:02:14.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.206+0000 7f04d16e7b80 1 Processor -- start 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4593220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc452a180 0x560cc44f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4528040 con 0x560cc44f0fe0 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc4593cb0 con 0x560cc452a180 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cc44eed80 con 0x560cc44f13c0 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 10 client.0 _ll_register_callbacks cb 0x560cc43e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58336/0 (socket says 172.21.15.154:58336) 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 -- 172.21.15.154:0/1912660416 learned_addr learned my addr 172.21.15.154:0/1912660416 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04be7fc700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4593220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04bf7fe700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc452a180 0x560cc44f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 -- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 msgr2=0x560cc4593220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4593220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 -- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc452a180 msgr2=0x560cc44f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc452a180 0x560cc44f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 -- 172.21.15.154:0/1912660416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00ac8e0 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04beffd700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007aa0 tx=0x7f04b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba61abdbdb6c930 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f049dffb700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40239c0 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04b800f040 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b401b040 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04b4023b20 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 10 client.5181 fetch_fsmap learned FSMap version 16 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.207+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x560cc42c9c80 con 0x560cc44f0fe0 2024-03-14T09:02:14.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800f040 con 0x560cc44f0fe0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f04b4023c80 con 0x560cc44f0fe0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 10 client.5181 fetch_fsmap finished waiting for FSMap version 16 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 10 client.5181 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 20 client.5181 populate_metadata read hostname 'smithi154' 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc44f1ce0 0x560cc44f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 4 client.5181 mds_command: new command op to 4956 tid=0[{"prefix": "status"}] 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 --> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560cc4292ca0 con 0x560cc44f1ce0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04be7fc700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc44f1ce0 0x560cc44f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 10 client.5181 ms_handle_connect on v2:172.21.15.154:6833/3981292790 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04be7fc700 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc44f1ce0 0x560cc44f20c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f9203a2d811d827 server_cookie=fd38e95a51e0998a in_seq=0 out_seq=0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 1 -- 172.21.15.154:0/1912660416 <== mds.0 v2:172.21.15.154:6833/3981292790 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560cc4292ca0 con 0x560cc44f1ce0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 10 client.5181 handle_command_reply: tid=0 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 client.5181 shutdown 2024-03-14T09:02:14.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 msgr2=0x560cc453e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007aa0 tx=0x7f04b4000ed0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 shutdown_connections 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560cc452a180 0x560cc44f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560cc44f0fe0 0x560cc453e1b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.154:6833/3981292790,v1:172.21.15.154:6835/3981292790] conn(0x560cc44f1ce0 0x560cc44f20c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 --2- 172.21.15.154:0/1912660416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x560cc44f13c0 0x560cc4593220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f049dffb700 0 client.5181 ms_handle_reset on v2:172.21.15.154:6833/3981292790 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.208+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 >> 172.21.15.154:0/1912660416 conn(0x560cc44671d0 msgr2=0x560cc44f0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 shutdown_connections 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04d16e7b80 1 -- 172.21.15.154:0/1912660416 wait complete. 2024-03-14T09:02:14.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04d16e7b80 20 client.5181 trim_cache size 0 max 16384 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "661f6023-01e4-4d6e-b4b3-5b5701ce7cc5", 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4956, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 10.479927985, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 16, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 32, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 32, 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 17.957877412999999 2024-03-14T09:02:14.216 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:02:14.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 msgr2=0x7f04ac06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 0x7f04ac06ccf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04b4000f60 tx=0x7f04b4007ed0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 msgr2=0x7f04c015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f04b00090b0 tx=0x7f04b0004750 comp rx=0 tx=0).stop 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 shutdown_connections 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c015d270 0x7f04c014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04c014eaf0 0x7f04c0151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04ac06a830 0x7f04ac06ccf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 --2- 172.21.15.154:0/2294994827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04c0155df0 0x7f04c015c5f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 >> 172.21.15.154:0/2294994827 conn(0x7f04c00c3880 msgr2=0x7f04c014dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 shutdown_connections 2024-03-14T09:02:14.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.209+0000 7f04c552e700 1 -- 172.21.15.154:0/2294994827 wait complete. 2024-03-14T09:02:14.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:02:14.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 Processor -- start 2024-03-14T09:02:14.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 -- start start 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f58340702f0 0x7f58340c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f583414fbd0 0x7f58340c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340acd10 con 0x7f58340c78c0 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340ac7b0 con 0x7f583414fbd0 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.347+0000 7f58390e0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5834150050 con 0x7f58340702f0 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f583414fbd0 0x7f58340c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f583414fbd0 0x7f58340c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58348/0 (socket says 172.21.15.154:58348) 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 -- 172.21.15.154:0/1373803805 learned_addr learned my addr 172.21.15.154:0/1373803805 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 -- 172.21.15.154:0/1373803805 >> v2:172.21.15.191:3300/0 conn(0x7f58340702f0 msgr2=0x7f58340c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 --2- 172.21.15.154:0/1373803805 >> v2:172.21.15.191:3300/0 conn(0x7f58340702f0 0x7f58340c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 -- 172.21.15.154:0/1373803805 >> v1:172.21.15.191:6789/0 conn(0x7f58340c78c0 legacy=0x7f58340c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 -- 172.21.15.154:0/1373803805 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340ac9c0 con 0x7f583414fbd0 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f583259c700 1 --2- 172.21.15.154:0/1373803805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340c7380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f58340c8c00 tx=0x7f5828009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0543abd52962cc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f5831d9b700 1 -- 172.21.15.154:0/1373803805 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828004230 con 0x7f583414fbd0 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f5831d9b700 1 -- 172.21.15.154:0/1373803805 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5828004390 con 0x7f583414fbd0 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.348+0000 7f5831d9b700 1 -- 172.21.15.154:0/1373803805 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58280036a0 con 0x7f583414fbd0 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 -- 172.21.15.154:0/1373803805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 msgr2=0x7f58340c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 --2- 172.21.15.154:0/1373803805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340c7380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f58340c8c00 tx=0x7f5828009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 -- 172.21.15.154:0/1373803805 shutdown_connections 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 --2- 172.21.15.154:0/1373803805 >> v2:172.21.15.191:3300/0 conn(0x7f58340702f0 0x7f58340c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 --2- 172.21.15.154:0/1373803805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340c7380 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 -- 172.21.15.154:0/1373803805 >> 172.21.15.154:0/1373803805 conn(0x7f58340a9bf0 msgr2=0x7f58340a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 -- 172.21.15.154:0/1373803805 shutdown_connections 2024-03-14T09:02:14.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.349+0000 7f58390e0700 1 -- 172.21.15.154:0/1373803805 wait complete. 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.350+0000 7f58390e0700 1 Processor -- start 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 -- start start 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58340702f0 0x7f5834141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58340c78c0 0x7f58340a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341dbc30 con 0x7f583414fbd0 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341dc1f0 con 0x7f58340702f0 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.351+0000 7f58390e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583413ec70 con 0x7f58340c78c0 2024-03-14T09:02:14.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58362/0 (socket says 172.21.15.154:58362) 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 -- 172.21.15.154:0/1588220945 learned_addr learned my addr 172.21.15.154:0/1588220945 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583259c700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58340c78c0 0x7f58340a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f5832d9d700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58340702f0 0x7f5834141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 -- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58340c78c0 msgr2=0x7f58340a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58340c78c0 0x7f58340a8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 -- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58340702f0 msgr2=0x7f5834141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58340702f0 0x7f5834141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 -- 172.21.15.154:0/1588220945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828009550 con 0x7f583414fbd0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f583359e700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c7a0 tx=0x7f582c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c003e10 con 0x7f583414fbd0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c005070 con 0x7f583414fbd0 2024-03-14T09:02:14.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c005420 con 0x7f583414fbd0 2024-03-14T09:02:14.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583406f880 con 0x7f583414fbd0 2024-03-14T09:02:14.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.352+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58341dbe80 con 0x7f583414fbd0 2024-03-14T09:02:14.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.353+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58340697d0 con 0x7f583414fbd0 2024-03-14T09:02:14.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.354+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f582c005580 con 0x7f583414fbd0 2024-03-14T09:02:14.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.354+0000 7f581b7fe700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 0x7f582006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:14.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.354+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f582c088060 con 0x7f583414fbd0 2024-03-14T09:02:14.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.354+0000 7f5832d9d700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 0x7f582006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:14.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.354+0000 7f5832d9d700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 0x7f582006ccf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58341426e0 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:14.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.358+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f582c053510 con 0x7f583414fbd0 2024-03-14T09:02:14.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.596+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58340c7160 con 0x7f583414fbd0 2024-03-14T09:02:14.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.596+0000 7f581b7fe700 1 -- 172.21.15.154:0/1588220945 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f582c017060 con 0x7f583414fbd0 2024-03-14T09:02:14.596 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 msgr2=0x7f582006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 0x7f582006ccf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58341426e0 tx=0x7f5824008040 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 msgr2=0x7f58340a90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c7a0 tx=0x7f582c00cb60 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 shutdown_connections 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f582006a830 0x7f582006ccf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583414fbd0 0x7f58340a90a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58340702f0 0x7f5834141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 --2- 172.21.15.154:0/1588220945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58340c78c0 0x7f58340a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 >> 172.21.15.154:0/1588220945 conn(0x7f58340a9bf0 msgr2=0x7f58340a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 shutdown_connections 2024-03-14T09:02:14.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:14.598+0000 7f58390e0700 1 -- 172.21.15.154:0/1588220945 wait complete. 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:02:14.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:02:14.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:14.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.611 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:02:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42325sec preferred_lft 42325sec 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:02:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:02:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:02:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:14.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.631 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:02:14.694 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:02:14.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.695 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:02:14.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.755 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:02:14.756 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:02:14.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:02:14.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:02:14.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:02:14.756 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:02:14.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.862 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:02:14.879 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:02:14.879 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:14.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:14.879 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:02:14.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:14.947 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:15.003 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:15.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:15.003 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:15.061 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T09:02:15.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:15.147 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:15.148 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:15.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:15.169 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:15.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:15.186 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:02:15.186 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:02:15.186 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:02:15.187 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:02:15.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:15.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.215 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:02:15.303 INFO:teuthology.orchestra.run.smithi154.stdout:client5190 2024-03-14T09:02:15.304 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:15.304 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:15.321 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:02:15.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.483+0000 7fe6ff179700 1 Processor -- start 2024-03-14T09:02:15.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.483+0000 7fe6ff179700 1 -- start start 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6ff179700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6f80c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6ff179700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f8070e10 con 0x7fe6f814c5c0 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6ff179700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f814c9a0 con 0x7fe6f80c2510 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6ff179700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c1bb0 con 0x7fe6f8152f80 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6fcf15700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe6f814c5c0 0x7fe6f8072290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:53340/0 (socket says 172.21.15.154:53340) 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6fcf15700 1 -- 172.21.15.154:0/569232248 learned_addr learned my addr 172.21.15.154:0/569232248 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6f80c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644659201 0 0) 0x7fe6f8070e10 con 0x7fe6f814c5c0 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.484+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e80035b0 con 0x7fe6f814c5c0 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410310338 0 0) 0x7fe6e80035b0 con 0x7fe6f814c5c0 2024-03-14T09:02:15.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 msgr2=0x7fe6f80c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6f80c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 >> v1:172.21.15.191:6789/0 conn(0x7fe6f80c2510 legacy=0x7fe6f815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80c1580 con 0x7fe6f814c5c0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fe6ec001670 con 0x7fe6f814c5c0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 >> v1:172.21.15.154:6789/0 conn(0x7fe6f814c5c0 legacy=0x7fe6f8072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e8003a30 0x7fe6e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6e80043e0 0x7fe6e80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6e80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c1bb0 con 0x7fe6e80043e0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e800bac0 con 0x7fe6e8003a30 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e800bc80 con 0x7fe6f8152f80 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6e80043e0 0x7fe6e80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6fcf15700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e8003a30 0x7fe6e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6fd716700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6e80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 -- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 msgr2=0x7fe6e80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6e80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 -- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e8003a30 msgr2=0x7fe6e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e8003a30 0x7fe6e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 -- 172.21.15.154:0/569232248 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80035b0 con 0x7fe6e80043e0 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6fd716700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f8152f80 0x7fe6e80051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6e80043e0 0x7fe6e80048b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b60 tx=0x7fe6e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9df9bf54774d361 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400e6c0 con 0x7fe6e80043e0 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400e820 con 0x7fe6e80043e0 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.485+0000 7fe6f77fe700 1 -- 172.21.15.154:0/569232248 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400eb90 con 0x7fe6e80043e0 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 -- 172.21.15.154:0/569232248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6e80043e0 msgr2=0x7fe6e80048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 --2- 172.21.15.154:0/569232248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6e80043e0 0x7fe6e80048b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009b60 tx=0x7fe6e8005880 comp rx=0 tx=0).stop 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6fd716700 1 -- 172.21.15.154:0/569232248 reap_dead start 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 -- 172.21.15.154:0/569232248 shutdown_connections 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 -- 172.21.15.154:0/569232248 >> 172.21.15.154:0/569232248 conn(0x7fe6f806a770 msgr2=0x7fe6f806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:15.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 -- 172.21.15.154:0/569232248 shutdown_connections 2024-03-14T09:02:15.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.486+0000 7fe6ff179700 1 -- 172.21.15.154:0/569232248 wait complete. 2024-03-14T09:02:15.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.487+0000 7fe6ff179700 1 Processor -- start 2024-03-14T09:02:15.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.487+0000 7fe6ff179700 1 -- start start 2024-03-14T09:02:15.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e80043e0 0x7fe6f814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f806c940 0x7fe6f81ed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f806ea00 con 0x7fe6f8153d30 2024-03-14T09:02:15.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f81538a0 con 0x7fe6e80043e0 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6ff179700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f814bce0 con 0x7fe6f806c940 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58388/0 (socket says 172.21.15.154:58388) 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 -- 172.21.15.154:0/1283743908 learned_addr learned my addr 172.21.15.154:0/1283743908 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6fcf15700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e80043e0 0x7fe6f814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6fd716700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f806c940 0x7fe6f81ed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 -- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f806c940 msgr2=0x7fe6f81ed4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f806c940 0x7fe6f81ed4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 -- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e80043e0 msgr2=0x7fe6f814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e80043e0 0x7fe6f814b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 -- 172.21.15.154:0/1283743908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f81de3a0 con 0x7fe6f8153d30 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.488+0000 7fe6f7fff700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8005810 tx=0x7fe6e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.489+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400e6c0 con 0x7fe6f8153d30 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.489+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e40038c0 con 0x7fe6f8153d30 2024-03-14T09:02:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.489+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4003c70 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.489+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f81de680 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.489+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f814b880 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.490+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f81de200 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.490+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe6e4013470 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.490+0000 7fe6f57fa700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 0x7fe6dc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.490+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6e408a470 con 0x7fe6f8153d30 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.491+0000 7fe6fcf15700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 0x7fe6dc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.491+0000 7fe6fcf15700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 0x7fe6dc06ccf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6f81ee560 tx=0x7fe6ec001de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:15.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.495+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe6f81de200 con 0x7fe6f8153d30 2024-03-14T09:02:15.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.733+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6f81de200 con 0x7fe6f8153d30 2024-03-14T09:02:15.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.734+0000 7fe6f57fa700 1 -- 172.21.15.154:0/1283743908 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe6f81de200 con 0x7fe6f8153d30 2024-03-14T09:02:15.734 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 msgr2=0x7fe6dc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 0x7fe6dc06ccf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe6f81ee560 tx=0x7fe6ec001de0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 msgr2=0x7fe6f80bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8005810 tx=0x7fe6e40047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 shutdown_connections 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6f806c940 0x7fe6f81ed4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6e80043e0 0x7fe6f814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6dc06a830 0x7fe6dc06ccf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 --2- 172.21.15.154:0/1283743908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6f8153d30 0x7fe6f80bf9c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 >> 172.21.15.154:0/1283743908 conn(0x7fe6f806a770 msgr2=0x7fe6f80a7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:15.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 shutdown_connections 2024-03-14T09:02:15.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:15.736+0000 7fe6ff179700 1 -- 172.21.15.154:0/1283743908 wait complete. 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:15.748 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:02:15.749 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:02:15.749 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:02:15.749 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:15.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.749 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:02:15.771 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:02:15.771 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:02:15.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:02:15.771 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:15.772 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42329sec preferred_lft 42329sec 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:02:15.773 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:02:15.774 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:15.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.774 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:02:15.843 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:02:15.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.845 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:02:15.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:02:15.909 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:02:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:16.020 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:02:16.038 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:02:16.038 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:16.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:16.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:02:16.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:16.108 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:16.165 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:16.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:16.165 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:16.221 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T09:02:16.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:16.221 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:16.306 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:16.306 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:16.326 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:16.326 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:16.327 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:16.345 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:02:16.346 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:02:16.346 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:02:16.348 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:02:16.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:16.348 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:16.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:16.375 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:02:16.447 INFO:teuthology.orchestra.run.smithi191.stdout:client5199 2024-03-14T09:02:16.447 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:16.447 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:16.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-03-14T09:02:16.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.636+0000 7f59f2f97700 1 Processor -- start 2024-03-14T09:02:16.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.636+0000 7f59f2f97700 1 -- start start 2024-03-14T09:02:16.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f2f97700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59ec0702f0 0x7f59ec0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f2f97700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f59ec14fbd0 0x7f59ec0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f2f97700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0acd10 con 0x7f59ec0c78c0 2024-03-14T09:02:16.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f2f97700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0ac7b0 con 0x7f59ec0702f0 2024-03-14T09:02:16.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f2f97700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec150050 con 0x7f59ec14fbd0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f0d33700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59ec0702f0 0x7f59ec0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f0d33700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59ec0702f0 0x7f59ec0c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58400/0 (socket says 172.21.15.154:58400) 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.637+0000 7f59f0d33700 1 -- 172.21.15.154:0/2027512043 learned_addr learned my addr 172.21.15.154:0/2027512043 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59ebfff700 1 --2- 172.21.15.154:0/2027512043 >> v2:172.21.15.191:3300/0 conn(0x7f59ec14fbd0 0x7f59ec0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f0d33700 1 -- 172.21.15.154:0/2027512043 >> v2:172.21.15.191:3300/0 conn(0x7f59ec14fbd0 msgr2=0x7f59ec0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f0d33700 1 --2- 172.21.15.154:0/2027512043 >> v2:172.21.15.191:3300/0 conn(0x7f59ec14fbd0 0x7f59ec0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f0d33700 1 -- 172.21.15.154:0/2027512043 >> v1:172.21.15.191:6789/0 conn(0x7f59ec0c78c0 legacy=0x7f59ec0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f0d33700 1 -- 172.21.15.154:0/2027512043 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0ac9c0 con 0x7f59ec0702f0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f0d33700 1 --2- 172.21.15.154:0/2027512043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec0c4380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0c5980 tx=0x7f59e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce4e911bd715d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59eb7fe700 1 -- 172.21.15.154:0/2027512043 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000b870 con 0x7f59ec0702f0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59eb7fe700 1 -- 172.21.15.154:0/2027512043 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e000b9d0 con 0x7f59ec0702f0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59eb7fe700 1 -- 172.21.15.154:0/2027512043 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000bd60 con 0x7f59ec0702f0 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f2f97700 1 -- 172.21.15.154:0/2027512043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 msgr2=0x7f59ec0c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.638+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2027512043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec0c4380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0c5980 tx=0x7f59e0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 -- 172.21.15.154:0/2027512043 shutdown_connections 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2027512043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec0c4380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2027512043 >> v2:172.21.15.191:3300/0 conn(0x7f59ec14fbd0 0x7f59ec0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 -- 172.21.15.154:0/2027512043 >> 172.21.15.154:0/2027512043 conn(0x7f59ec0a9bf0 msgr2=0x7f59ec0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 -- 172.21.15.154:0/2027512043 shutdown_connections 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.639+0000 7f59f2f97700 1 -- 172.21.15.154:0/2027512043 wait complete. 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.640+0000 7f59f2f97700 1 Processor -- start 2024-03-14T09:02:16.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.640+0000 7f59f2f97700 1 -- start start 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.640+0000 7f59f2f97700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59ec0c78c0 0x7f59ec1e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59ec14fbd0 0x7f59ec145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec151830 con 0x7f59ec0702f0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec149e70 con 0x7f59ec14fbd0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec1511b0 con 0x7f59ec0c78c0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58402/0 (socket says 172.21.15.154:58402) 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 -- 172.21.15.154:0/2659678038 learned_addr learned my addr 172.21.15.154:0/2659678038 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f1534700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59ec14fbd0 0x7f59ec145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59ebfff700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59ec0c78c0 0x7f59ec1e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 -- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59ec0c78c0 msgr2=0x7f59ec1e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59ec0c78c0 0x7f59ec1e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 -- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59ec14fbd0 msgr2=0x7f59ec145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59ec14fbd0 0x7f59ec145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 -- 172.21.15.154:0/2659678038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0009550 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f0d33700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f59e0000c00 tx=0x7f59e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004540 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e00036a0 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0003a30 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec14a0c0 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.641+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec14a3a0 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.642+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec151970 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.643+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f59e0021070 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.643+0000 7f59e97fa700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 0x7f59d006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.643+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59e008a4c0 con 0x7f59ec0702f0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.643+0000 7f59ebfff700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 0x7f59d006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:16.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.644+0000 7f59ebfff700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 0x7f59d006ccf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59ec1522e0 tx=0x7f59d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:16.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.648+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f59ec151970 con 0x7f59ec0702f0 2024-03-14T09:02:16.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.920+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f59ec151970 con 0x7f59ec0702f0 2024-03-14T09:02:16.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.920+0000 7f59e97fa700 1 -- 172.21.15.154:0/2659678038 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v15) v1 ==== 84+0+0 (secure 0 0 0) 0x7f59ec151970 con 0x7f59ec0702f0 2024-03-14T09:02:16.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.922+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 msgr2=0x7f59d006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.922+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 0x7f59d006ccf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59ec1522e0 tx=0x7f59d8008040 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.922+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 msgr2=0x7f59ec151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.922+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f59e0000c00 tx=0x7f59e0004750 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 shutdown_connections 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59ec14fbd0 0x7f59ec145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59d006a830 0x7f59d006ccf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ec0702f0 0x7f59ec151c50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 --2- 172.21.15.154:0/2659678038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59ec0c78c0 0x7f59ec1e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 >> 172.21.15.154:0/2659678038 conn(0x7f59ec0a9bf0 msgr2=0x7f59ec14d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 shutdown_connections 2024-03-14T09:02:16.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:16.923+0000 7f59f2f97700 1 -- 172.21.15.154:0/2659678038 wait complete. 2024-03-14T09:02:16.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:17.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.058+0000 7fd22b935700 1 Processor -- start 2024-03-14T09:02:17.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.058+0000 7fd22b935700 1 -- start start 2024-03-14T09:02:17.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd22b935700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2240c4a80 0x7fd2240c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd22b935700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd224158370 0x7fd224158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd22b935700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22414ed60 con 0x7fd22414e980 2024-03-14T09:02:17.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd22b935700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240c6b10 con 0x7fd2240c4a80 2024-03-14T09:02:17.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd22b935700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240c6cf0 con 0x7fd224158370 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2240c4a80 0x7fd2240c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2240c4a80 0x7fd2240c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58408/0 (socket says 172.21.15.154:58408) 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 -- 172.21.15.154:0/1462120905 learned_addr learned my addr 172.21.15.154:0/1462120905 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd229ed2700 1 --2- 172.21.15.154:0/1462120905 >> v2:172.21.15.191:3300/0 conn(0x7fd224158370 0x7fd224158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 -- 172.21.15.154:0/1462120905 >> v2:172.21.15.191:3300/0 conn(0x7fd224158370 msgr2=0x7fd224158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 --2- 172.21.15.154:0/1462120905 >> v2:172.21.15.191:3300/0 conn(0x7fd224158370 0x7fd224158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 -- 172.21.15.154:0/1462120905 >> v1:172.21.15.191:6789/0 conn(0x7fd22414e980 legacy=0x7fd2241552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.059+0000 7fd228ed0700 1 -- 172.21.15.154:0/1462120905 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240c6fd0 con 0x7fd2240c4a80 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd228ed0700 1 --2- 172.21.15.154:0/1462120905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2240c4a80 0x7fd2240c4f10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd224152430 tx=0x7fd220009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeacb8c580a79a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd21bfff700 1 -- 172.21.15.154:0/1462120905 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220011830 con 0x7fd2240c4a80 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd21bfff700 1 -- 172.21.15.154:0/1462120905 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220011990 con 0x7fd2240c4a80 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd21bfff700 1 -- 172.21.15.154:0/1462120905 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220011d20 con 0x7fd2240c4a80 2024-03-14T09:02:17.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 -- 172.21.15.154:0/1462120905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2240c4a80 msgr2=0x7fd2240c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 --2- 172.21.15.154:0/1462120905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2240c4a80 0x7fd2240c4f10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd224152430 tx=0x7fd220009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 -- 172.21.15.154:0/1462120905 shutdown_connections 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 --2- 172.21.15.154:0/1462120905 >> v2:172.21.15.191:3300/0 conn(0x7fd224158370 0x7fd224158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 --2- 172.21.15.154:0/1462120905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2240c4a80 0x7fd2240c4f10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 -- 172.21.15.154:0/1462120905 >> 172.21.15.154:0/1462120905 conn(0x7fd2240abcf0 msgr2=0x7fd2240ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 -- 172.21.15.154:0/1462120905 shutdown_connections 2024-03-14T09:02:17.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.060+0000 7fd22b935700 1 -- 172.21.15.154:0/1462120905 wait complete. 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.062+0000 7fd22b935700 1 Processor -- start 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.062+0000 7fd22b935700 1 -- start start 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 0x7fd22414ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd22414e980 0x7fd22414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240af220 con 0x7fd224158370 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240c02e0 con 0x7fd2240c4a80 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd22b935700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240c05c0 con 0x7fd22414e980 2024-03-14T09:02:17.065 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58410/0 (socket says 172.21.15.154:58410) 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 -- 172.21.15.154:0/2671045751 learned_addr learned my addr 172.21.15.154:0/2671045751 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd2296d1700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 0x7fd22414ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd228ed0700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd22414e980 0x7fd22414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 -- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd22414e980 msgr2=0x7fd22414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd22414e980 0x7fd22414b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 -- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 msgr2=0x7fd22414ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 0x7fd22414ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 -- 172.21.15.154:0/2671045751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd220009550 con 0x7fd224158370 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd2296d1700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 0x7fd22414ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd229ed2700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00efc0 tx=0x7fd21c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c009ba0 con 0x7fd224158370 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.063+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c00f040 con 0x7fd224158370 2024-03-14T09:02:17.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.064+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2240c08a0 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.064+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c014480 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.064+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2240acff0 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.065+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2240697d0 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.065+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd21c0145e0 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.065+0000 7fd219ffb700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 0x7fd20806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.065+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd21c089690 con 0x7fd224158370 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.066+0000 7fd2296d1700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 0x7fd20806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.066+0000 7fd2296d1700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 0x7fd20806ccf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd214000f60 tx=0x7fd214009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.070+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd21c054aa0 con 0x7fd224158370 2024-03-14T09:02:17.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.262+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd22406cc80 con 0x7fd224158370 2024-03-14T09:02:17.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.263+0000 7fd219ffb700 1 -- 172.21.15.154:0/2671045751 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16149 (secure 0 0 0) 0x7fd21c0580c0 con 0x7fd224158370 2024-03-14T09:02:17.263 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:17.263 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:04.729742+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 msgr2=0x7fd20806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 0x7fd20806ccf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd214000f60 tx=0x7fd214009040 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 msgr2=0x7fd2241ed5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00efc0 tx=0x7fd21c00c3a0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 shutdown_connections 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd20806a830 0x7fd20806ccf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158370 0x7fd2241ed5a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2240c4a80 0x7fd22414ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 --2- 172.21.15.154:0/2671045751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd22414e980 0x7fd22414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 >> 172.21.15.154:0/2671045751 conn(0x7fd2240abcf0 msgr2=0x7fd22414c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 shutdown_connections 2024-03-14T09:02:17.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.266+0000 7fd22b935700 1 -- 172.21.15.154:0/2671045751 wait complete. 2024-03-14T09:02:17.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:01.699649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:02.688126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:17.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 Processor -- start 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 -- start start 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb17c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb17c072710 0x7fb17c0c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c070e10 con 0x7fb17c072290 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17c0c1bb0 con 0x7fb17c072710 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.405+0000 7fb1820e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0c6170 con 0x7fb17c14c5c0 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17affd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb17c072290 0x7fb17c0c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:53348/0 (socket says 172.21.15.154:53348) 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17affd700 1 -- 172.21.15.154:0/1890812705 learned_addr learned my addr 172.21.15.154:0/1890812705 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17bfff700 1 --2- 172.21.15.154:0/1890812705 >> v2:172.21.15.191:3300/0 conn(0x7fb17c072710 0x7fb17c0c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17b7fe700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb17c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677107900 0 0) 0x7fb17c070e10 con 0x7fb17c072290 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1640035b0 con 0x7fb17c072290 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2027573020 0 0) 0x7fb1640035b0 con 0x7fb17c072290 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 msgr2=0x7fb17c152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb17c152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 >> v2:172.21.15.191:3300/0 conn(0x7fb17c072710 msgr2=0x7fb17c0c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 --2- 172.21.15.154:0/1890812705 >> v2:172.21.15.191:3300/0 conn(0x7fb17c072710 0x7fb17c0c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c14c9a0 con 0x7fb17c072290 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fb170001670 con 0x7fb17c072290 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 >> v1:172.21.15.154:6789/0 conn(0x7fb17c072290 legacy=0x7fb17c0c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb164003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb164004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb164004d90 0x7fb164005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0c1bb0 con 0x7fb164003f80 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0c6170 con 0x7fb164004d90 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16400ba60 con 0x7fb17c14c5c0 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17affd700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb164004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17b7fe700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb164003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.406+0000 7fb17bfff700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb164004d90 0x7fb164005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 -- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 msgr2=0x7fb164003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14c5c0 0x7fb164003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 -- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb164004d90 msgr2=0x7fb164005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb164004d90 0x7fb164005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 -- 172.21.15.154:0/1890812705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1640035b0 con 0x7fb164003f80 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17affd700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb164004850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c270 tx=0x7fb164005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8f3fa719003b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000cdd0 con 0x7fb164003f80 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb170007ba0 con 0x7fb164003f80 2024-03-14T09:02:17.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17a7fc700 1 -- 172.21.15.154:0/1890812705 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170005b80 con 0x7fb164003f80 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb1820e7700 1 -- 172.21.15.154:0/1890812705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 msgr2=0x7fb164004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb1820e7700 1 --2- 172.21.15.154:0/1890812705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb164004850 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c270 tx=0x7fb164005880 comp rx=0 tx=0).stop 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb17bfff700 1 -- 172.21.15.154:0/1890812705 reap_dead start 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb1820e7700 1 -- 172.21.15.154:0/1890812705 shutdown_connections 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.407+0000 7fb1820e7700 1 -- 172.21.15.154:0/1890812705 >> 172.21.15.154:0/1890812705 conn(0x7fb17c06a770 msgr2=0x7fb17c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.408+0000 7fb1820e7700 1 -- 172.21.15.154:0/1890812705 shutdown_connections 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.408+0000 7fb1820e7700 1 -- 172.21.15.154:0/1890812705 wait complete. 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.409+0000 7fb1820e7700 1 Processor -- start 2024-03-14T09:02:17.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.409+0000 7fb1820e7700 1 -- start start 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.409+0000 7fb1820e7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.409+0000 7fb1820e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb17c14f340 0x7fb17c14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.409+0000 7fb1820e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14e8d0 0x7fb17c0a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb1820e7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0aa750 con 0x7fb164003f80 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb1820e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c14fa90 con 0x7fb17c14f340 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb1820e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0ac370 con 0x7fb17c14e8d0 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58428/0 (socket says 172.21.15.154:58428) 2024-03-14T09:02:17.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 -- 172.21.15.154:0/3422042697 learned_addr learned my addr 172.21.15.154:0/3422042697 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 -- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14e8d0 msgr2=0x7fb17c0a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14e8d0 0x7fb17c0a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 -- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb17c14f340 msgr2=0x7fb17c14e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb17c14f340 0x7fb17c14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 -- 172.21.15.154:0/3422042697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0ac610 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb17b7fe700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb16c011ad0 tx=0x7fb16c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00f980 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c00fae0 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c14d8b0 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00fe90 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.410+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c14de70 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.411+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c0697d0 con 0x7fb164003f80 2024-03-14T09:02:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.412+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb16c014440 con 0x7fb164003f80 2024-03-14T09:02:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.412+0000 7fb163fff700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 0x7fb16806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.412+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb16c089330 con 0x7fb164003f80 2024-03-14T09:02:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.412+0000 7fb17affd700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 0x7fb16806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.413+0000 7fb17affd700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 0x7fb16806ccf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb17c1de450 tx=0x7fb17000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.416+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb16c0548a0 con 0x7fb164003f80 2024-03-14T09:02:17.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.609+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb17c0c53a0 con 0x7fb164003f80 2024-03-14T09:02:17.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.609+0000 7fb163fff700 1 -- 172.21.15.154:0/3422042697 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb16c057ec0 con 0x7fb164003f80 2024-03-14T09:02:17.610 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 msgr2=0x7fb16806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 0x7fb16806ccf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb17c1de450 tx=0x7fb17000f000 comp rx=0 tx=0).stop 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 msgr2=0x7fb17c1dc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb16c011ad0 tx=0x7fb16c011e90 comp rx=0 tx=0).stop 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 shutdown_connections 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb17c14e8d0 0x7fb17c0a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb16806a830 0x7fb16806ccf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb164003f80 0x7fb17c1dc990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 --2- 172.21.15.154:0/3422042697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb17c14f340 0x7fb17c14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 >> 172.21.15.154:0/3422042697 conn(0x7fb17c06a770 msgr2=0x7fb17c0c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 shutdown_connections 2024-03-14T09:02:17.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.612+0000 7fb1820e7700 1 -- 172.21.15.154:0/3422042697 wait complete. 2024-03-14T09:02:17.624 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:02:17.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.748+0000 7f82befed700 1 Processor -- start 2024-03-14T09:02:17.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.748+0000 7f82befed700 1 -- start start 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82befed700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f82b814eaf0 0x7f82b806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82befed700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82b806acb0 0x7f82b8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82befed700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b80ac8e0 con 0x7f82b8156330 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82befed700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b806b310 con 0x7f82b806acb0 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82befed700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b806de30 con 0x7f82b814eaf0 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82b7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82b806acb0 0x7f82b8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82b7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82b806acb0 0x7f82b8155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58438/0 (socket says 172.21.15.154:58438) 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.749+0000 7f82b7fff700 1 -- 172.21.15.154:0/1064746385 learned_addr learned my addr 172.21.15.154:0/1064746385 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82bcd89700 1 --2- 172.21.15.154:0/1064746385 >> v2:172.21.15.191:3300/0 conn(0x7f82b814eaf0 0x7f82b806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b7fff700 1 -- 172.21.15.154:0/1064746385 >> v2:172.21.15.191:3300/0 conn(0x7f82b814eaf0 msgr2=0x7f82b806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b7fff700 1 --2- 172.21.15.154:0/1064746385 >> v2:172.21.15.191:3300/0 conn(0x7f82b814eaf0 0x7f82b806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b7fff700 1 -- 172.21.15.154:0/1064746385 >> v1:172.21.15.191:6789/0 conn(0x7f82b8156330 legacy=0x7f82b8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b7fff700 1 -- 172.21.15.154:0/1064746385 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80acb30 con 0x7f82b806acb0 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b7fff700 1 --2- 172.21.15.154:0/1064746385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b806acb0 0x7f82b8155df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f82b8070da0 tx=0x7f82a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01f135d0fe795da server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b77fe700 1 -- 172.21.15.154:0/1064746385 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4004230 con 0x7f82b806acb0 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b77fe700 1 -- 172.21.15.154:0/1064746385 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4004390 con 0x7f82b806acb0 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.750+0000 7f82b77fe700 1 -- 172.21.15.154:0/1064746385 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40036a0 con 0x7f82b806acb0 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 -- 172.21.15.154:0/1064746385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b806acb0 msgr2=0x7f82b8155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 --2- 172.21.15.154:0/1064746385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b806acb0 0x7f82b8155df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f82b8070da0 tx=0x7f82a4009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 -- 172.21.15.154:0/1064746385 shutdown_connections 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 --2- 172.21.15.154:0/1064746385 >> v2:172.21.15.191:3300/0 conn(0x7f82b814eaf0 0x7f82b806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 --2- 172.21.15.154:0/1064746385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b806acb0 0x7f82b8155df0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 -- 172.21.15.154:0/1064746385 >> 172.21.15.154:0/1064746385 conn(0x7f82b80c3880 msgr2=0x7f82b80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 -- 172.21.15.154:0/1064746385 shutdown_connections 2024-03-14T09:02:17.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.751+0000 7f82befed700 1 -- 172.21.15.154:0/1064746385 wait complete. 2024-03-14T09:02:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.752+0000 7f82befed700 1 Processor -- start 2024-03-14T09:02:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.752+0000 7f82befed700 1 -- start start 2024-03-14T09:02:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82b806acb0 0x7f82b8153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 0x7f82b81ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8154880 con 0x7f82b814eaf0 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8153c70 con 0x7f82b8156330 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8153f50 con 0x7f82b806acb0 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58442/0 (socket says 172.21.15.154:58442) 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 -- 172.21.15.154:0/3780193908 learned_addr learned my addr 172.21.15.154:0/3780193908 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82bcd89700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82b806acb0 0x7f82b8153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82bd58a700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 0x7f82b81ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 -- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82b806acb0 msgr2=0x7f82b8153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82b806acb0 0x7f82b8153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 -- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 msgr2=0x7f82b81ed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 0x7f82b81ed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 -- 172.21.15.154:0/3780193908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009550 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82bd58a700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 0x7f82b81ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b7fff700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009130 tx=0x7f82a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4011630 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a4011790 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4011b40 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b8154230 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.753+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b81537d0 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.754+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b81546f0 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.755+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f82a4011ca0 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.755+0000 7f82b57fa700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 0x7f829c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.755+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82a408a360 con 0x7f82b814eaf0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.755+0000 7f82bcd89700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 0x7f829c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:17.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.756+0000 7f82bcd89700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 0x7f829c06cd20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82b80c3850 tx=0x7f82ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:17.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.760+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f82a4055820 con 0x7f82b814eaf0 2024-03-14T09:02:17.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.952+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82b8154a30 con 0x7f82b814eaf0 2024-03-14T09:02:17.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.952+0000 7f82b57fa700 1 -- 172.21.15.154:0/3780193908 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82b8154a30 con 0x7f82b814eaf0 2024-03-14T09:02:17.952 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:17.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 msgr2=0x7f829c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 0x7f829c06cd20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82b80c3850 tx=0x7f82ac008040 comp rx=0 tx=0).stop 2024-03-14T09:02:17.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 msgr2=0x7f82b8147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009130 tx=0x7f82a4004750 comp rx=0 tx=0).stop 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 shutdown_connections 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82b8156330 0x7f82b81ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82b806acb0 0x7f82b8153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f829c06a860 0x7f829c06cd20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 --2- 172.21.15.154:0/3780193908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82b814eaf0 0x7f82b8147620 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 >> 172.21.15.154:0/3780193908 conn(0x7f82b80c3880 msgr2=0x7f82b8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 shutdown_connections 2024-03-14T09:02:17.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:17.954+0000 7f82befed700 1 -- 172.21.15.154:0/3780193908 wait complete. 2024-03-14T09:02:17.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:02:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.090+0000 7fadf7e89700 1 Processor -- start 2024-03-14T09:02:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.090+0000 7fadf7e89700 1 -- start start 2024-03-14T09:02:18.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf7e89700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf7e89700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fadf0156500 0x7fadf015f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf7e89700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00ac8e0 con 0x7fadf006acb0 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf7e89700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf006b310 con 0x7fadf0156500 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf7e89700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf006de30 con 0x7fadf014eaf0 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf6426700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fadf0156500 0x7fadf015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf6426700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fadf0156500 0x7fadf015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:48036/0 (socket says 172.21.15.154:48036) 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf6426700 1 -- 172.21.15.154:0/2852512406 learned_addr learned my addr 172.21.15.154:0/2852512406 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf5424700 1 --1- 172.21.15.154:0/2852512406 >> v1:172.21.15.191:6789/0 conn(0x7fadf006acb0 0x7fadf0155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:42642/0 (socket says 172.21.15.154:42642) 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.091+0000 7fadf5c25700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf6426700 1 -- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 msgr2=0x7fadf006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:18.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf6426700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf6426700 1 -- 172.21.15.154:0/2852512406 >> v1:172.21.15.191:6789/0 conn(0x7fadf006acb0 legacy=0x7fadf0155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf6426700 1 -- 172.21.15.154:0/2852512406 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0156950 con 0x7fadf0156500 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf4c23700 1 -- 172.21.15.154:0/2852512406 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641476786 0 0) 0x7fadf00ac8e0 con 0x7fadf006acb0 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf6426700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf0156500 0x7fadf015f280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33feca5c696c32a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf4c23700 1 -- 172.21.15.154:0/2852512406 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800e070 con 0x7fadf0156500 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf4c23700 1 -- 172.21.15.154:0/2852512406 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade800b910 con 0x7fadf0156500 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.092+0000 7fadf4c23700 1 -- 172.21.15.154:0/2852512406 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800bcc0 con 0x7fadf0156500 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 -- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf0156500 msgr2=0x7fadf015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf0156500 0x7fadf015f280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 -- 172.21.15.154:0/2852512406 shutdown_connections 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf0156500 0x7fadf015f280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 --2- 172.21.15.154:0/2852512406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 -- 172.21.15.154:0/2852512406 >> 172.21.15.154:0/2852512406 conn(0x7fadf00c3880 msgr2=0x7fadf00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:18.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 -- 172.21.15.154:0/2852512406 shutdown_connections 2024-03-14T09:02:18.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.093+0000 7fadf7e89700 1 -- 172.21.15.154:0/2852512406 wait complete. 2024-03-14T09:02:18.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.094+0000 7fadf7e89700 1 Processor -- start 2024-03-14T09:02:18.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.094+0000 7fadf7e89700 1 -- start start 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf006acb0 0x7fadf0141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf0141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01530e0 con 0x7fadf0156500 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0142090 con 0x7fadf006acb0 2024-03-14T09:02:18.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0152bb0 con 0x7fadf014eaf0 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58452/0 (socket says 172.21.15.154:58452) 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 -- 172.21.15.154:0/96435366 learned_addr learned my addr 172.21.15.154:0/96435366 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf5c25700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf006acb0 0x7fadf0141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf5424700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf0141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 -- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 msgr2=0x7fadf0141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf0141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 -- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf006acb0 msgr2=0x7fadf0141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf006acb0 0x7fadf0141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 -- 172.21.15.154:0/96435366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8009b30 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf5424700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf0141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf6426700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fade800b680 tx=0x7fade8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8004960 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade80092e0 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade801c640 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.095+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf0152e00 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.096+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf00c25a0 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.096+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf00c2120 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.097+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fade801c7a0 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.097+0000 7fade67fc700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 0x7fadd406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.097+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fade808a520 con 0x7fadf0156500 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.097+0000 7fadf5c25700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 0x7fadd406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.098+0000 7fadf5c25700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 0x7fadd406ccf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadf0154900 tx=0x7fade0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.102+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fadf00c2120 con 0x7fadf0156500 2024-03-14T09:02:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.293+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadf00c2120 con 0x7fadf0156500 2024-03-14T09:02:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.293+0000 7fade67fc700 1 -- 172.21.15.154:0/96435366 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadf00c2120 con 0x7fadf0156500 2024-03-14T09:02:18.293 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:18.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 msgr2=0x7fadd406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 0x7fadd406ccf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadf0154900 tx=0x7fade0009210 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 msgr2=0x7fadf0153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fade800b680 tx=0x7fade8004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 shutdown_connections 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fadd406a830 0x7fadd406ccf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.295+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fadf0156500 0x7fadf0153490 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.296+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadf006acb0 0x7fadf0141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.296+0000 7fadf7e89700 1 --2- 172.21.15.154:0/96435366 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fadf014eaf0 0x7fadf0141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.296+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 >> 172.21.15.154:0/96435366 conn(0x7fadf00c3880 msgr2=0x7fadf0156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.296+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 shutdown_connections 2024-03-14T09:02:18.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.296+0000 7fadf7e89700 1 -- 172.21.15.154:0/96435366 wait complete. 2024-03-14T09:02:18.307 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:02:18.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.429+0000 7f051f6aa700 1 Processor -- start 2024-03-14T09:02:18.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.429+0000 7f051f6aa700 1 -- start start 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051f6aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 0x7f051815df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051f6aa700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05180acd10 con 0x7f05180c4380 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051f6aa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05180ac9c0 con 0x7f05180702f0 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051f6aa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05180ac7b0 con 0x7f051814fbd0 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051cc45700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f05180c4380 0x7f05180c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:53364/0 (socket says 172.21.15.154:53364) 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.430+0000 7f051cc45700 1 -- 172.21.15.154:0/37696897 learned_addr learned my addr 172.21.15.154:0/37696897 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f051dc47700 1 --2- 172.21.15.154:0/37696897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 0x7f051815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385922489 0 0) 0x7f05180acd10 con 0x7f05180c4380 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05040035b0 con 0x7f05180c4380 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367225259 0 0) 0x7f05180ac9c0 con 0x7f05180702f0 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05180acd10 con 0x7f05180702f0 2024-03-14T09:02:18.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f051dc47700 1 -- 172.21.15.154:0/37696897 >> v1:172.21.15.191:6789/0 conn(0x7f05180702f0 legacy=0x7f05180c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f051dc47700 1 -- 172.21.15.154:0/37696897 >> v1:172.21.15.154:6789/0 conn(0x7f05180c4380 legacy=0x7f05180c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403322690 0 0) 0x7f05040035b0 con 0x7f05180c4380 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f051dc47700 1 -- 172.21.15.154:0/37696897 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518150020 con 0x7f051814fbd0 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f051dc47700 1 --2- 172.21.15.154:0/37696897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 0x7f051815df60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0510009dc0 tx=0x7f051000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32862c5d437d62a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000e070 con 0x7f051814fbd0 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510003a20 con 0x7f051814fbd0 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.431+0000 7f050bfff700 1 -- 172.21.15.154:0/37696897 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c001670 con 0x7f051814fbd0 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 -- 172.21.15.154:0/37696897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 msgr2=0x7f051815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 --2- 172.21.15.154:0/37696897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 0x7f051815df60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0510009dc0 tx=0x7f051000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 -- 172.21.15.154:0/37696897 shutdown_connections 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 --2- 172.21.15.154:0/37696897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f051814fbd0 0x7f051815df60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 -- 172.21.15.154:0/37696897 >> 172.21.15.154:0/37696897 conn(0x7f05180a9bf0 msgr2=0x7f05180a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 -- 172.21.15.154:0/37696897 shutdown_connections 2024-03-14T09:02:18.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.432+0000 7f051f6aa700 1 -- 172.21.15.154:0/37696897 wait complete. 2024-03-14T09:02:18.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.433+0000 7f051f6aa700 1 Processor -- start 2024-03-14T09:02:18.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.433+0000 7f051f6aa700 1 -- start start 2024-03-14T09:02:18.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05180702f0 0x7f051806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 0x7f0518154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0518156320 con 0x7f051814fbd0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05181567e0 con 0x7f05180702f0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05181530a0 con 0x7f05180c4380 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58464/0 (socket says 172.21.15.154:58464) 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 -- 172.21.15.154:0/2583401535 learned_addr learned my addr 172.21.15.154:0/2583401535 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051d446700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05180702f0 0x7f051806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051cc45700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 0x7f0518154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 -- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 msgr2=0x7f0518154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 0x7f0518154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 -- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05180702f0 msgr2=0x7f051806f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05180702f0 0x7f051806f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 -- 172.21.15.154:0/2583401535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0510009a70 con 0x7f051814fbd0 2024-03-14T09:02:18.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051cc45700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 0x7f0518154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051dc47700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0510009d90 tx=0x7f0510004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510004340 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05100044c0 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05181534f0 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.434+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510004870 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.435+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05181ee0b0 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.435+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05180697d0 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.436+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f051001c460 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.436+0000 7f0509ffb700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 0x7f04fc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.436+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f051008a260 con 0x7f051814fbd0 2024-03-14T09:02:18.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.437+0000 7f051d446700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 0x7f04fc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.437+0000 7f051d446700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 0x7f04fc06ccf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05140011e0 tx=0x7f0514009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.441+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0510059120 con 0x7f051814fbd0 2024-03-14T09:02:18.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.638+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05180c2d70 con 0x7f051814fbd0 2024-03-14T09:02:18.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.639+0000 7f0509ffb700 1 -- 172.21.15.154:0/2583401535 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+207 (secure 0 0 0) 0x7f0510058cb0 con 0x7f051814fbd0 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 4, 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_metadata" 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:18.639 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 5, 2024-03-14T09:02:18.640 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_data" 2024-03-14T09:02:18.640 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:18.640 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 msgr2=0x7f04fc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 0x7f04fc06ccf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05140011e0 tx=0x7f0514009040 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 msgr2=0x7f05181539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0510009d90 tx=0x7f0510004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 shutdown_connections 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f04fc06a830 0x7f04fc06ccf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f051814fbd0 0x7f05181539c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05180702f0 0x7f051806f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 --2- 172.21.15.154:0/2583401535 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05180c4380 0x7f0518154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 >> 172.21.15.154:0/2583401535 conn(0x7f05180a9bf0 msgr2=0x7f05180a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 shutdown_connections 2024-03-14T09:02:18.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.641+0000 7f051f6aa700 1 -- 172.21.15.154:0/2583401535 wait complete. 2024-03-14T09:02:18.653 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-14T09:02:18.653 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-14T09:02:18.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.778+0000 7f8d3f1ae700 1 Processor -- start 2024-03-14T09:02:18.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.778+0000 7f8d3f1ae700 1 -- start start 2024-03-14T09:02:18.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3f1ae700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d3806d050 0x7f8d38158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3f1ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8d38155060 0x7f8d380c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3f1ae700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d38158c90 con 0x7f8d381554e0 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3f1ae700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3806b110 con 0x7f8d3806d050 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3f1ae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3806ab50 con 0x7f8d38155060 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d3806d050 0x7f8d38158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3d74b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8d381554e0 0x7f8d380c18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:53368/0 (socket says 172.21.15.154:53368) 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3d74b700 1 -- 172.21.15.154:0/2709688002 learned_addr learned my addr 172.21.15.154:0/2709688002 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d37fff700 1 --2- 172.21.15.154:0/2709688002 >> v2:172.21.15.191:3300/0 conn(0x7f8d38155060 0x7f8d380c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 -- 172.21.15.154:0/2709688002 >> v2:172.21.15.191:3300/0 conn(0x7f8d38155060 msgr2=0x7f8d380c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/2709688002 >> v2:172.21.15.191:3300/0 conn(0x7f8d38155060 0x7f8d380c2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 -- 172.21.15.154:0/2709688002 >> v1:172.21.15.154:6789/0 conn(0x7f8d381554e0 legacy=0x7f8d380c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d377fe700 1 -- 172.21.15.154:0/2709688002 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227108176 0 0) 0x7f8d38158c90 con 0x7f8d381554e0 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 -- 172.21.15.154:0/2709688002 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3806ae30 con 0x7f8d3806d050 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/2709688002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d3806d050 0x7f8d38158750 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d380723f0 tx=0x7f8d2c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f6d4afca30e693 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d377fe700 1 -- 172.21.15.154:0/2709688002 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c011830 con 0x7f8d3806d050 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d377fe700 1 -- 172.21.15.154:0/2709688002 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c011990 con 0x7f8d3806d050 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.779+0000 7f8d377fe700 1 -- 172.21.15.154:0/2709688002 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c011d20 con 0x7f8d3806d050 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/2709688002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d3806d050 msgr2=0x7f8d38158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/2709688002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d3806d050 0x7f8d38158750 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8d380723f0 tx=0x7f8d2c009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/2709688002 shutdown_connections 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/2709688002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d3806d050 0x7f8d38158750 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/2709688002 >> v2:172.21.15.191:3300/0 conn(0x7f8d38155060 0x7f8d380c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/2709688002 >> 172.21.15.154:0/2709688002 conn(0x7f8d380c39f0 msgr2=0x7f8d380c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:18.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/2709688002 shutdown_connections 2024-03-14T09:02:18.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.780+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/2709688002 wait complete. 2024-03-14T09:02:18.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.781+0000 7f8d3f1ae700 1 Processor -- start 2024-03-14T09:02:18.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.782+0000 7f8d3f1ae700 1 -- start start 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.782+0000 7f8d3f1ae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 0x7f8d380c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.782+0000 7f8d3f1ae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d381554e0 0x7f8d3814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38148a60 con 0x7f8d38155060 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3814a3b0 con 0x7f8d3806d050 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381edb10 con 0x7f8d381554e0 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:58492/0 (socket says 172.21.15.154:58492) 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 -- 172.21.15.154:0/3290825150 learned_addr learned my addr 172.21.15.154:0/3290825150 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 0x7f8d380c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3d74b700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d381554e0 0x7f8d3814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 -- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d381554e0 msgr2=0x7f8d3814ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d381554e0 0x7f8d3814ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 -- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 msgr2=0x7f8d380c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 0x7f8d380c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 -- 172.21.15.154:0/3290825150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0094a0 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 0x7f8d380c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d37fff700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c7f0 tx=0x7f8d2400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24011800 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24011960 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d381eddf0 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24011cf0 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.783+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d38148f50 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.784+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d380697d0 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.785+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8d24018360 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.785+0000 7f8d357fa700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 0x7f8d1c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.785+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d24088d50 con 0x7f8d38155060 2024-03-14T09:02:18.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.785+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 0x7f8d1c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:18.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.786+0000 7f8d3cf4a700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 0x7f8d1c06ccf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0090b0 tx=0x7f8d2c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:18.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.789+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8d2408c030 con 0x7f8d38155060 2024-03-14T09:02:18.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:18.983+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d3814b8e0 con 0x7f8d38155060 2024-03-14T09:02:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.337+0000 7f8d357fa700 1 -- 172.21.15.154:0/3290825150 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v33) v1 ==== 140+0+0 (secure 0 0 0) 0x7f8d24057c10 con 0x7f8d38155060 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 msgr2=0x7f8d1c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 0x7f8d1c06ccf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0090b0 tx=0x7f8d2c008030 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 msgr2=0x7f8d3814a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c7f0 tx=0x7f8d2400cbb0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 shutdown_connections 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d381554e0 0x7f8d3814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d3806d050 0x7f8d380c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d1c06a830 0x7f8d1c06ccf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 --2- 172.21.15.154:0/3290825150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d38155060 0x7f8d3814a8d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 >> 172.21.15.154:0/3290825150 conn(0x7f8d380c39f0 msgr2=0x7f8d38071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 shutdown_connections 2024-03-14T09:02:19.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.340+0000 7f8d3f1ae700 1 -- 172.21.15.154:0/3290825150 wait complete. 2024-03-14T09:02:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' created 2024-03-14T09:02:19.356 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:02:19.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.484+0000 7fd13515f700 1 Processor -- start 2024-03-14T09:02:19.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd13515f700 1 -- start start 2024-03-14T09:02:19.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd13515f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd1300a8540 0x7fd13013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd13515f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13013ade0 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd13515f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13006fb30 con 0x7fd1300c3250 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd13515f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13006ec30 con 0x7fd1300a8540 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd12f59e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1300c1780 0x7fd1300c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33662/0 (socket says 172.21.15.154:33662) 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.485+0000 7fd12f59e700 1 -- 172.21.15.154:0/4174431297 learned_addr learned my addr 172.21.15.154:0/4174431297 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138128728 0 0) 0x7fd128001490 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12ed9d700 1 --2- 172.21.15.154:0/4174431297 >> v2:172.21.15.191:3300/0 conn(0x7fd1300a8540 0x7fd13013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1180035b0 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967739121 0 0) 0x7fd1180035b0 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 >> v2:172.21.15.191:3300/0 conn(0x7fd1300a8540 msgr2=0x7fd13013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 --2- 172.21.15.154:0/4174431297 >> v2:172.21.15.191:3300/0 conn(0x7fd1300a8540 0x7fd13013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 >> v1:172.21.15.191:6789/0 conn(0x7fd1300c3250 legacy=0x7fd130154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13006ef10 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fd128001b80 con 0x7fd1300c1780 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 >> v1:172.21.15.154:6789/0 conn(0x7fd1300c1780 legacy=0x7fd1300c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd118003a00 0x7fd118003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 0x7fd118004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118005330 0x7fd11800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13006ec30 con 0x7fd118004850 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11800c200 con 0x7fd118003a00 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11800e5d0 con 0x7fd118005330 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 0x7fd118004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12ed9d700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd118003a00 0x7fd118003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12f59e700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118005330 0x7fd11800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 -- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118005330 msgr2=0x7fd11800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118005330 0x7fd11800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 -- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd118003a00 msgr2=0x7fd118003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd118003a00 0x7fd118003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12e59c700 1 -- 172.21.15.154:0/4174431297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180035b0 con 0x7fd118004850 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.486+0000 7fd12f59e700 1 -- 172.21.15.154:0/4174431297 reap_dead start 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd12e59c700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 0x7fd118004cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd120007fd0 tx=0x7fd118005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2a5cd1efc22293 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:19.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200052d0 con 0x7fd118004850 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120005470 con 0x7fd118004850 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd12dd9b700 1 -- 172.21.15.154:0/4174431297 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200038c0 con 0x7fd118004850 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 -- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 msgr2=0x7fd118004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 0x7fd118004cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd120007fd0 tx=0x7fd118005880 comp rx=0 tx=0).stop 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 -- 172.21.15.154:0/4174431297 shutdown_connections 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 --2- 172.21.15.154:0/4174431297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd118004850 0x7fd118004cd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 -- 172.21.15.154:0/4174431297 >> 172.21.15.154:0/4174431297 conn(0x7fd1300c3880 msgr2=0x7fd1300c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 -- 172.21.15.154:0/4174431297 shutdown_connections 2024-03-14T09:02:19.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.487+0000 7fd13515f700 1 -- 172.21.15.154:0/4174431297 wait complete. 2024-03-14T09:02:19.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.489+0000 7fd13515f700 1 Processor -- start 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.489+0000 7fd13515f700 1 -- start start 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.489+0000 7fd13515f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118004850 0x7fd13014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.489+0000 7fd13515f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.489+0000 7fd13515f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1301e9860 0x7fd1301512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd13515f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13014ea50 con 0x7fd13014e5d0 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd13515f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130150820 con 0x7fd1301e9860 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd13515f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301daf50 con 0x7fd118004850 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47704/0 (socket says 172.21.15.154:47704) 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 -- 172.21.15.154:0/2422993244 learned_addr learned my addr 172.21.15.154:0/2422993244 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 -- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118004850 msgr2=0x7fd13014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12ed9d700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118004850 0x7fd13014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12f59e700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1301e9860 0x7fd1301512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118004850 0x7fd13014e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 -- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1301e9860 msgr2=0x7fd1301512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1301e9860 0x7fd1301512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 -- 172.21.15.154:0/2422993244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130150ac0 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd12e59c700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd120000c00 tx=0x7fd120001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120001290 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120007040 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd130150da0 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1200073b0 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.490+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301db230 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.491+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1300697d0 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.492+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd120007510 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.492+0000 7fd1177fe700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 0x7fd11c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.493+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6080+0+0 (secure 0 0 0) 0x7fd120089a10 con 0x7fd13014e5d0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.493+0000 7fd12ed9d700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 0x7fd11c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:19.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.493+0000 7fd12ed9d700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 0x7fd11c06ccf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1300695e0 tx=0x7fd124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:19.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.497+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd120054df0 con 0x7fd13014e5d0 2024-03-14T09:02:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:19.693+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd13006d420 con 0x7fd13014e5d0 2024-03-14T09:02:20.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.344+0000 7fd1177fe700 1 -- 172.21.15.154:0/2422993244 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v34) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd1200210d0 con 0x7fd13014e5d0 2024-03-14T09:02:20.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.353+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 msgr2=0x7fd11c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.353+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 0x7fd11c06ccf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1300695e0 tx=0x7fd124008040 comp rx=0 tx=0).stop 2024-03-14T09:02:20.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.353+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 msgr2=0x7fd1301e9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.353+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd120000c00 tx=0x7fd120001120 comp rx=0 tx=0).stop 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 shutdown_connections 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1301e9860 0x7fd1301512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd118004850 0x7fd13014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd11c06a830 0x7fd11c06ccf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 --2- 172.21.15.154:0/2422993244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd13014e5d0 0x7fd1301e9320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 >> 172.21.15.154:0/2422993244 conn(0x7fd1300c3880 msgr2=0x7fd1300c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.355+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 shutdown_connections 2024-03-14T09:02:20.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.356+0000 7fd13515f700 1 -- 172.21.15.154:0/2422993244 wait complete. 2024-03-14T09:02:20.358 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' created 2024-03-14T09:02:20.373 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-14T09:02:20.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.498+0000 7f67cb2ca700 1 Processor -- start 2024-03-14T09:02:20.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.498+0000 7f67cb2ca700 1 -- start start 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67cb2ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67c40c4380 0x7f67c414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67cb2ca700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40acd10 con 0x7f67c40c5d60 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67cb2ca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40ac9c0 con 0x7f67c40702f0 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67cb2ca700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c4150110 con 0x7f67c40c4380 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c8865700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67c40c4380 0x7f67c414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c9867700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f67c40c5d60 0x7f67c415cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33664/0 (socket says 172.21.15.154:33664) 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c9867700 1 -- 172.21.15.154:0/3455354168 learned_addr learned my addr 172.21.15.154:0/3455354168 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:20.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c8865700 1 -- 172.21.15.154:0/3455354168 >> v1:172.21.15.191:6789/0 conn(0x7f67c40702f0 legacy=0x7f67c40c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c8865700 1 -- 172.21.15.154:0/3455354168 >> v1:172.21.15.154:6789/0 conn(0x7f67c40c5d60 legacy=0x7f67c415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c8865700 1 -- 172.21.15.154:0/3455354168 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40707f0 con 0x7f67c40c4380 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67bbfff700 1 -- 172.21.15.154:0/3455354168 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066305174 0 0) 0x7f67c40acd10 con 0x7f67c40c5d60 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67c8865700 1 --2- 172.21.15.154:0/3455354168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c4380 0x7f67c414fbd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009840 tx=0x7f67b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c585855e6e4e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67bbfff700 1 -- 172.21.15.154:0/3455354168 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4004230 con 0x7f67c40c4380 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67bbfff700 1 -- 172.21.15.154:0/3455354168 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4004390 con 0x7f67c40c4380 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.499+0000 7f67bbfff700 1 -- 172.21.15.154:0/3455354168 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40036a0 con 0x7f67c40c4380 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 -- 172.21.15.154:0/3455354168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c4380 msgr2=0x7f67c414fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/3455354168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c4380 0x7f67c414fbd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009840 tx=0x7f67b4009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 -- 172.21.15.154:0/3455354168 shutdown_connections 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/3455354168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c4380 0x7f67c414fbd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 -- 172.21.15.154:0/3455354168 >> 172.21.15.154:0/3455354168 conn(0x7f67c40a9bf0 msgr2=0x7f67c40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 -- 172.21.15.154:0/3455354168 shutdown_connections 2024-03-14T09:02:20.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.500+0000 7f67cb2ca700 1 -- 172.21.15.154:0/3455354168 wait complete. 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.501+0000 7f67cb2ca700 1 Processor -- start 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.501+0000 7f67cb2ca700 1 -- start start 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c40702f0 0x7f67c41ede20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c40c4380 0x7f67c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c414cb40 con 0x7f67c40c5d60 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c414d070 con 0x7f67c40702f0 2024-03-14T09:02:20.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67cb2ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41ee360 con 0x7f67c40c4380 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47726/0 (socket says 172.21.15.154:47726) 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 -- 172.21.15.154:0/2930774872 learned_addr learned my addr 172.21.15.154:0/2930774872 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9066700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c40702f0 0x7f67c41ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c8865700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c40c4380 0x7f67c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 -- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c40c4380 msgr2=0x7f67c406f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c40c4380 0x7f67c406f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 -- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c40702f0 msgr2=0x7f67c41ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c40702f0 0x7f67c41ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.502+0000 7f67c9867700 1 -- 172.21.15.154:0/2930774872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40094a0 con 0x7f67c40c5d60 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67c9867700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00cba0 tx=0x7f67bc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00aaa0 con 0x7f67c40c5d60 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc00ac20 con 0x7f67c40c5d60 2024-03-14T09:02:20.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c414cd90 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0070d0 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.503+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c406d1b0 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.504+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c40697d0 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.504+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f67bc007230 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.504+0000 7f67b9ffb700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 0x7f67a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.505+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f67bc088a50 con 0x7f67c40c5d60 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.505+0000 7f67c9066700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 0x7f67a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.505+0000 7f67c9066700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 0x7f67a806ccf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007fd0 tx=0x7f67c0007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.509+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f67bc053c10 con 0x7f67c40c5d60 2024-03-14T09:02:20.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.751+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f67c41491f0 con 0x7f67c40c5d60 2024-03-14T09:02:20.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.758+0000 7f67b9ffb700 1 -- 172.21.15.154:0/2930774872 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 6 and data pool 7 v17) v1 ==== 175+0+0 (secure 0 0 0) 0x7f67bc018070 con 0x7f67c40c5d60 2024-03-14T09:02:20.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.761+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 msgr2=0x7f67a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.761+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 0x7f67a806ccf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007fd0 tx=0x7f67c0007ec0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.761+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 msgr2=0x7f67c4143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.761+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00cba0 tx=0x7f67bc00ceb0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 shutdown_connections 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f67a806a830 0x7f67a806ccf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67c40c5d60 0x7f67c4143b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c40702f0 0x7f67c41ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 --2- 172.21.15.154:0/2930774872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c40c4380 0x7f67c406f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.762+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 >> 172.21.15.154:0/2930774872 conn(0x7f67c40a9bf0 msgr2=0x7f67c414ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:20.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.764+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 shutdown_connections 2024-03-14T09:02:20.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.766+0000 7f67cb2ca700 1 -- 172.21.15.154:0/2930774872 wait complete. 2024-03-14T09:02:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 6 and data pool 7 2024-03-14T09:02:20.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:20.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.911+0000 7ff1f38b1700 1 Processor -- start 2024-03-14T09:02:20.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.911+0000 7ff1f38b1700 1 -- start start 2024-03-14T09:02:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f38b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff1ec06c6e0 0x7ff1ec06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f38b1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ec070a10 0x7ff1ec0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f38b1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0c5f10 con 0x7ff1ec06d600 2024-03-14T09:02:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f38b1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c7d50 con 0x7ff1ec070a10 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f38b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c8030 con 0x7ff1ec06c6e0 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ec070a10 0x7ff1ec0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f1e4e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1ec06d600 0x7ff1ec0c68e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33680/0 (socket says 172.21.15.154:33680) 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f1e4e700 1 -- 172.21.15.154:0/4154509188 learned_addr learned my addr 172.21.15.154:0/4154509188 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f164d700 1 --2- 172.21.15.154:0/4154509188 >> v2:172.21.15.191:3300/0 conn(0x7ff1ec06c6e0 0x7ff1ec06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/4154509188 >> v2:172.21.15.191:3300/0 conn(0x7ff1ec06c6e0 msgr2=0x7ff1ec06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 --2- 172.21.15.154:0/4154509188 >> v2:172.21.15.191:3300/0 conn(0x7ff1ec06c6e0 0x7ff1ec06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/4154509188 >> v1:172.21.15.154:6789/0 conn(0x7ff1ec06d600 legacy=0x7ff1ec0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/4154509188 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0c8310 con 0x7ff1ec070a10 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1e3fff700 1 -- 172.21.15.154:0/4154509188 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944981381 0 0) 0x7ff1ec0c5f10 con 0x7ff1ec06d600 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1f0e4c700 1 --2- 172.21.15.154:0/4154509188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec070a10 0x7ff1ec0c63a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec073f80 tx=0x7ff1dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d720c4e7f6580d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1e3fff700 1 -- 172.21.15.154:0/4154509188 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011830 con 0x7ff1ec070a10 2024-03-14T09:02:20.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1e3fff700 1 -- 172.21.15.154:0/4154509188 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc011990 con 0x7ff1ec070a10 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.912+0000 7ff1e3fff700 1 -- 172.21.15.154:0/4154509188 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc011d20 con 0x7ff1ec070a10 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 -- 172.21.15.154:0/4154509188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec070a10 msgr2=0x7ff1ec0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/4154509188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec070a10 0x7ff1ec0c63a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec073f80 tx=0x7ff1dc009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 -- 172.21.15.154:0/4154509188 shutdown_connections 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/4154509188 >> v2:172.21.15.191:3300/0 conn(0x7ff1ec06c6e0 0x7ff1ec06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/4154509188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec070a10 0x7ff1ec0c63a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 -- 172.21.15.154:0/4154509188 >> 172.21.15.154:0/4154509188 conn(0x7ff1ec0a9bf0 msgr2=0x7ff1ec0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 -- 172.21.15.154:0/4154509188 shutdown_connections 2024-03-14T09:02:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.913+0000 7ff1f38b1700 1 -- 172.21.15.154:0/4154509188 wait complete. 2024-03-14T09:02:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.914+0000 7ff1f38b1700 1 Processor -- start 2024-03-14T09:02:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.914+0000 7ff1f38b1700 1 -- start start 2024-03-14T09:02:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec06c6e0 0x7ff1ec1ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1ec070a10 0x7ff1ec1ea140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec09bcb0 con 0x7ff1ec06d600 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1ed6b0 con 0x7ff1ec06c6e0 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f38b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1ee600 con 0x7ff1ec070a10 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f0e4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f0e4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47734/0 (socket says 172.21.15.154:47734) 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/583532771 learned_addr learned my addr 172.21.15.154:0/583532771 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f164d700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec06c6e0 0x7ff1ec1ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.915+0000 7ff1f1e4e700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1ec070a10 0x7ff1ec1ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1ec070a10 msgr2=0x7ff1ec1ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1ec070a10 0x7ff1ec1ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec06c6e0 msgr2=0x7ff1ec1ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec06c6e0 0x7ff1ec1ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 -- 172.21.15.154:0/583532771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0094a0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f0e4c700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc014040 tx=0x7ff1dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0036a0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc01e070 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003a60 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0a6fc0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.916+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec1ed8c0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.917+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec0a7f10 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.918+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff1dc003bc0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.918+0000 7ff1e1ffb700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 0x7ff1d006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.918+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff1dc089eb0 con 0x7ff1ec06d600 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.918+0000 7ff1f164d700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 0x7ff1d006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.919+0000 7ff1f164d700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 0x7ff1d006ccf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0acb90 tx=0x7ff1e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:20.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:20.922+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff1dc055000 con 0x7ff1ec06d600 2024-03-14T09:02:21.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.118+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1ec0697d0 con 0x7ff1ec06d600 2024-03-14T09:02:21.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.119+0000 7ff1e1ffb700 1 -- 172.21.15.154:0/583532771 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+19822 (secure 0 0 0) 0x7ff1dc015090 con 0x7ff1ec06d600 2024-03-14T09:02:21.119 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:21.119 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:20.752977+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:21.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.121+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 msgr2=0x7ff1d006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.121+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 0x7ff1d006ccf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0acb90 tx=0x7ff1e8008040 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.121+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 msgr2=0x7ff1ec1dc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.121+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc014040 tx=0x7ff1dc004750 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 shutdown_connections 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1ec070a10 0x7ff1ec1ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec06c6e0 0x7ff1ec1ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff1d006a830 0x7ff1d006ccf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 --2- 172.21.15.154:0/583532771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec06d600 0x7ff1ec1dc5e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 >> 172.21.15.154:0/583532771 conn(0x7ff1ec0a9bf0 msgr2=0x7ff1ec0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 shutdown_connections 2024-03-14T09:02:21.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.122+0000 7ff1f38b1700 1 -- 172.21.15.154:0/583532771 wait complete. 2024-03-14T09:02:21.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:21.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.256+0000 7fc332187700 1 Processor -- start 2024-03-14T09:02:21.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.256+0000 7fc332187700 1 -- start start 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc332187700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc32c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc332187700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c0ac8e0 con 0x7fc32c156500 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc332187700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c06de30 con 0x7fc32c06acb0 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc332187700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c06b310 con 0x7fc32c14eaf0 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc32bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc32c156500 0x7fc32c15f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33690/0 (socket says 172.21.15.154:33690) 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.257+0000 7fc32bfff700 1 -- 172.21.15.154:0/3053176306 learned_addr learned my addr 172.21.15.154:0/3053176306 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32b7fe700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc32c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428652282 0 0) 0x7fc32c0ac8e0 con 0x7fc32c156500 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3140035b0 con 0x7fc32c156500 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134958870 0 0) 0x7fc3140035b0 con 0x7fc32c156500 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 msgr2=0x7fc32c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc32c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 >> v1:172.21.15.191:6789/0 conn(0x7fc32c06acb0 legacy=0x7fc32c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c156950 con 0x7fc32c156500 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fc3240019f0 con 0x7fc32c156500 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 >> v1:172.21.15.154:6789/0 conn(0x7fc32c156500 legacy=0x7fc32c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc314003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc314003fb0 0x7fc314004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc314004d90 0x7fc314005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c06b310 con 0x7fc314004d90 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31400bc50 con 0x7fc314003fb0 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31400be10 con 0x7fc32c14eaf0 2024-03-14T09:02:21.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc314004d90 0x7fc314005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32b7fe700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc314003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32affd700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc314003fb0 0x7fc314004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 -- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 msgr2=0x7fc314003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c14eaf0 0x7fc314003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 -- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc314003fb0 msgr2=0x7fc314004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc314003fb0 0x7fc314004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 -- 172.21.15.154:0/3053176306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140035b0 con 0x7fc314004d90 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.258+0000 7fc32bfff700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc314004d90 0x7fc314005210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a820 tx=0x7fc314005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997cde984402ca91 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32400a2f0 con 0x7fc314004d90 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc324012600 con 0x7fc314004d90 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc32a7fc700 1 -- 172.21.15.154:0/3053176306 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3240129b0 con 0x7fc314004d90 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 -- 172.21.15.154:0/3053176306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc314004d90 msgr2=0x7fc314005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 --2- 172.21.15.154:0/3053176306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc314004d90 0x7fc314005210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a820 tx=0x7fc314005880 comp rx=0 tx=0).stop 2024-03-14T09:02:21.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc32bfff700 1 -- 172.21.15.154:0/3053176306 reap_dead start 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 -- 172.21.15.154:0/3053176306 shutdown_connections 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 -- 172.21.15.154:0/3053176306 >> 172.21.15.154:0/3053176306 conn(0x7fc32c0c3880 msgr2=0x7fc32c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 -- 172.21.15.154:0/3053176306 shutdown_connections 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.259+0000 7fc332187700 1 -- 172.21.15.154:0/3053176306 wait complete. 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.260+0000 7fc332187700 1 Processor -- start 2024-03-14T09:02:21.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 -- start start 2024-03-14T09:02:21.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 0x7fc32c13b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 0x7fc32c143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1430c0 con 0x7fc32c1edd90 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c14e680 con 0x7fc32c1437f0 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc332187700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0c2f30 con 0x7fc32c13b540 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47748/0 (socket says 172.21.15.154:47748) 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.261+0000 7fc32b7fe700 1 -- 172.21.15.154:0/1045190969 learned_addr learned my addr 172.21.15.154:0/1045190969 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32bfff700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 0x7fc32c143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32affd700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 0x7fc32c13b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 -- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 msgr2=0x7fc32c13b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 0x7fc32c13b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 -- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 msgr2=0x7fc32c143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 0x7fc32c143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 -- 172.21.15.154:0/1045190969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0c3210 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32bfff700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 0x7fc32c143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32affd700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 0x7fc32c13b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc32b7fe700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc31c011aa0 tx=0x7fc31c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00f980 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00fae0 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c0c34f0 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00fe90 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.262+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c140660 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.263+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c143510 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.263+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc31c014440 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.264+0000 7fc313fff700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 0x7fc31806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.264+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc31c0898c0 con 0x7fc32c1edd90 2024-03-14T09:02:21.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.264+0000 7fc32affd700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 0x7fc31806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.265+0000 7fc32affd700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 0x7fc31806ccf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc32c1eb6e0 tx=0x7fc320009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.268+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc32c143510 con 0x7fc32c1edd90 2024-03-14T09:02:21.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.466+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc32c0697d0 con 0x7fc32c1edd90 2024-03-14T09:02:21.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.467+0000 7fc313fff700 1 -- 172.21.15.154:0/1045190969 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+19822 (secure 0 0 0) 0x7fc31c022090 con 0x7fc32c1edd90 2024-03-14T09:02:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:21.468 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:20.752977+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:21.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.469+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 msgr2=0x7fc31806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.469+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 0x7fc31806ccf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc32c1eb6e0 tx=0x7fc320009040 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.469+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 msgr2=0x7fc32c1ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.469+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc31c011aa0 tx=0x7fc31c011e60 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.469+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 shutdown_connections 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc32c1437f0 0x7fc32c143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31806a830 0x7fc31806ccf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32c1edd90 0x7fc32c1ee170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 --2- 172.21.15.154:0/1045190969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc32c13b540 0x7fc32c13b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 >> 172.21.15.154:0/1045190969 conn(0x7fc32c0c3880 msgr2=0x7fc32c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 shutdown_connections 2024-03-14T09:02:21.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.470+0000 7fc332187700 1 -- 172.21.15.154:0/1045190969 wait complete. 2024-03-14T09:02:21.482 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-14T09:02:21.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.606+0000 7f8bbb05c700 1 Processor -- start 2024-03-14T09:02:21.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.606+0000 7f8bbb05c700 1 -- start start 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bbb05c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 0x7f8bb41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bbb05c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb4157e80 con 0x7f8bb40c7f70 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bbb05c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb4158120 con 0x7f8bb4154d30 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bbb05c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4158ac0 con 0x7f8bb40c8350 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb8df8700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8bb40c7f70 0x7f8bb40c49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33706/0 (socket says 172.21.15.154:33706) 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb8df8700 1 -- 172.21.15.154:0/2660366660 learned_addr learned my addr 172.21.15.154:0/2660366660 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:21.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb3fff700 1 --2- 172.21.15.154:0/2660366660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 0x7f8bb41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014257719 0 0) 0x7f8bb4157e80 con 0x7f8bb40c7f70 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba40035b0 con 0x7f8bb40c7f70 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485139072 0 0) 0x7f8bb4158120 con 0x7f8bb4154d30 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb4157e80 con 0x7f8bb4154d30 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468028100 0 0) 0x7f8ba40035b0 con 0x7f8bb40c7f70 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2660366660 >> v1:172.21.15.191:6789/0 conn(0x7f8bb4154d30 legacy=0x7f8bb415f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2660366660 >> v1:172.21.15.154:6789/0 conn(0x7f8bb40c7f70 legacy=0x7f8bb40c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.607+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2660366660 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4158400 con 0x7f8bb40c8350 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bb3fff700 1 --2- 172.21.15.154:0/2660366660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 0x7f8bb41547f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009870 tx=0x7f8ba0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5299304443fc5ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8001670 con 0x7f8bb40c8350 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0004030 con 0x7f8bb40c8350 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bb37fe700 1 -- 172.21.15.154:0/2660366660 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00043a0 con 0x7f8bb40c8350 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2660366660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 msgr2=0x7f8bb41547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2660366660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 0x7f8bb41547f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009870 tx=0x7f8ba0009b80 comp rx=0 tx=0).stop 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2660366660 shutdown_connections 2024-03-14T09:02:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2660366660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c8350 0x7f8bb41547f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2660366660 >> 172.21.15.154:0/2660366660 conn(0x7f8bb4072080 msgr2=0x7f8bb4072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:21.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2660366660 shutdown_connections 2024-03-14T09:02:21.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.608+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2660366660 wait complete. 2024-03-14T09:02:21.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 Processor -- start 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 -- start start 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c7f70 0x7f8bb41eab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 0x7f8bb40af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4157910 con 0x7f8bb40c8350 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4157bb0 con 0x7f8bb4154d30 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.610+0000 7f8bbb05c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41ea090 con 0x7f8bb40c7f70 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47762/0 (socket says 172.21.15.154:47762) 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2569548496 learned_addr learned my addr 172.21.15.154:0/2569548496 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb8df8700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c7f70 0x7f8bb41eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb95f9700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 0x7f8bb40af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c7f70 msgr2=0x7f8bb41eab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c7f70 0x7f8bb41eab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 msgr2=0x7f8bb40af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 0x7f8bb40af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 -- 172.21.15.154:0/2569548496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00094a0 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb95f9700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 0x7f8bb40af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb3fff700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0000c00 tx=0x7f8ba0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0003d90 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba001f070 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0011870 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb41ea370 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.611+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb40af770 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.612+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb40697d0 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.613+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8ba00119d0 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.613+0000 7f8bb17fa700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 0x7f8b9806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.613+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8ba0021070 con 0x7f8bb40c8350 2024-03-14T09:02:21.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.613+0000 7f8bb8df8700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 0x7f8b9806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:21.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.614+0000 7f8bb8df8700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 0x7f8b9806ccf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c0cf0 tx=0x7f8ba8000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:21.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.617+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8ba0055830 con 0x7f8bb40c8350 2024-03-14T09:02:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:21.859+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8bb4153f70 con 0x7f8bb40c8350 2024-03-14T09:02:22.099 INFO:tasks.ceph.mds.a.smithi154.stderr:2024-03-14T09:02:22.099+0000 7fd7d8ae1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:02:22.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.760+0000 7f8bb17fa700 1 -- 172.21.15.154:0/2569548496 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 119+0+0 (secure 0 0 0) 0x7f8ba0058e50 con 0x7f8bb40c8350 2024-03-14T09:02:22.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 msgr2=0x7f8b9806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 0x7f8b9806ccf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c0cf0 tx=0x7f8ba8000bc0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 msgr2=0x7f8bb40ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0000c00 tx=0x7f8ba0004750 comp rx=0 tx=0).stop 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 shutdown_connections 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb4154d30 0x7f8bb40af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bb40c7f70 0x7f8bb41eab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b9806a830 0x7f8b9806ccf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 --2- 172.21.15.154:0/2569548496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb40c8350 0x7f8bb40ad6e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.763+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 >> 172.21.15.154:0/2569548496 conn(0x7f8bb4072080 msgr2=0x7f8bb40c48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.764+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 shutdown_connections 2024-03-14T09:02:22.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.764+0000 7f8bbb05c700 1 -- 172.21.15.154:0/2569548496 wait complete. 2024-03-14T09:02:22.778 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:02:22.778 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-14T09:02:22.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.908+0000 7f035ec75700 1 Processor -- start 2024-03-14T09:02:22.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.908+0000 7f035ec75700 1 -- start start 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ec75700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f035815e280 0x7f0358160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ec75700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03580ac8e0 con 0x7f035814eaf0 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ec75700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035806de30 con 0x7f0358155df0 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ec75700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035806b310 con 0x7f035815e280 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ca11700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f035814eaf0 0x7f035806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33716/0 (socket says 172.21.15.154:33716) 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.909+0000 7f035ca11700 1 -- 172.21.15.154:0/2589670126 learned_addr learned my addr 172.21.15.154:0/2589670126 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f035d212700 1 --2- 172.21.15.154:0/2589670126 >> v2:172.21.15.191:3300/0 conn(0x7f035815e280 0x7f0358160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382195751 0 0) 0x7f03580ac8e0 con 0x7f035814eaf0 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03480035b0 con 0x7f035814eaf0 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f035d212700 1 -- 172.21.15.154:0/2589670126 >> v1:172.21.15.191:6789/0 conn(0x7f0358155df0 legacy=0x7f0358156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f035d212700 1 -- 172.21.15.154:0/2589670126 >> v1:172.21.15.154:6789/0 conn(0x7f035814eaf0 legacy=0x7f035806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 996339162 0 0) 0x7f03480035b0 con 0x7f035814eaf0 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f035d212700 1 -- 172.21.15.154:0/2589670126 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358156960 con 0x7f035815e280 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f035d212700 1 --2- 172.21.15.154:0/2589670126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035815e280 0x7f0358160670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8361fe0f22aeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035000e070 con 0x7f035815e280 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0350003a20 con 0x7f035815e280 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.910+0000 7f03577fe700 1 -- 172.21.15.154:0/2589670126 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c001670 con 0x7f035815e280 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 -- 172.21.15.154:0/2589670126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035815e280 msgr2=0x7f0358160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 --2- 172.21.15.154:0/2589670126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035815e280 0x7f0358160670 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 -- 172.21.15.154:0/2589670126 shutdown_connections 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 --2- 172.21.15.154:0/2589670126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035815e280 0x7f0358160670 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 -- 172.21.15.154:0/2589670126 >> 172.21.15.154:0/2589670126 conn(0x7f03580c3880 msgr2=0x7f03580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 -- 172.21.15.154:0/2589670126 shutdown_connections 2024-03-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.911+0000 7f035ec75700 1 -- 172.21.15.154:0/2589670126 wait complete. 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.912+0000 7f035ec75700 1 Processor -- start 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.912+0000 7f035ec75700 1 -- start start 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0358155df0 0x7f035814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035815e280 0x7f0358150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0358156f30 con 0x7f035814eaf0 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c2f80 con 0x7f0358155df0 2024-03-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ec75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0358156c50 con 0x7f035815e280 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47764/0 (socket says 172.21.15.154:47764) 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 -- 172.21.15.154:0/1386941664 learned_addr learned my addr 172.21.15.154:0/1386941664 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f0357fff700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0358155df0 0x7f035814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035d212700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035815e280 0x7f0358150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 -- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035815e280 msgr2=0x7f0358150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035815e280 0x7f0358150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 -- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0358155df0 msgr2=0x7f035814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0358155df0 0x7f035814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 -- 172.21.15.154:0/1386941664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350009a70 con 0x7f035814eaf0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.913+0000 7f035ca11700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f034c0011e0 tx=0x7f034c00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.914+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c004c80 con 0x7f035814eaf0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.914+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c004de0 con 0x7f035814eaf0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.914+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03581ee2b0 con 0x7f035814eaf0 2024-03-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.914+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0072d0 con 0x7f035814eaf0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.914+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03581ee590 con 0x7f035814eaf0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.915+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03581ee0e0 con 0x7f035814eaf0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.915+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f034c007430 con 0x7f035814eaf0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.915+0000 7f03557fa700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 0x7f033c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.915+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f034c088ef0 con 0x7f035814eaf0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.916+0000 7f0357fff700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 0x7f033c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.916+0000 7f0357fff700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 0x7f033c06ccf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0358143da0 tx=0x7f0344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:22.920+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f034c053fc0 con 0x7f035814eaf0 2024-03-14T09:02:23.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.156+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f03580697d0 con 0x7f035814eaf0 2024-03-14T09:02:23.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.763+0000 7f03557fa700 1 -- 172.21.15.154:0/1386941664 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7f034c0575e0 con 0x7f035814eaf0 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 msgr2=0x7f033c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 0x7f033c06ccf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0358143da0 tx=0x7f0344009040 comp rx=0 tx=0).stop 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 msgr2=0x7f0358143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f034c0011e0 tx=0x7f034c00ca50 comp rx=0 tx=0).stop 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 shutdown_connections 2024-03-14T09:02:23.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035815e280 0x7f0358150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f033c06a830 0x7f033c06ccf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f035814eaf0 0x7f0358143530 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 --2- 172.21.15.154:0/1386941664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0358155df0 0x7f035814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 >> 172.21.15.154:0/1386941664 conn(0x7f03580c3880 msgr2=0x7f0358156250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 shutdown_connections 2024-03-14T09:02:23.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.765+0000 7f035ec75700 1 -- 172.21.15.154:0/1386941664 wait complete. 2024-03-14T09:02:23.779 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-14T09:02:23.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:23.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.905+0000 7fa604eba700 1 Processor -- start 2024-03-14T09:02:23.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.905+0000 7fa604eba700 1 -- start start 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa604eba700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa600158760 0x7fa600154c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa604eba700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 0x7fa60015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa604eba700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6000c0f90 con 0x7fa600152020 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa604eba700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6000c05e0 con 0x7fa600158760 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa604eba700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000666a0 con 0x7fa600155150 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa5fe59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa600152020 0x7fa6000c5b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33732/0 (socket says 172.21.15.154:33732) 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa5fe59c700 1 -- 172.21.15.154:0/2382577143 learned_addr learned my addr 172.21.15.154:0/2382577143 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa5fed9d700 1 --2- 172.21.15.154:0/2382577143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 0x7fa60015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa5fdd9b700 1 --2- 172.21.15.154:0/2382577143 >> v2:172.21.15.191:3300/0 conn(0x7fa600158760 0x7fa600154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.906+0000 7fa5fd59a700 1 -- 172.21.15.154:0/2382577143 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662667225 0 0) 0x7fa6000c0f90 con 0x7fa600152020 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fd59a700 1 -- 172.21.15.154:0/2382577143 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5e80035b0 con 0x7fa600152020 2024-03-14T09:02:23.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fed9d700 1 -- 172.21.15.154:0/2382577143 >> v2:172.21.15.191:3300/0 conn(0x7fa600158760 msgr2=0x7fa600154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fed9d700 1 --2- 172.21.15.154:0/2382577143 >> v2:172.21.15.191:3300/0 conn(0x7fa600158760 0x7fa600154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fed9d700 1 -- 172.21.15.154:0/2382577143 >> v1:172.21.15.154:6789/0 conn(0x7fa600152020 legacy=0x7fa6000c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fed9d700 1 -- 172.21.15.154:0/2382577143 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000bfa60 con 0x7fa600155150 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fed9d700 1 --2- 172.21.15.154:0/2382577143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 0x7fa60015f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7892192c1d1e8ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fd59a700 1 -- 172.21.15.154:0/2382577143 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800e070 con 0x7fa600155150 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fd59a700 1 -- 172.21.15.154:0/2382577143 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f800bb20 con 0x7fa600155150 2024-03-14T09:02:23.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.907+0000 7fa5fd59a700 1 -- 172.21.15.154:0/2382577143 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8012430 con 0x7fa600155150 2024-03-14T09:02:23.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 -- 172.21.15.154:0/2382577143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 msgr2=0x7fa60015f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 --2- 172.21.15.154:0/2382577143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 0x7fa60015f0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009d90 tx=0x7fa5f800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 -- 172.21.15.154:0/2382577143 shutdown_connections 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 --2- 172.21.15.154:0/2382577143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600155150 0x7fa60015f0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 --2- 172.21.15.154:0/2382577143 >> v2:172.21.15.191:3300/0 conn(0x7fa600158760 0x7fa600154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 -- 172.21.15.154:0/2382577143 >> 172.21.15.154:0/2382577143 conn(0x7fa60006a770 msgr2=0x7fa60006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 -- 172.21.15.154:0/2382577143 shutdown_connections 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.908+0000 7fa604eba700 1 -- 172.21.15.154:0/2382577143 wait complete. 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.909+0000 7fa604eba700 1 Processor -- start 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.909+0000 7fa604eba700 1 -- start start 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa600152020 0x7fa600154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600158760 0x7fa6000a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600152d00 con 0x7fa600155150 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600150480 con 0x7fa600152020 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa600154800 con 0x7fa600158760 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47768/0 (socket says 172.21.15.154:47768) 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 -- 172.21.15.154:0/552772649 learned_addr learned my addr 172.21.15.154:0/552772649 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fe59c700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa600152020 0x7fa600154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 -- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600158760 msgr2=0x7fa6000a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fed9d700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600158760 0x7fa6000a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600158760 0x7fa6000a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 -- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa600152020 msgr2=0x7fa600154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa600152020 0x7fa600154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 -- 172.21.15.154:0/552772649 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009a70 con 0x7fa600155150 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5fdd9b700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000e8a0 tx=0x7fa5f000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009590 con 0x7fa600155150 2024-03-14T09:02:23.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000eea0 con 0x7fa600155150 2024-03-14T09:02:23.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa600152740 con 0x7fa600155150 2024-03-14T09:02:23.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.910+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000ca50 con 0x7fa600155150 2024-03-14T09:02:23.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.911+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6000a7f50 con 0x7fa600155150 2024-03-14T09:02:23.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.911+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6000697d0 con 0x7fa600155150 2024-03-14T09:02:23.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.912+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa5f000cbb0 con 0x7fa600155150 2024-03-14T09:02:23.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.912+0000 7fa5eeffd700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 0x7fa5e006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:23.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.912+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa5f0089420 con 0x7fa600155150 2024-03-14T09:02:23.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.913+0000 7fa5fe59c700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 0x7fa5e006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:23.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.913+0000 7fa5fe59c700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 0x7fa5e006ccf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40011e0 tx=0x7fa5f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:23.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:23.917+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa5f00544e0 con 0x7fa600155150 2024-03-14T09:02:24.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.155+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa600150810 con 0x7fa600155150 2024-03-14T09:02:24.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.155+0000 7fa5eeffd700 1 -- 172.21.15.154:0/552772649 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5951 (secure 0 0 0) 0x7fa5f0057b00 con 0x7fa600155150 2024-03-14T09:02:24.155 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:24.155 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.157+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 msgr2=0x7fa5e006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.157+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 0x7fa5e006ccf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40011e0 tx=0x7fa5f4009040 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 msgr2=0x7fa600070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000e8a0 tx=0x7fa5f000ec60 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 shutdown_connections 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa600158760 0x7fa6000a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa600152020 0x7fa600154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa5e006a830 0x7fa5e006ccf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 --2- 172.21.15.154:0/552772649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa600155150 0x7fa600070f70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 >> 172.21.15.154:0/552772649 conn(0x7fa60006a770 msgr2=0x7fa60014c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 shutdown_connections 2024-03-14T09:02:24.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.158+0000 7fa604eba700 1 -- 172.21.15.154:0/552772649 wait complete. 2024-03-14T09:02:24.159 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 20 2024-03-14T09:02:24.170 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.291+0000 7f2e2feb7700 1 Processor -- start 2024-03-14T09:02:24.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.291+0000 7f2e2feb7700 1 -- start start 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2feb7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e2813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2feb7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e280c1780 0x7f2e280c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2feb7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2813ade0 con 0x7f2e280a8540 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2feb7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2806fb30 con 0x7f2e280c1780 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2feb7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2806ec30 con 0x7f2e280c3250 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e280c1780 0x7f2e280c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2dc53700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2e280a8540 0x7f2e28154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33742/0 (socket says 172.21.15.154:33742) 2024-03-14T09:02:24.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2dc53700 1 -- 172.21.15.154:0/4202976976 learned_addr learned my addr 172.21.15.154:0/4202976976 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2d452700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e2813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 -- 172.21.15.154:0/4202976976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 msgr2=0x7f2e2813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e2813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 -- 172.21.15.154:0/4202976976 >> v1:172.21.15.154:6789/0 conn(0x7f2e280a8540 legacy=0x7f2e28154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 -- 172.21.15.154:0/4202976976 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2806ef10 con 0x7f2e280c1780 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2cc51700 1 -- 172.21.15.154:0/4202976976 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597564770 0 0) 0x7f2e18001110 con 0x7f2e280a8540 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2e454700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e280c1c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000bfa0 tx=0x7f2e2000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d7c83d39dad99b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2cc51700 1 -- 172.21.15.154:0/4202976976 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20010070 con 0x7f2e280c1780 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2cc51700 1 -- 172.21.15.154:0/4202976976 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000b2e0 con 0x7f2e280c1780 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.292+0000 7f2e2cc51700 1 -- 172.21.15.154:0/4202976976 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000dcf0 con 0x7f2e280c1780 2024-03-14T09:02:24.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 -- 172.21.15.154:0/4202976976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 msgr2=0x7f2e280c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e280c1c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000bfa0 tx=0x7f2e2000d330 comp rx=0 tx=0).stop 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 -- 172.21.15.154:0/4202976976 shutdown_connections 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e280c1c10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/4202976976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e2813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 -- 172.21.15.154:0/4202976976 >> 172.21.15.154:0/4202976976 conn(0x7f2e280c3880 msgr2=0x7f2e280c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 -- 172.21.15.154:0/4202976976 shutdown_connections 2024-03-14T09:02:24.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.293+0000 7f2e2feb7700 1 -- 172.21.15.154:0/4202976976 wait complete. 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.294+0000 7f2e2feb7700 1 Processor -- start 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.294+0000 7f2e2feb7700 1 -- start start 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e280a8540 0x7f2e281dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e281db510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e281db080 con 0x7f2e280c1780 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28151f20 con 0x7f2e280a8540 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2feb7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280aba50 con 0x7f2e280c3250 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47782/0 (socket says 172.21.15.154:47782) 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 -- 172.21.15.154:0/1802918149 learned_addr learned my addr 172.21.15.154:0/1802918149 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2dc53700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e280a8540 0x7f2e281dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2e454700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e281db510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 -- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 msgr2=0x7f2e281db510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e281db510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 -- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e280a8540 msgr2=0x7f2e281dbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e280a8540 0x7f2e281dbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 -- 172.21.15.154:0/1802918149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2000bc80 con 0x7f2e280c1780 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2e454700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e281db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.295+0000 7f2e2d452700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e840 tx=0x7f2e2400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24013610 con 0x7f2e280c1780 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24013770 con 0x7f2e280c1780 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e281dc280 con 0x7f2e280c1780 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24013b20 con 0x7f2e280c1780 2024-03-14T09:02:24.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e28151960 con 0x7f2e280c1780 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.296+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28066980 con 0x7f2e280c1780 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.297+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2e24013c80 con 0x7f2e280c1780 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.297+0000 7f2e1e7fc700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 0x7f2e0c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.297+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2e24088d10 con 0x7f2e280c1780 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.298+0000 7f2e2dc53700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 0x7f2e0c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.298+0000 7f2e2dc53700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 0x7f2e0c06ccf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000f60 tx=0x7f2e18007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.302+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2e24053de0 con 0x7f2e280c1780 2024-03-14T09:02:24.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.499+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e280697d0 con 0x7f2e280c1780 2024-03-14T09:02:24.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.501+0000 7f2e1e7fc700 1 -- 172.21.15.154:0/1802918149 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7f2e24057400 con 0x7f2e280c1780 2024-03-14T09:02:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:24.501 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 msgr2=0x7f2e0c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 0x7f2e0c06ccf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000f60 tx=0x7f2e18007e70 comp rx=0 tx=0).stop 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 msgr2=0x7f2e28153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e840 tx=0x7f2e2400eb50 comp rx=0 tx=0).stop 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 shutdown_connections 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e280c3250 0x7f2e281db510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e280a8540 0x7f2e281dbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e0c06a830 0x7f2e0c06ccf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 --2- 172.21.15.154:0/1802918149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e280c1780 0x7f2e28153d60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 >> 172.21.15.154:0/1802918149 conn(0x7f2e280c3880 msgr2=0x7f2e280c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:24.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 shutdown_connections 2024-03-14T09:02:24.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.503+0000 7f2e2feb7700 1 -- 172.21.15.154:0/1802918149 wait complete. 2024-03-14T09:02:24.515 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:02:24.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:24.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.637+0000 7f306f3a4700 1 Processor -- start 2024-03-14T09:02:24.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.637+0000 7f306f3a4700 1 -- start start 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306f3a4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f306806c6e0 0x7f306806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306f3a4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680c5f10 con 0x7f306806d600 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306f3a4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680c7d50 con 0x7f3068070a10 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306f3a4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30680c8030 con 0x7f306806c6e0 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d140700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f306806c6e0 0x7f306806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d941700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f306806d600 0x7f30680c6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33744/0 (socket says 172.21.15.154:33744) 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d941700 1 -- 172.21.15.154:0/889878585 learned_addr learned my addr 172.21.15.154:0/889878585 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d140700 1 -- 172.21.15.154:0/889878585 >> v1:172.21.15.191:6789/0 conn(0x7f3068070a10 legacy=0x7f30680c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d140700 1 -- 172.21.15.154:0/889878585 >> v1:172.21.15.154:6789/0 conn(0x7f306806d600 legacy=0x7f30680c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d140700 1 -- 172.21.15.154:0/889878585 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680c8310 con 0x7f306806c6e0 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.638+0000 7f306d140700 1 --2- 172.21.15.154:0/889878585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306806c6e0 0x7f306806d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3058009840 tx=0x7f3058009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0bcb4f9074dd09 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f305ffff700 1 -- 172.21.15.154:0/889878585 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058011830 con 0x7f306806c6e0 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f305ffff700 1 -- 172.21.15.154:0/889878585 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058011990 con 0x7f306806c6e0 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f305ffff700 1 -- 172.21.15.154:0/889878585 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058011d20 con 0x7f306806c6e0 2024-03-14T09:02:24.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 -- 172.21.15.154:0/889878585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306806c6e0 msgr2=0x7f306806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 --2- 172.21.15.154:0/889878585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306806c6e0 0x7f306806d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3058009840 tx=0x7f3058009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 -- 172.21.15.154:0/889878585 shutdown_connections 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 --2- 172.21.15.154:0/889878585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f306806c6e0 0x7f306806d0c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 -- 172.21.15.154:0/889878585 >> 172.21.15.154:0/889878585 conn(0x7f30680a9bf0 msgr2=0x7f30680a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 -- 172.21.15.154:0/889878585 shutdown_connections 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.639+0000 7f306f3a4700 1 -- 172.21.15.154:0/889878585 wait complete. 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.640+0000 7f306f3a4700 1 Processor -- start 2024-03-14T09:02:24.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.640+0000 7f306f3a4700 1 -- start start 2024-03-14T09:02:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f306806c6e0 0x7f30681dd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f306806d600 0x7f30681d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30681dacf0 con 0x7f3068070a10 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30681ee0b0 con 0x7f306806c6e0 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306f3a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30681ddfe0 con 0x7f306806d600 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47804/0 (socket says 172.21.15.154:47804) 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 -- 172.21.15.154:0/890616755 learned_addr learned my addr 172.21.15.154:0/890616755 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306c93f700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f306806d600 0x7f30681d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d140700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f306806c6e0 0x7f30681dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 -- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f306806d600 msgr2=0x7f30681d7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f306806d600 0x7f30681d7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 -- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f306806c6e0 msgr2=0x7f30681dd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.641+0000 7f306d941700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f306806c6e0 0x7f30681dd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f306d941700 1 -- 172.21.15.154:0/890616755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580094a0 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f306d941700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f306000cbc0 tx=0x7f306000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000a9e0 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000ab40 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680c6850 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060012400 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.642+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30681eda60 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.643+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30681ee220 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.643+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f306001b3b0 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.644+0000 7f305dffb700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 0x7f304c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.644+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f3060088b20 con 0x7f3068070a10 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.644+0000 7f306d140700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 0x7f304c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:24.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.644+0000 7f306d140700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 0x7f304c06ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f30580090b0 tx=0x7f3058008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.648+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3060053d20 con 0x7f3068070a10 2024-03-14T09:02:24.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.836+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30680697d0 con 0x7f304c06a830 2024-03-14T09:02:24.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.851+0000 7f305dffb700 1 -- 172.21.15.154:0/890616755 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527878 (secure 0 0 0) 0x7f30680697d0 con 0x7f304c06a830 2024-03-14T09:02:24.861 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 msgr2=0x7f304c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 0x7f304c06ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f30580090b0 tx=0x7f3058008030 comp rx=0 tx=0).stop 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 msgr2=0x7f30681ea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f306000cbc0 tx=0x7f306000ced0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 shutdown_connections 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f304c06a830 0x7f304c06ccf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3068070a10 0x7f30681ea3c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f306806c6e0 0x7f30681dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 --2- 172.21.15.154:0/890616755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f306806d600 0x7f30681d7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 >> 172.21.15.154:0/890616755 conn(0x7f30680a9bf0 msgr2=0x7f30680a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 shutdown_connections 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:24.855+0000 7f306f3a4700 1 -- 172.21.15.154:0/890616755 wait complete. 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:24.868 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-14T09:02:23.532378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":310,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220716,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4595712,"data_stored":2537110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.194186"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767107+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:02:20.767107+0000","last_peered":"2024-03-14T09:02:20.767107+0000","last_clean":"2024-03-14T09:02:20.767107+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:02:20.767107+0000","last_undegraded":"2024-03-14T09:02:20.767107+0000","last_fullsized":"2024-03-14T09:02:20.767107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763302+0000","last_change":"2024-03-14T09:02:03.333342+0000","last_active":"2024-03-14T09:02:20.763302+0000","last_peered":"2024-03-14T09:02:20.763302+0000","last_clean":"2024-03-14T09:02:20.763302+0000","last_became_active":"2024-03-14T09:02:03.333078+0000","last_became_peered":"2024-03-14T09:02:03.333078+0000","last_unstale":"2024-03-14T09:02:20.763302+0000","last_undegraded":"2024-03-14T09:02:20.763302+0000","last_fullsized":"2024-03-14T09:02:20.763302+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767560+0000","last_change":"2024-03-14T09:02:03.758083+0000","last_active":"2024-03-14T09:02:20.767560+0000","last_peered":"2024-03-14T09:02:20.767560+0000","last_clean":"2024-03-14T09:02:20.767560+0000","last_became_active":"2024-03-14T09:02:03.757724+0000","last_became_peered":"2024-03-14T09:02:03.757724+0000","last_unstale":"2024-03-14T09:02:20.767560+0000","last_undegraded":"2024-03-14T09:02:20.767560+0000","last_fullsized":"2024-03-14T09:02:20.767560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939200+0000","last_change":"2024-03-14T09:02:20.357631+0000","last_active":"2024-03-14T09:02:21.939200+0000","last_peered":"2024-03-14T09:02:21.939200+0000","last_clean":"2024-03-14T09:02:21.939200+0000","last_became_active":"2024-03-14T09:02:20.356947+0000","last_became_peered":"2024-03-14T09:02:20.356947+0000","last_unstale":"2024-03-14T09:02:21.939200+0000","last_undegraded":"2024-03-14T09:02:21.939200+0000","last_fullsized":"2024-03-14T09:02:21.939200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:19:06.899281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937444+0000","last_change":"2024-03-14T09:02:20.780469+0000","last_active":"2024-03-14T09:02:21.937444+0000","last_peered":"2024-03-14T09:02:21.937444+0000","last_clean":"2024-03-14T09:02:21.937444+0000","last_became_active":"2024-03-14T09:02:20.780078+0000","last_became_peered":"2024-03-14T09:02:20.780078+0000","last_unstale":"2024-03-14T09:02:21.937444+0000","last_undegraded":"2024-03-14T09:02:21.937444+0000","last_fullsized":"2024-03-14T09:02:21.937444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:35.201864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763756+0000","last_change":"2024-03-14T09:02:03.335789+0000","last_active":"2024-03-14T09:02:20.763756+0000","last_peered":"2024-03-14T09:02:20.763756+0000","last_clean":"2024-03-14T09:02:20.763756+0000","last_became_active":"2024-03-14T09:02:03.335514+0000","last_became_peered":"2024-03-14T09:02:03.335514+0000","last_unstale":"2024-03-14T09:02:20.763756+0000","last_undegraded":"2024-03-14T09:02:20.763756+0000","last_fullsized":"2024-03-14T09:02:20.763756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764073+0000","last_change":"2024-03-14T09:02:03.749871+0000","last_active":"2024-03-14T09:02:20.764073+0000","last_peered":"2024-03-14T09:02:20.764073+0000","last_clean":"2024-03-14T09:02:20.764073+0000","last_became_active":"2024-03-14T09:02:03.748885+0000","last_became_peered":"2024-03-14T09:02:03.748885+0000","last_unstale":"2024-03-14T09:02:20.764073+0000","last_undegraded":"2024-03-14T09:02:20.764073+0000","last_fullsized":"2024-03-14T09:02:20.764073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939256+0000","last_change":"2024-03-14T09:02:20.357221+0000","last_active":"2024-03-14T09:02:21.939256+0000","last_peered":"2024-03-14T09:02:21.939256+0000","last_clean":"2024-03-14T09:02:21.939256+0000","last_became_active":"2024-03-14T09:02:20.356857+0000","last_became_peered":"2024-03-14T09:02:20.356857+0000","last_unstale":"2024-03-14T09:02:21.939256+0000","last_undegraded":"2024-03-14T09:02:21.939256+0000","last_fullsized":"2024-03-14T09:02:21.939256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:40.867971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764405+0000","last_change":"2024-03-14T09:02:20.784094+0000","last_active":"2024-03-14T09:02:21.764405+0000","last_peered":"2024-03-14T09:02:21.764405+0000","last_clean":"2024-03-14T09:02:21.764405+0000","last_became_active":"2024-03-14T09:02:20.783828+0000","last_became_peered":"2024-03-14T09:02:20.783828+0000","last_unstale":"2024-03-14T09:02:21.764405+0000","last_undegraded":"2024-03-14T09:02:21.764405+0000","last_fullsized":"2024-03-14T09:02:21.764405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.686979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764341+0000","last_change":"2024-03-14T09:02:03.332151+0000","last_active":"2024-03-14T09:02:20.764341+0000","last_peered":"2024-03-14T09:02:20.764341+0000","last_clean":"2024-03-14T09:02:20.764341+0000","last_became_active":"2024-03-14T09:02:03.331768+0000","last_became_peered":"2024-03-14T09:02:03.331768+0000","last_unstale":"2024-03-14T09:02:20.764341+0000","last_undegraded":"2024-03-14T09:02:20.764341+0000","last_fullsized":"2024-03-14T09:02:20.764341+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759955+0000","last_change":"2024-03-14T09:02:03.754008+0000","last_active":"2024-03-14T09:02:20.759955+0000","last_peered":"2024-03-14T09:02:20.759955+0000","last_clean":"2024-03-14T09:02:20.759955+0000","last_became_active":"2024-03-14T09:02:03.753707+0000","last_became_peered":"2024-03-14T09:02:03.753707+0000","last_unstale":"2024-03-14T09:02:20.759955+0000","last_undegraded":"2024-03-14T09:02:20.759955+0000","last_fullsized":"2024-03-14T09:02:20.759955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:23.291407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937022+0000","last_change":"2024-03-14T09:02:20.362219+0000","last_active":"2024-03-14T09:02:21.937022+0000","last_peered":"2024-03-14T09:02:21.937022+0000","last_clean":"2024-03-14T09:02:21.937022+0000","last_became_active":"2024-03-14T09:02:20.361957+0000","last_became_peered":"2024-03-14T09:02:20.361957+0000","last_unstale":"2024-03-14T09:02:21.937022+0000","last_undegraded":"2024-03-14T09:02:21.937022+0000","last_fullsized":"2024-03-14T09:02:21.937022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:08.180878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764262+0000","last_change":"2024-03-14T09:02:20.778325+0000","last_active":"2024-03-14T09:02:21.764262+0000","last_peered":"2024-03-14T09:02:21.764262+0000","last_clean":"2024-03-14T09:02:21.764262+0000","last_became_active":"2024-03-14T09:02:20.777176+0000","last_became_peered":"2024-03-14T09:02:20.777176+0000","last_unstale":"2024-03-14T09:02:21.764262+0000","last_undegraded":"2024-03-14T09:02:21.764262+0000","last_fullsized":"2024-03-14T09:02:21.764262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:36.109417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763578+0000","last_change":"2024-03-14T09:02:03.323982+0000","last_active":"2024-03-14T09:02:20.763578+0000","last_peered":"2024-03-14T09:02:20.763578+0000","last_clean":"2024-03-14T09:02:20.763578+0000","last_became_active":"2024-03-14T09:02:03.323752+0000","last_became_peered":"2024-03-14T09:02:03.323752+0000","last_unstale":"2024-03-14T09:02:20.763578+0000","last_undegraded":"2024-03-14T09:02:20.763578+0000","last_fullsized":"2024-03-14T09:02:20.763578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763454+0000","last_change":"2024-03-14T09:02:03.750281+0000","last_active":"2024-03-14T09:02:20.763454+0000","last_peered":"2024-03-14T09:02:20.763454+0000","last_clean":"2024-03-14T09:02:20.763454+0000","last_became_active":"2024-03-14T09:02:03.750044+0000","last_became_peered":"2024-03-14T09:02:03.750044+0000","last_unstale":"2024-03-14T09:02:20.763454+0000","last_undegraded":"2024-03-14T09:02:20.763454+0000","last_fullsized":"2024-03-14T09:02:20.763454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936203+0000","last_change":"2024-03-14T09:02:20.358027+0000","last_active":"2024-03-14T09:02:21.936203+0000","last_peered":"2024-03-14T09:02:21.936203+0000","last_clean":"2024-03-14T09:02:21.936203+0000","last_became_active":"2024-03-14T09:02:20.357394+0000","last_became_peered":"2024-03-14T09:02:20.357394+0000","last_unstale":"2024-03-14T09:02:21.936203+0000","last_undegraded":"2024-03-14T09:02:21.936203+0000","last_fullsized":"2024-03-14T09:02:21.936203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:14:26.739393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937296+0000","last_change":"2024-03-14T09:02:20.779583+0000","last_active":"2024-03-14T09:02:21.937296+0000","last_peered":"2024-03-14T09:02:21.937296+0000","last_clean":"2024-03-14T09:02:21.937296+0000","last_became_active":"2024-03-14T09:02:20.779312+0000","last_became_peered":"2024-03-14T09:02:20.779312+0000","last_unstale":"2024-03-14T09:02:21.937296+0000","last_undegraded":"2024-03-14T09:02:21.937296+0000","last_fullsized":"2024-03-14T09:02:21.937296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:01.997032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763343+0000","last_change":"2024-03-14T09:02:03.332108+0000","last_active":"2024-03-14T09:02:20.763343+0000","last_peered":"2024-03-14T09:02:20.763343+0000","last_clean":"2024-03-14T09:02:20.763343+0000","last_became_active":"2024-03-14T09:02:03.331581+0000","last_became_peered":"2024-03-14T09:02:03.331581+0000","last_unstale":"2024-03-14T09:02:20.763343+0000","last_undegraded":"2024-03-14T09:02:20.763343+0000","last_fullsized":"2024-03-14T09:02:20.763343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.760070+0000","last_change":"2024-03-14T09:02:03.750999+0000","last_active":"2024-03-14T09:02:20.760070+0000","last_peered":"2024-03-14T09:02:20.760070+0000","last_clean":"2024-03-14T09:02:20.760070+0000","last_became_active":"2024-03-14T09:02:03.750615+0000","last_became_peered":"2024-03-14T09:02:03.750615+0000","last_unstale":"2024-03-14T09:02:20.760070+0000","last_undegraded":"2024-03-14T09:02:20.760070+0000","last_fullsized":"2024-03-14T09:02:20.760070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936526+0000","last_change":"2024-03-14T09:02:20.361477+0000","last_active":"2024-03-14T09:02:21.936526+0000","last_peered":"2024-03-14T09:02:21.936526+0000","last_clean":"2024-03-14T09:02:21.936526+0000","last_became_active":"2024-03-14T09:02:20.361258+0000","last_became_peered":"2024-03-14T09:02:20.361258+0000","last_unstale":"2024-03-14T09:02:21.936526+0000","last_undegraded":"2024-03-14T09:02:21.936526+0000","last_fullsized":"2024-03-14T09:02:21.936526+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:35.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936391+0000","last_change":"2024-03-14T09:02:20.774132+0000","last_active":"2024-03-14T09:02:21.936391+0000","last_peered":"2024-03-14T09:02:21.936391+0000","last_clean":"2024-03-14T09:02:21.936391+0000","last_became_active":"2024-03-14T09:02:20.773888+0000","last_became_peered":"2024-03-14T09:02:20.773888+0000","last_unstale":"2024-03-14T09:02:21.936391+0000","last_undegraded":"2024-03-14T09:02:21.936391+0000","last_fullsized":"2024-03-14T09:02:21.936391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:52.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764208+0000","last_change":"2024-03-14T09:02:03.329704+0000","last_active":"2024-03-14T09:02:20.764208+0000","last_peered":"2024-03-14T09:02:20.764208+0000","last_clean":"2024-03-14T09:02:20.764208+0000","last_became_active":"2024-03-14T09:02:03.328905+0000","last_became_peered":"2024-03-14T09:02:03.328905+0000","last_unstale":"2024-03-14T09:02:20.764208+0000","last_undegraded":"2024-03-14T09:02:20.764208+0000","last_fullsized":"2024-03-14T09:02:20.764208+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761908+0000","last_change":"2024-03-14T09:02:03.745980+0000","last_active":"2024-03-14T09:02:20.761908+0000","last_peered":"2024-03-14T09:02:20.761908+0000","last_clean":"2024-03-14T09:02:20.761908+0000","last_became_active":"2024-03-14T09:02:03.745710+0000","last_became_peered":"2024-03-14T09:02:03.745710+0000","last_unstale":"2024-03-14T09:02:20.761908+0000","last_undegraded":"2024-03-14T09:02:20.761908+0000","last_fullsized":"2024-03-14T09:02:20.761908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764918+0000","last_change":"2024-03-14T09:02:20.357859+0000","last_active":"2024-03-14T09:02:21.764918+0000","last_peered":"2024-03-14T09:02:21.764918+0000","last_clean":"2024-03-14T09:02:21.764918+0000","last_became_active":"2024-03-14T09:02:20.357581+0000","last_became_peered":"2024-03-14T09:02:20.357581+0000","last_unstale":"2024-03-14T09:02:21.764918+0000","last_undegraded":"2024-03-14T09:02:21.764918+0000","last_fullsized":"2024-03-14T09:02:21.764918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:22.827855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763702+0000","last_change":"2024-03-14T09:02:20.778300+0000","last_active":"2024-03-14T09:02:21.763702+0000","last_peered":"2024-03-14T09:02:21.763702+0000","last_clean":"2024-03-14T09:02:21.763702+0000","last_became_active":"2024-03-14T09:02:20.778021+0000","last_became_peered":"2024-03-14T09:02:20.778021+0000","last_unstale":"2024-03-14T09:02:21.763702+0000","last_undegraded":"2024-03-14T09:02:21.763702+0000","last_fullsized":"2024-03-14T09:02:21.763702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:12:16.436058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761145+0000","last_change":"2024-03-14T09:02:03.331634+0000","last_active":"2024-03-14T09:02:20.761145+0000","last_peered":"2024-03-14T09:02:20.761145+0000","last_clean":"2024-03-14T09:02:20.761145+0000","last_became_active":"2024-03-14T09:02:03.331213+0000","last_became_peered":"2024-03-14T09:02:03.331213+0000","last_unstale":"2024-03-14T09:02:20.761145+0000","last_undegraded":"2024-03-14T09:02:20.761145+0000","last_fullsized":"2024-03-14T09:02:20.761145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768513+0000","last_change":"2024-03-14T09:02:03.750602+0000","last_active":"2024-03-14T09:02:20.768513+0000","last_peered":"2024-03-14T09:02:20.768513+0000","last_clean":"2024-03-14T09:02:20.768513+0000","last_became_active":"2024-03-14T09:02:03.750193+0000","last_became_peered":"2024-03-14T09:02:03.750193+0000","last_unstale":"2024-03-14T09:02:20.768513+0000","last_undegraded":"2024-03-14T09:02:20.768513+0000","last_fullsized":"2024-03-14T09:02:20.768513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936487+0000","last_change":"2024-03-14T09:02:20.360009+0000","last_active":"2024-03-14T09:02:21.936487+0000","last_peered":"2024-03-14T09:02:21.936487+0000","last_clean":"2024-03-14T09:02:21.936487+0000","last_became_active":"2024-03-14T09:02:20.359817+0000","last_became_peered":"2024-03-14T09:02:20.359817+0000","last_unstale":"2024-03-14T09:02:21.936487+0000","last_undegraded":"2024-03-14T09:02:21.936487+0000","last_fullsized":"2024-03-14T09:02:21.936487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:47.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937746+0000","last_change":"2024-03-14T09:02:20.777827+0000","last_active":"2024-03-14T09:02:21.937746+0000","last_peered":"2024-03-14T09:02:21.937746+0000","last_clean":"2024-03-14T09:02:21.937746+0000","last_became_active":"2024-03-14T09:02:20.777510+0000","last_became_peered":"2024-03-14T09:02:20.777510+0000","last_unstale":"2024-03-14T09:02:21.937746+0000","last_undegraded":"2024-03-14T09:02:21.937746+0000","last_fullsized":"2024-03-14T09:02:21.937746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:01:55.985257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765374+0000","last_change":"2024-03-14T09:02:03.324982+0000","last_active":"2024-03-14T09:02:20.765374+0000","last_peered":"2024-03-14T09:02:20.765374+0000","last_clean":"2024-03-14T09:02:20.765374+0000","last_became_active":"2024-03-14T09:02:03.324739+0000","last_became_peered":"2024-03-14T09:02:03.324739+0000","last_unstale":"2024-03-14T09:02:20.765374+0000","last_undegraded":"2024-03-14T09:02:20.765374+0000","last_fullsized":"2024-03-14T09:02:20.765374+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767968+0000","last_change":"2024-03-14T09:02:03.751539+0000","last_active":"2024-03-14T09:02:20.767968+0000","last_peered":"2024-03-14T09:02:20.767968+0000","last_clean":"2024-03-14T09:02:20.767968+0000","last_became_active":"2024-03-14T09:02:03.751299+0000","last_became_peered":"2024-03-14T09:02:03.751299+0000","last_unstale":"2024-03-14T09:02:20.767968+0000","last_undegraded":"2024-03-14T09:02:20.767968+0000","last_fullsized":"2024-03-14T09:02:20.767968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:38.365353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937512+0000","last_change":"2024-03-14T09:02:20.359861+0000","last_active":"2024-03-14T09:02:21.937512+0000","last_peered":"2024-03-14T09:02:21.937512+0000","last_clean":"2024-03-14T09:02:21.937512+0000","last_became_active":"2024-03-14T09:02:20.359576+0000","last_became_peered":"2024-03-14T09:02:20.359576+0000","last_unstale":"2024-03-14T09:02:21.937512+0000","last_undegraded":"2024-03-14T09:02:21.937512+0000","last_fullsized":"2024-03-14T09:02:21.937512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:02.654716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937317+0000","last_change":"2024-03-14T09:02:20.771966+0000","last_active":"2024-03-14T09:02:21.937317+0000","last_peered":"2024-03-14T09:02:21.937317+0000","last_clean":"2024-03-14T09:02:21.937317+0000","last_became_active":"2024-03-14T09:02:20.771325+0000","last_became_peered":"2024-03-14T09:02:20.771325+0000","last_unstale":"2024-03-14T09:02:21.937317+0000","last_undegraded":"2024-03-14T09:02:21.937317+0000","last_fullsized":"2024-03-14T09:02:21.937317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:05:42.439216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766382+0000","last_change":"2024-03-14T09:02:03.336025+0000","last_active":"2024-03-14T09:02:20.766382+0000","last_peered":"2024-03-14T09:02:20.766382+0000","last_clean":"2024-03-14T09:02:20.766382+0000","last_became_active":"2024-03-14T09:02:03.335598+0000","last_became_peered":"2024-03-14T09:02:03.335598+0000","last_unstale":"2024-03-14T09:02:20.766382+0000","last_undegraded":"2024-03-14T09:02:20.766382+0000","last_fullsized":"2024-03-14T09:02:20.766382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766846+0000","last_change":"2024-03-14T09:02:03.751604+0000","last_active":"2024-03-14T09:02:20.766846+0000","last_peered":"2024-03-14T09:02:20.766846+0000","last_clean":"2024-03-14T09:02:20.766846+0000","last_became_active":"2024-03-14T09:02:03.751176+0000","last_became_peered":"2024-03-14T09:02:03.751176+0000","last_unstale":"2024-03-14T09:02:20.766846+0000","last_undegraded":"2024-03-14T09:02:20.766846+0000","last_fullsized":"2024-03-14T09:02:20.766846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938070+0000","last_change":"2024-03-14T09:02:20.359696+0000","last_active":"2024-03-14T09:02:21.938070+0000","last_peered":"2024-03-14T09:02:21.938070+0000","last_clean":"2024-03-14T09:02:21.938070+0000","last_became_active":"2024-03-14T09:02:20.359360+0000","last_became_peered":"2024-03-14T09:02:20.359360+0000","last_unstale":"2024-03-14T09:02:21.938070+0000","last_undegraded":"2024-03-14T09:02:21.938070+0000","last_fullsized":"2024-03-14T09:02:21.938070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:19.333358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937509+0000","last_change":"2024-03-14T09:02:20.778225+0000","last_active":"2024-03-14T09:02:21.937509+0000","last_peered":"2024-03-14T09:02:21.937509+0000","last_clean":"2024-03-14T09:02:21.937509+0000","last_became_active":"2024-03-14T09:02:20.777932+0000","last_became_peered":"2024-03-14T09:02:20.777932+0000","last_unstale":"2024-03-14T09:02:21.937509+0000","last_undegraded":"2024-03-14T09:02:21.937509+0000","last_fullsized":"2024-03-14T09:02:21.937509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:06.064788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764244+0000","last_change":"2024-03-14T09:02:03.331842+0000","last_active":"2024-03-14T09:02:20.764244+0000","last_peered":"2024-03-14T09:02:20.764244+0000","last_clean":"2024-03-14T09:02:20.764244+0000","last_became_active":"2024-03-14T09:02:03.331505+0000","last_became_peered":"2024-03-14T09:02:03.331505+0000","last_unstale":"2024-03-14T09:02:20.764244+0000","last_undegraded":"2024-03-14T09:02:20.764244+0000","last_fullsized":"2024-03-14T09:02:20.764244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766857+0000","last_change":"2024-03-14T09:02:03.752238+0000","last_active":"2024-03-14T09:02:20.766857+0000","last_peered":"2024-03-14T09:02:20.766857+0000","last_clean":"2024-03-14T09:02:20.766857+0000","last_became_active":"2024-03-14T09:02:03.751848+0000","last_became_peered":"2024-03-14T09:02:03.751848+0000","last_unstale":"2024-03-14T09:02:20.766857+0000","last_undegraded":"2024-03-14T09:02:20.766857+0000","last_fullsized":"2024-03-14T09:02:20.766857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:12:11.855082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764353+0000","last_change":"2024-03-14T09:02:20.359596+0000","last_active":"2024-03-14T09:02:21.764353+0000","last_peered":"2024-03-14T09:02:21.764353+0000","last_clean":"2024-03-14T09:02:21.764353+0000","last_became_active":"2024-03-14T09:02:20.359020+0000","last_became_peered":"2024-03-14T09:02:20.359020+0000","last_unstale":"2024-03-14T09:02:21.764353+0000","last_undegraded":"2024-03-14T09:02:21.764353+0000","last_fullsized":"2024-03-14T09:02:21.764353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:51.977178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764174+0000","last_change":"2024-03-14T09:02:20.779161+0000","last_active":"2024-03-14T09:02:21.764174+0000","last_peered":"2024-03-14T09:02:21.764174+0000","last_clean":"2024-03-14T09:02:21.764174+0000","last_became_active":"2024-03-14T09:02:20.778628+0000","last_became_peered":"2024-03-14T09:02:20.778628+0000","last_unstale":"2024-03-14T09:02:21.764174+0000","last_undegraded":"2024-03-14T09:02:21.764174+0000","last_fullsized":"2024-03-14T09:02:21.764174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:35:40.762593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759569+0000","last_change":"2024-03-14T09:02:03.331651+0000","last_active":"2024-03-14T09:02:20.759569+0000","last_peered":"2024-03-14T09:02:20.759569+0000","last_clean":"2024-03-14T09:02:20.759569+0000","last_became_active":"2024-03-14T09:02:03.331386+0000","last_became_peered":"2024-03-14T09:02:03.331386+0000","last_unstale":"2024-03-14T09:02:20.759569+0000","last_undegraded":"2024-03-14T09:02:20.759569+0000","last_fullsized":"2024-03-14T09:02:20.759569+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767367+0000","last_change":"2024-03-14T09:02:03.754570+0000","last_active":"2024-03-14T09:02:20.767367+0000","last_peered":"2024-03-14T09:02:20.767367+0000","last_clean":"2024-03-14T09:02:20.767367+0000","last_became_active":"2024-03-14T09:02:03.752638+0000","last_became_peered":"2024-03-14T09:02:03.752638+0000","last_unstale":"2024-03-14T09:02:20.767367+0000","last_undegraded":"2024-03-14T09:02:20.767367+0000","last_fullsized":"2024-03-14T09:02:20.767367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:51.536705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936544+0000","last_change":"2024-03-14T09:02:20.355560+0000","last_active":"2024-03-14T09:02:21.936544+0000","last_peered":"2024-03-14T09:02:21.936544+0000","last_clean":"2024-03-14T09:02:21.936544+0000","last_became_active":"2024-03-14T09:02:20.355211+0000","last_became_peered":"2024-03-14T09:02:20.355211+0000","last_unstale":"2024-03-14T09:02:21.936544+0000","last_undegraded":"2024-03-14T09:02:21.936544+0000","last_fullsized":"2024-03-14T09:02:21.936544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:48:42.780123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936724+0000","last_change":"2024-03-14T09:02:20.771692+0000","last_active":"2024-03-14T09:02:21.936724+0000","last_peered":"2024-03-14T09:02:21.936724+0000","last_clean":"2024-03-14T09:02:21.936724+0000","last_became_active":"2024-03-14T09:02:20.771404+0000","last_became_peered":"2024-03-14T09:02:20.771404+0000","last_unstale":"2024-03-14T09:02:21.936724+0000","last_undegraded":"2024-03-14T09:02:21.936724+0000","last_fullsized":"2024-03-14T09:02:21.936724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763409+0000","last_change":"2024-03-14T09:02:03.332262+0000","last_active":"2024-03-14T09:02:20.763409+0000","last_peered":"2024-03-14T09:02:20.763409+0000","last_clean":"2024-03-14T09:02:20.763409+0000","last_became_active":"2024-03-14T09:02:03.331880+0000","last_became_peered":"2024-03-14T09:02:03.331880+0000","last_unstale":"2024-03-14T09:02:20.763409+0000","last_undegraded":"2024-03-14T09:02:20.763409+0000","last_fullsized":"2024-03-14T09:02:20.763409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766928+0000","last_change":"2024-03-14T09:02:03.750636+0000","last_active":"2024-03-14T09:02:20.766928+0000","last_peered":"2024-03-14T09:02:20.766928+0000","last_clean":"2024-03-14T09:02:20.766928+0000","last_became_active":"2024-03-14T09:02:03.750345+0000","last_became_peered":"2024-03-14T09:02:03.750345+0000","last_unstale":"2024-03-14T09:02:20.766928+0000","last_undegraded":"2024-03-14T09:02:20.766928+0000","last_fullsized":"2024-03-14T09:02:20.766928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:21.995078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936944+0000","last_change":"2024-03-14T09:02:20.359348+0000","last_active":"2024-03-14T09:02:21.936944+0000","last_peered":"2024-03-14T09:02:21.936944+0000","last_clean":"2024-03-14T09:02:21.936944+0000","last_became_active":"2024-03-14T09:02:20.359048+0000","last_became_peered":"2024-03-14T09:02:20.359048+0000","last_unstale":"2024-03-14T09:02:21.936944+0000","last_undegraded":"2024-03-14T09:02:21.936944+0000","last_fullsized":"2024-03-14T09:02:21.936944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:39.295329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758086+0000","last_change":"2024-03-14T09:02:20.773469+0000","last_active":"2024-03-14T09:02:21.758086+0000","last_peered":"2024-03-14T09:02:21.758086+0000","last_clean":"2024-03-14T09:02:21.758086+0000","last_became_active":"2024-03-14T09:02:20.773205+0000","last_became_peered":"2024-03-14T09:02:20.773205+0000","last_unstale":"2024-03-14T09:02:21.758086+0000","last_undegraded":"2024-03-14T09:02:21.758086+0000","last_fullsized":"2024-03-14T09:02:21.758086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:39:03.334981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766306+0000","last_change":"2024-03-14T09:02:03.333934+0000","last_active":"2024-03-14T09:02:20.766306+0000","last_peered":"2024-03-14T09:02:20.766306+0000","last_clean":"2024-03-14T09:02:20.766306+0000","last_became_active":"2024-03-14T09:02:03.332400+0000","last_became_peered":"2024-03-14T09:02:03.332400+0000","last_unstale":"2024-03-14T09:02:20.766306+0000","last_undegraded":"2024-03-14T09:02:20.766306+0000","last_fullsized":"2024-03-14T09:02:20.766306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767780+0000","last_change":"2024-03-14T09:02:03.750997+0000","last_active":"2024-03-14T09:02:20.767780+0000","last_peered":"2024-03-14T09:02:20.767780+0000","last_clean":"2024-03-14T09:02:20.767780+0000","last_became_active":"2024-03-14T09:02:03.750760+0000","last_became_peered":"2024-03-14T09:02:03.750760+0000","last_unstale":"2024-03-14T09:02:20.767780+0000","last_undegraded":"2024-03-14T09:02:20.767780+0000","last_fullsized":"2024-03-14T09:02:20.767780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:48.882749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937379+0000","last_change":"2024-03-14T09:02:20.357730+0000","last_active":"2024-03-14T09:02:21.937379+0000","last_peered":"2024-03-14T09:02:21.937379+0000","last_clean":"2024-03-14T09:02:21.937379+0000","last_became_active":"2024-03-14T09:02:20.357422+0000","last_became_peered":"2024-03-14T09:02:20.357422+0000","last_unstale":"2024-03-14T09:02:21.937379+0000","last_undegraded":"2024-03-14T09:02:21.937379+0000","last_fullsized":"2024-03-14T09:02:21.937379+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:31:48.692241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936681+0000","last_change":"2024-03-14T09:02:20.782899+0000","last_active":"2024-03-14T09:02:21.936681+0000","last_peered":"2024-03-14T09:02:21.936681+0000","last_clean":"2024-03-14T09:02:21.936681+0000","last_became_active":"2024-03-14T09:02:20.782406+0000","last_became_peered":"2024-03-14T09:02:20.782406+0000","last_unstale":"2024-03-14T09:02:21.936681+0000","last_undegraded":"2024-03-14T09:02:21.936681+0000","last_fullsized":"2024-03-14T09:02:21.936681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:01.997032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766650+0000","last_change":"2024-03-14T09:02:03.333813+0000","last_active":"2024-03-14T09:02:20.766650+0000","last_peered":"2024-03-14T09:02:20.766650+0000","last_clean":"2024-03-14T09:02:20.766650+0000","last_became_active":"2024-03-14T09:02:03.333439+0000","last_became_peered":"2024-03-14T09:02:03.333439+0000","last_unstale":"2024-03-14T09:02:20.766650+0000","last_undegraded":"2024-03-14T09:02:20.766650+0000","last_fullsized":"2024-03-14T09:02:20.766650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764111+0000","last_change":"2024-03-14T09:02:03.746425+0000","last_active":"2024-03-14T09:02:20.764111+0000","last_peered":"2024-03-14T09:02:20.764111+0000","last_clean":"2024-03-14T09:02:20.764111+0000","last_became_active":"2024-03-14T09:02:03.746141+0000","last_became_peered":"2024-03-14T09:02:03.746141+0000","last_unstale":"2024-03-14T09:02:20.764111+0000","last_undegraded":"2024-03-14T09:02:20.764111+0000","last_fullsized":"2024-03-14T09:02:20.764111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758801+0000","last_change":"2024-03-14T09:02:20.356289+0000","last_active":"2024-03-14T09:02:21.758801+0000","last_peered":"2024-03-14T09:02:21.758801+0000","last_clean":"2024-03-14T09:02:21.758801+0000","last_became_active":"2024-03-14T09:02:20.355902+0000","last_became_peered":"2024-03-14T09:02:20.355902+0000","last_unstale":"2024-03-14T09:02:21.758801+0000","last_undegraded":"2024-03-14T09:02:21.758801+0000","last_fullsized":"2024-03-14T09:02:21.758801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:55.212546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758657+0000","last_change":"2024-03-14T09:02:20.778213+0000","last_active":"2024-03-14T09:02:21.758657+0000","last_peered":"2024-03-14T09:02:21.758657+0000","last_clean":"2024-03-14T09:02:21.758657+0000","last_became_active":"2024-03-14T09:02:20.777980+0000","last_became_peered":"2024-03-14T09:02:20.777980+0000","last_unstale":"2024-03-14T09:02:21.758657+0000","last_undegraded":"2024-03-14T09:02:21.758657+0000","last_fullsized":"2024-03-14T09:02:21.758657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:40.316285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766955+0000","last_change":"2024-03-14T09:02:03.330088+0000","last_active":"2024-03-14T09:02:20.766955+0000","last_peered":"2024-03-14T09:02:20.766955+0000","last_clean":"2024-03-14T09:02:20.766955+0000","last_became_active":"2024-03-14T09:02:03.329788+0000","last_became_peered":"2024-03-14T09:02:03.329788+0000","last_unstale":"2024-03-14T09:02:20.766955+0000","last_undegraded":"2024-03-14T09:02:20.766955+0000","last_fullsized":"2024-03-14T09:02:20.766955+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768651+0000","last_change":"2024-03-14T09:02:03.746864+0000","last_active":"2024-03-14T09:02:20.768651+0000","last_peered":"2024-03-14T09:02:20.768651+0000","last_clean":"2024-03-14T09:02:20.768651+0000","last_became_active":"2024-03-14T09:02:03.746405+0000","last_became_peered":"2024-03-14T09:02:03.746405+0000","last_unstale":"2024-03-14T09:02:20.768651+0000","last_undegraded":"2024-03-14T09:02:20.768651+0000","last_fullsized":"2024-03-14T09:02:20.768651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764546+0000","last_change":"2024-03-14T09:02:20.353880+0000","last_active":"2024-03-14T09:02:21.764546+0000","last_peered":"2024-03-14T09:02:21.764546+0000","last_clean":"2024-03-14T09:02:21.764546+0000","last_became_active":"2024-03-14T09:02:20.353617+0000","last_became_peered":"2024-03-14T09:02:20.353617+0000","last_unstale":"2024-03-14T09:02:21.764546+0000","last_undegraded":"2024-03-14T09:02:21.764546+0000","last_fullsized":"2024-03-14T09:02:21.764546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.027692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763874+0000","last_change":"2024-03-14T09:02:20.773271+0000","last_active":"2024-03-14T09:02:21.763874+0000","last_peered":"2024-03-14T09:02:21.763874+0000","last_clean":"2024-03-14T09:02:21.763874+0000","last_became_active":"2024-03-14T09:02:20.772993+0000","last_became_peered":"2024-03-14T09:02:20.772993+0000","last_unstale":"2024-03-14T09:02:21.763874+0000","last_undegraded":"2024-03-14T09:02:21.763874+0000","last_fullsized":"2024-03-14T09:02:21.763874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:06.064788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765232+0000","last_change":"2024-03-14T09:02:03.328451+0000","last_active":"2024-03-14T09:02:20.765232+0000","last_peered":"2024-03-14T09:02:20.765232+0000","last_clean":"2024-03-14T09:02:20.765232+0000","last_became_active":"2024-03-14T09:02:03.328184+0000","last_became_peered":"2024-03-14T09:02:03.328184+0000","last_unstale":"2024-03-14T09:02:20.765232+0000","last_undegraded":"2024-03-14T09:02:20.765232+0000","last_fullsized":"2024-03-14T09:02:20.765232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766842+0000","last_change":"2024-03-14T09:02:03.743189+0000","last_active":"2024-03-14T09:02:20.766842+0000","last_peered":"2024-03-14T09:02:20.766842+0000","last_clean":"2024-03-14T09:02:20.766842+0000","last_became_active":"2024-03-14T09:02:03.742865+0000","last_became_peered":"2024-03-14T09:02:03.742865+0000","last_unstale":"2024-03-14T09:02:20.766842+0000","last_undegraded":"2024-03-14T09:02:20.766842+0000","last_fullsized":"2024-03-14T09:02:20.766842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938770+0000","last_change":"2024-03-14T09:02:20.360163+0000","last_active":"2024-03-14T09:02:21.938770+0000","last_peered":"2024-03-14T09:02:21.938770+0000","last_clean":"2024-03-14T09:02:21.938770+0000","last_became_active":"2024-03-14T09:02:20.359889+0000","last_became_peered":"2024-03-14T09:02:20.359889+0000","last_unstale":"2024-03-14T09:02:21.938770+0000","last_undegraded":"2024-03-14T09:02:21.938770+0000","last_fullsized":"2024-03-14T09:02:21.938770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:08.180878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763461+0000","last_change":"2024-03-14T09:02:20.778702+0000","last_active":"2024-03-14T09:02:21.763461+0000","last_peered":"2024-03-14T09:02:21.763461+0000","last_clean":"2024-03-14T09:02:21.763461+0000","last_became_active":"2024-03-14T09:02:20.778394+0000","last_became_peered":"2024-03-14T09:02:20.778394+0000","last_unstale":"2024-03-14T09:02:21.763461+0000","last_undegraded":"2024-03-14T09:02:21.763461+0000","last_fullsized":"2024-03-14T09:02:21.763461+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:46.738348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763940+0000","last_change":"2024-03-14T09:02:03.330619+0000","last_active":"2024-03-14T09:02:20.763940+0000","last_peered":"2024-03-14T09:02:20.763940+0000","last_clean":"2024-03-14T09:02:20.763940+0000","last_became_active":"2024-03-14T09:02:03.330237+0000","last_became_peered":"2024-03-14T09:02:03.330237+0000","last_unstale":"2024-03-14T09:02:20.763940+0000","last_undegraded":"2024-03-14T09:02:20.763940+0000","last_fullsized":"2024-03-14T09:02:20.763940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759749+0000","last_change":"2024-03-14T09:02:03.749453+0000","last_active":"2024-03-14T09:02:20.759749+0000","last_peered":"2024-03-14T09:02:20.759749+0000","last_clean":"2024-03-14T09:02:20.759749+0000","last_became_active":"2024-03-14T09:02:03.749186+0000","last_became_peered":"2024-03-14T09:02:03.749186+0000","last_unstale":"2024-03-14T09:02:20.759749+0000","last_undegraded":"2024-03-14T09:02:20.759749+0000","last_fullsized":"2024-03-14T09:02:20.759749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936590+0000","last_change":"2024-03-14T09:02:20.355286+0000","last_active":"2024-03-14T09:02:21.936590+0000","last_peered":"2024-03-14T09:02:21.936590+0000","last_clean":"2024-03-14T09:02:21.936590+0000","last_became_active":"2024-03-14T09:02:20.355110+0000","last_became_peered":"2024-03-14T09:02:20.355110+0000","last_unstale":"2024-03-14T09:02:21.936590+0000","last_undegraded":"2024-03-14T09:02:21.936590+0000","last_fullsized":"2024-03-14T09:02:21.936590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:53.714361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936438+0000","last_change":"2024-03-14T09:02:20.776834+0000","last_active":"2024-03-14T09:02:21.936438+0000","last_peered":"2024-03-14T09:02:21.936438+0000","last_clean":"2024-03-14T09:02:21.936438+0000","last_became_active":"2024-03-14T09:02:20.776322+0000","last_became_peered":"2024-03-14T09:02:20.776322+0000","last_unstale":"2024-03-14T09:02:21.936438+0000","last_undegraded":"2024-03-14T09:02:21.936438+0000","last_fullsized":"2024-03-14T09:02:21.936438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:52.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764129+0000","last_change":"2024-03-14T09:02:03.335208+0000","last_active":"2024-03-14T09:02:20.764129+0000","last_peered":"2024-03-14T09:02:20.764129+0000","last_clean":"2024-03-14T09:02:20.764129+0000","last_became_active":"2024-03-14T09:02:03.334929+0000","last_became_peered":"2024-03-14T09:02:03.334929+0000","last_unstale":"2024-03-14T09:02:20.764129+0000","last_undegraded":"2024-03-14T09:02:20.764129+0000","last_fullsized":"2024-03-14T09:02:20.764129+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767394+0000","last_change":"2024-03-14T09:02:03.747948+0000","last_active":"2024-03-14T09:02:20.767394+0000","last_peered":"2024-03-14T09:02:20.767394+0000","last_clean":"2024-03-14T09:02:20.767394+0000","last_became_active":"2024-03-14T09:02:03.747471+0000","last_became_peered":"2024-03-14T09:02:03.747471+0000","last_unstale":"2024-03-14T09:02:20.767394+0000","last_undegraded":"2024-03-14T09:02:20.767394+0000","last_fullsized":"2024-03-14T09:02:20.767394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764495+0000","last_change":"2024-03-14T09:02:20.358769+0000","last_active":"2024-03-14T09:02:21.764495+0000","last_peered":"2024-03-14T09:02:21.764495+0000","last_clean":"2024-03-14T09:02:21.764495+0000","last_became_active":"2024-03-14T09:02:20.358536+0000","last_became_peered":"2024-03-14T09:02:20.358536+0000","last_unstale":"2024-03-14T09:02:21.764495+0000","last_undegraded":"2024-03-14T09:02:21.764495+0000","last_fullsized":"2024-03-14T09:02:21.764495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:32.013553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763435+0000","last_change":"2024-03-14T09:02:20.781046+0000","last_active":"2024-03-14T09:02:21.763435+0000","last_peered":"2024-03-14T09:02:21.763435+0000","last_clean":"2024-03-14T09:02:21.763435+0000","last_became_active":"2024-03-14T09:02:20.780783+0000","last_became_peered":"2024-03-14T09:02:20.780783+0000","last_unstale":"2024-03-14T09:02:21.763435+0000","last_undegraded":"2024-03-14T09:02:21.763435+0000","last_fullsized":"2024-03-14T09:02:21.763435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:36:27.894843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763651+0000","last_change":"2024-03-14T09:02:03.329056+0000","last_active":"2024-03-14T09:02:20.763651+0000","last_peered":"2024-03-14T09:02:20.763651+0000","last_clean":"2024-03-14T09:02:20.763651+0000","last_became_active":"2024-03-14T09:02:03.328772+0000","last_became_peered":"2024-03-14T09:02:03.328772+0000","last_unstale":"2024-03-14T09:02:20.763651+0000","last_undegraded":"2024-03-14T09:02:20.763651+0000","last_fullsized":"2024-03-14T09:02:20.763651+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766053+0000","last_change":"2024-03-14T09:02:03.751429+0000","last_active":"2024-03-14T09:02:20.766053+0000","last_peered":"2024-03-14T09:02:20.766053+0000","last_clean":"2024-03-14T09:02:20.766053+0000","last_became_active":"2024-03-14T09:02:03.751167+0000","last_became_peered":"2024-03-14T09:02:03.751167+0000","last_unstale":"2024-03-14T09:02:20.766053+0000","last_undegraded":"2024-03-14T09:02:20.766053+0000","last_fullsized":"2024-03-14T09:02:20.766053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936191+0000","last_change":"2024-03-14T09:02:20.358584+0000","last_active":"2024-03-14T09:02:21.936191+0000","last_peered":"2024-03-14T09:02:21.936191+0000","last_clean":"2024-03-14T09:02:21.936191+0000","last_became_active":"2024-03-14T09:02:20.358323+0000","last_became_peered":"2024-03-14T09:02:20.358323+0000","last_unstale":"2024-03-14T09:02:21.936191+0000","last_undegraded":"2024-03-14T09:02:21.936191+0000","last_fullsized":"2024-03-14T09:02:21.936191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:39:02.329849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937396+0000","last_change":"2024-03-14T09:02:20.777813+0000","last_active":"2024-03-14T09:02:21.937396+0000","last_peered":"2024-03-14T09:02:21.937396+0000","last_clean":"2024-03-14T09:02:21.937396+0000","last_became_active":"2024-03-14T09:02:20.775422+0000","last_became_peered":"2024-03-14T09:02:20.775422+0000","last_unstale":"2024-03-14T09:02:21.937396+0000","last_undegraded":"2024-03-14T09:02:21.937396+0000","last_fullsized":"2024-03-14T09:02:21.937396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:57:04.137876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766139+0000","last_change":"2024-03-14T09:02:03.329164+0000","last_active":"2024-03-14T09:02:20.766139+0000","last_peered":"2024-03-14T09:02:20.766139+0000","last_clean":"2024-03-14T09:02:20.766139+0000","last_became_active":"2024-03-14T09:02:03.328895+0000","last_became_peered":"2024-03-14T09:02:03.328895+0000","last_unstale":"2024-03-14T09:02:20.766139+0000","last_undegraded":"2024-03-14T09:02:20.766139+0000","last_fullsized":"2024-03-14T09:02:20.766139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763499+0000","last_change":"2024-03-14T09:02:03.740610+0000","last_active":"2024-03-14T09:02:20.763499+0000","last_peered":"2024-03-14T09:02:20.763499+0000","last_clean":"2024-03-14T09:02:20.763499+0000","last_became_active":"2024-03-14T09:02:03.740348+0000","last_became_peered":"2024-03-14T09:02:03.740348+0000","last_unstale":"2024-03-14T09:02:20.763499+0000","last_undegraded":"2024-03-14T09:02:20.763499+0000","last_fullsized":"2024-03-14T09:02:20.763499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763973+0000","last_change":"2024-03-14T09:02:20.359924+0000","last_active":"2024-03-14T09:02:21.763973+0000","last_peered":"2024-03-14T09:02:21.763973+0000","last_clean":"2024-03-14T09:02:21.763973+0000","last_became_active":"2024-03-14T09:02:20.358221+0000","last_became_peered":"2024-03-14T09:02:20.358221+0000","last_unstale":"2024-03-14T09:02:21.763973+0000","last_undegraded":"2024-03-14T09:02:21.763973+0000","last_fullsized":"2024-03-14T09:02:21.763973+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:34.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763855+0000","last_change":"2024-03-14T09:02:20.773647+0000","last_active":"2024-03-14T09:02:21.763855+0000","last_peered":"2024-03-14T09:02:21.763855+0000","last_clean":"2024-03-14T09:02:21.763855+0000","last_became_active":"2024-03-14T09:02:20.773223+0000","last_became_peered":"2024-03-14T09:02:20.773223+0000","last_unstale":"2024-03-14T09:02:21.763855+0000","last_undegraded":"2024-03-14T09:02:21.763855+0000","last_fullsized":"2024-03-14T09:02:21.763855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:04:06.449049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'9","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766593+0000","last_change":"2024-03-14T09:02:03.333899+0000","last_active":"2024-03-14T09:02:20.766593+0000","last_peered":"2024-03-14T09:02:20.766593+0000","last_clean":"2024-03-14T09:02:20.766593+0000","last_became_active":"2024-03-14T09:02:03.333685+0000","last_became_peered":"2024-03-14T09:02:03.333685+0000","last_unstale":"2024-03-14T09:02:20.766593+0000","last_undegraded":"2024-03-14T09:02:20.766593+0000","last_fullsized":"2024-03-14T09:02:20.766593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768264+0000","last_change":"2024-03-14T09:02:03.741618+0000","last_active":"2024-03-14T09:02:20.768264+0000","last_peered":"2024-03-14T09:02:20.768264+0000","last_clean":"2024-03-14T09:02:20.768264+0000","last_became_active":"2024-03-14T09:02:03.740972+0000","last_became_peered":"2024-03-14T09:02:03.740972+0000","last_unstale":"2024-03-14T09:02:20.768264+0000","last_undegraded":"2024-03-14T09:02:20.768264+0000","last_fullsized":"2024-03-14T09:02:20.768264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937355+0000","last_change":"2024-03-14T09:02:20.359395+0000","last_active":"2024-03-14T09:02:21.937355+0000","last_peered":"2024-03-14T09:02:21.937355+0000","last_clean":"2024-03-14T09:02:21.937355+0000","last_became_active":"2024-03-14T09:02:20.359126+0000","last_became_peered":"2024-03-14T09:02:20.359126+0000","last_unstale":"2024-03-14T09:02:21.937355+0000","last_undegraded":"2024-03-14T09:02:21.937355+0000","last_fullsized":"2024-03-14T09:02:21.937355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:38:51.483975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937871+0000","last_change":"2024-03-14T09:02:20.778281+0000","last_active":"2024-03-14T09:02:21.937871+0000","last_peered":"2024-03-14T09:02:21.937871+0000","last_clean":"2024-03-14T09:02:21.937871+0000","last_became_active":"2024-03-14T09:02:20.778022+0000","last_became_peered":"2024-03-14T09:02:20.778022+0000","last_unstale":"2024-03-14T09:02:21.937871+0000","last_undegraded":"2024-03-14T09:02:21.937871+0000","last_fullsized":"2024-03-14T09:02:21.937871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:00.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768351+0000","last_change":"2024-03-14T09:02:03.335856+0000","last_active":"2024-03-14T09:02:20.768351+0000","last_peered":"2024-03-14T09:02:20.768351+0000","last_clean":"2024-03-14T09:02:20.768351+0000","last_became_active":"2024-03-14T09:02:03.335559+0000","last_became_peered":"2024-03-14T09:02:03.335559+0000","last_unstale":"2024-03-14T09:02:20.768351+0000","last_undegraded":"2024-03-14T09:02:20.768351+0000","last_fullsized":"2024-03-14T09:02:20.768351+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766674+0000","last_change":"2024-03-14T09:02:03.748711+0000","last_active":"2024-03-14T09:02:20.766674+0000","last_peered":"2024-03-14T09:02:20.766674+0000","last_clean":"2024-03-14T09:02:20.766674+0000","last_became_active":"2024-03-14T09:02:03.748483+0000","last_became_peered":"2024-03-14T09:02:03.748483+0000","last_unstale":"2024-03-14T09:02:20.766674+0000","last_undegraded":"2024-03-14T09:02:20.766674+0000","last_fullsized":"2024-03-14T09:02:20.766674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:43.215729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937350+0000","last_change":"2024-03-14T09:02:20.358762+0000","last_active":"2024-03-14T09:02:21.937350+0000","last_peered":"2024-03-14T09:02:21.937350+0000","last_clean":"2024-03-14T09:02:21.937350+0000","last_became_active":"2024-03-14T09:02:20.358405+0000","last_became_peered":"2024-03-14T09:02:20.358405+0000","last_unstale":"2024-03-14T09:02:21.937350+0000","last_undegraded":"2024-03-14T09:02:21.937350+0000","last_fullsized":"2024-03-14T09:02:21.937350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:35.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937568+0000","last_change":"2024-03-14T09:02:20.773160+0000","last_active":"2024-03-14T09:02:21.937568+0000","last_peered":"2024-03-14T09:02:21.937568+0000","last_clean":"2024-03-14T09:02:21.937568+0000","last_became_active":"2024-03-14T09:02:20.772180+0000","last_became_peered":"2024-03-14T09:02:20.772180+0000","last_unstale":"2024-03-14T09:02:21.937568+0000","last_undegraded":"2024-03-14T09:02:21.937568+0000","last_fullsized":"2024-03-14T09:02:21.937568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:43.032823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761610+0000","last_change":"2024-03-14T09:02:03.331300+0000","last_active":"2024-03-14T09:02:20.761610+0000","last_peered":"2024-03-14T09:02:20.761610+0000","last_clean":"2024-03-14T09:02:20.761610+0000","last_became_active":"2024-03-14T09:02:03.331033+0000","last_became_peered":"2024-03-14T09:02:03.331033+0000","last_unstale":"2024-03-14T09:02:20.761610+0000","last_undegraded":"2024-03-14T09:02:20.761610+0000","last_fullsized":"2024-03-14T09:02:20.761610+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765781+0000","last_change":"2024-03-14T09:02:03.758670+0000","last_active":"2024-03-14T09:02:20.765781+0000","last_peered":"2024-03-14T09:02:20.765781+0000","last_clean":"2024-03-14T09:02:20.765781+0000","last_became_active":"2024-03-14T09:02:03.758426+0000","last_became_peered":"2024-03-14T09:02:03.758426+0000","last_unstale":"2024-03-14T09:02:20.765781+0000","last_undegraded":"2024-03-14T09:02:20.765781+0000","last_fullsized":"2024-03-14T09:02:20.765781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:37:44.347728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"35'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758655+0000","last_change":"2024-03-14T09:02:20.350124+0000","last_active":"2024-03-14T09:02:21.758655+0000","last_peered":"2024-03-14T09:02:21.758655+0000","last_clean":"2024-03-14T09:02:21.758655+0000","last_became_active":"2024-03-14T09:02:20.349815+0000","last_became_peered":"2024-03-14T09:02:20.349815+0000","last_unstale":"2024-03-14T09:02:21.758655+0000","last_undegraded":"2024-03-14T09:02:21.758655+0000","last_fullsized":"2024-03-14T09:02:21.758655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:04:08.179245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936817+0000","last_change":"2024-03-14T09:02:20.781627+0000","last_active":"2024-03-14T09:02:21.936817+0000","last_peered":"2024-03-14T09:02:21.936817+0000","last_clean":"2024-03-14T09:02:21.936817+0000","last_became_active":"2024-03-14T09:02:20.781053+0000","last_became_peered":"2024-03-14T09:02:20.781053+0000","last_unstale":"2024-03-14T09:02:21.936817+0000","last_undegraded":"2024-03-14T09:02:21.936817+0000","last_fullsized":"2024-03-14T09:02:21.936817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:16.470852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765594+0000","last_change":"2024-03-14T09:02:03.335707+0000","last_active":"2024-03-14T09:02:20.765594+0000","last_peered":"2024-03-14T09:02:20.765594+0000","last_clean":"2024-03-14T09:02:20.765594+0000","last_became_active":"2024-03-14T09:02:03.335448+0000","last_became_peered":"2024-03-14T09:02:03.335448+0000","last_unstale":"2024-03-14T09:02:20.765594+0000","last_undegraded":"2024-03-14T09:02:20.765594+0000","last_fullsized":"2024-03-14T09:02:20.765594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.771658+0000","last_change":"2024-03-14T09:02:03.757103+0000","last_active":"2024-03-14T09:02:20.771658+0000","last_peered":"2024-03-14T09:02:20.771658+0000","last_clean":"2024-03-14T09:02:20.771658+0000","last_became_active":"2024-03-14T09:02:03.756572+0000","last_became_peered":"2024-03-14T09:02:03.756572+0000","last_unstale":"2024-03-14T09:02:20.771658+0000","last_undegraded":"2024-03-14T09:02:20.771658+0000","last_fullsized":"2024-03-14T09:02:20.771658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937732+0000","last_change":"2024-03-14T09:02:20.361261+0000","last_active":"2024-03-14T09:02:21.937732+0000","last_peered":"2024-03-14T09:02:21.937732+0000","last_clean":"2024-03-14T09:02:21.937732+0000","last_became_active":"2024-03-14T09:02:20.360979+0000","last_became_peered":"2024-03-14T09:02:20.360979+0000","last_unstale":"2024-03-14T09:02:21.937732+0000","last_undegraded":"2024-03-14T09:02:21.937732+0000","last_fullsized":"2024-03-14T09:02:21.937732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.027692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764350+0000","last_change":"2024-03-14T09:02:20.784002+0000","last_active":"2024-03-14T09:02:21.764350+0000","last_peered":"2024-03-14T09:02:21.764350+0000","last_clean":"2024-03-14T09:02:21.764350+0000","last_became_active":"2024-03-14T09:02:20.783551+0000","last_became_peered":"2024-03-14T09:02:20.783551+0000","last_unstale":"2024-03-14T09:02:21.764350+0000","last_undegraded":"2024-03-14T09:02:21.764350+0000","last_fullsized":"2024-03-14T09:02:21.764350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:14.007036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764576+0000","last_change":"2024-03-14T09:02:03.330919+0000","last_active":"2024-03-14T09:02:20.764576+0000","last_peered":"2024-03-14T09:02:20.764576+0000","last_clean":"2024-03-14T09:02:20.764576+0000","last_became_active":"2024-03-14T09:02:03.330613+0000","last_became_peered":"2024-03-14T09:02:03.330613+0000","last_unstale":"2024-03-14T09:02:20.764576+0000","last_undegraded":"2024-03-14T09:02:20.764576+0000","last_fullsized":"2024-03-14T09:02:20.764576+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.762617+0000","last_change":"2024-03-14T09:02:03.740713+0000","last_active":"2024-03-14T09:02:20.762617+0000","last_peered":"2024-03-14T09:02:20.762617+0000","last_clean":"2024-03-14T09:02:20.762617+0000","last_became_active":"2024-03-14T09:02:03.740458+0000","last_became_peered":"2024-03-14T09:02:03.740458+0000","last_unstale":"2024-03-14T09:02:20.762617+0000","last_undegraded":"2024-03-14T09:02:20.762617+0000","last_fullsized":"2024-03-14T09:02:20.762617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939323+0000","last_change":"2024-03-14T09:02:20.358973+0000","last_active":"2024-03-14T09:02:21.939323+0000","last_peered":"2024-03-14T09:02:21.939323+0000","last_clean":"2024-03-14T09:02:21.939323+0000","last_became_active":"2024-03-14T09:02:20.358681+0000","last_became_peered":"2024-03-14T09:02:20.358681+0000","last_unstale":"2024-03-14T09:02:21.939323+0000","last_undegraded":"2024-03-14T09:02:21.939323+0000","last_fullsized":"2024-03-14T09:02:21.939323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:43:04.366071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936343+0000","last_change":"2024-03-14T09:02:20.776985+0000","last_active":"2024-03-14T09:02:21.936343+0000","last_peered":"2024-03-14T09:02:21.936343+0000","last_clean":"2024-03-14T09:02:21.936343+0000","last_became_active":"2024-03-14T09:02:20.776630+0000","last_became_peered":"2024-03-14T09:02:20.776630+0000","last_unstale":"2024-03-14T09:02:21.936343+0000","last_undegraded":"2024-03-14T09:02:21.936343+0000","last_fullsized":"2024-03-14T09:02:21.936343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:54.719492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763983+0000","last_change":"2024-03-14T09:02:03.327627+0000","last_active":"2024-03-14T09:02:20.763983+0000","last_peered":"2024-03-14T09:02:20.763983+0000","last_clean":"2024-03-14T09:02:20.763983+0000","last_became_active":"2024-03-14T09:02:03.327348+0000","last_became_peered":"2024-03-14T09:02:03.327348+0000","last_unstale":"2024-03-14T09:02:20.763983+0000","last_undegraded":"2024-03-14T09:02:20.763983+0000","last_fullsized":"2024-03-14T09:02:20.763983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764657+0000","last_change":"2024-03-14T09:02:03.743458+0000","last_active":"2024-03-14T09:02:20.764657+0000","last_peered":"2024-03-14T09:02:20.764657+0000","last_clean":"2024-03-14T09:02:20.764657+0000","last_became_active":"2024-03-14T09:02:03.742694+0000","last_became_peered":"2024-03-14T09:02:03.742694+0000","last_unstale":"2024-03-14T09:02:20.764657+0000","last_undegraded":"2024-03-14T09:02:20.764657+0000","last_fullsized":"2024-03-14T09:02:20.764657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764183+0000","last_change":"2024-03-14T09:02:20.359756+0000","last_active":"2024-03-14T09:02:21.764183+0000","last_peered":"2024-03-14T09:02:21.764183+0000","last_clean":"2024-03-14T09:02:21.764183+0000","last_became_active":"2024-03-14T09:02:20.359362+0000","last_became_peered":"2024-03-14T09:02:20.359362+0000","last_unstale":"2024-03-14T09:02:21.764183+0000","last_undegraded":"2024-03-14T09:02:21.764183+0000","last_fullsized":"2024-03-14T09:02:21.764183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:38:17.194878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939056+0000","last_change":"2024-03-14T09:02:20.770488+0000","last_active":"2024-03-14T09:02:21.939056+0000","last_peered":"2024-03-14T09:02:21.939056+0000","last_clean":"2024-03-14T09:02:21.939056+0000","last_became_active":"2024-03-14T09:02:20.770177+0000","last_became_peered":"2024-03-14T09:02:20.770177+0000","last_unstale":"2024-03-14T09:02:21.939056+0000","last_undegraded":"2024-03-14T09:02:21.939056+0000","last_fullsized":"2024-03-14T09:02:21.939056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767417+0000","last_change":"2024-03-14T09:02:03.330899+0000","last_active":"2024-03-14T09:02:20.767417+0000","last_peered":"2024-03-14T09:02:20.767417+0000","last_clean":"2024-03-14T09:02:20.767417+0000","last_became_active":"2024-03-14T09:02:03.330649+0000","last_became_peered":"2024-03-14T09:02:03.330649+0000","last_unstale":"2024-03-14T09:02:20.767417+0000","last_undegraded":"2024-03-14T09:02:20.767417+0000","last_fullsized":"2024-03-14T09:02:20.767417+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.769282+0000","last_change":"2024-03-14T09:02:03.738226+0000","last_active":"2024-03-14T09:02:20.769282+0000","last_peered":"2024-03-14T09:02:20.769282+0000","last_clean":"2024-03-14T09:02:20.769282+0000","last_became_active":"2024-03-14T09:02:03.737960+0000","last_became_peered":"2024-03-14T09:02:03.737960+0000","last_unstale":"2024-03-14T09:02:20.769282+0000","last_undegraded":"2024-03-14T09:02:20.769282+0000","last_fullsized":"2024-03-14T09:02:20.769282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936337+0000","last_change":"2024-03-14T09:02:20.355279+0000","last_active":"2024-03-14T09:02:21.936337+0000","last_peered":"2024-03-14T09:02:21.936337+0000","last_clean":"2024-03-14T09:02:21.936337+0000","last_became_active":"2024-03-14T09:02:20.355030+0000","last_became_peered":"2024-03-14T09:02:20.355030+0000","last_unstale":"2024-03-14T09:02:21.936337+0000","last_undegraded":"2024-03-14T09:02:21.936337+0000","last_fullsized":"2024-03-14T09:02:21.936337+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:05.269571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763053+0000","last_change":"2024-03-14T09:02:20.771327+0000","last_active":"2024-03-14T09:02:21.763053+0000","last_peered":"2024-03-14T09:02:21.763053+0000","last_clean":"2024-03-14T09:02:21.763053+0000","last_became_active":"2024-03-14T09:02:20.770890+0000","last_became_peered":"2024-03-14T09:02:20.770890+0000","last_unstale":"2024-03-14T09:02:21.763053+0000","last_undegraded":"2024-03-14T09:02:21.763053+0000","last_fullsized":"2024-03-14T09:02:21.763053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:19:07.904413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766027+0000","last_change":"2024-03-14T09:02:03.331030+0000","last_active":"2024-03-14T09:02:20.766027+0000","last_peered":"2024-03-14T09:02:20.766027+0000","last_clean":"2024-03-14T09:02:20.766027+0000","last_became_active":"2024-03-14T09:02:03.330772+0000","last_became_peered":"2024-03-14T09:02:03.330772+0000","last_unstale":"2024-03-14T09:02:20.766027+0000","last_undegraded":"2024-03-14T09:02:20.766027+0000","last_fullsized":"2024-03-14T09:02:20.766027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766207+0000","last_change":"2024-03-14T09:02:03.747325+0000","last_active":"2024-03-14T09:02:20.766207+0000","last_peered":"2024-03-14T09:02:20.766207+0000","last_clean":"2024-03-14T09:02:20.766207+0000","last_became_active":"2024-03-14T09:02:03.747029+0000","last_became_peered":"2024-03-14T09:02:03.747029+0000","last_unstale":"2024-03-14T09:02:20.766207+0000","last_undegraded":"2024-03-14T09:02:20.766207+0000","last_fullsized":"2024-03-14T09:02:20.766207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938315+0000","last_change":"2024-03-14T09:02:20.360468+0000","last_active":"2024-03-14T09:02:21.938315+0000","last_peered":"2024-03-14T09:02:21.938315+0000","last_clean":"2024-03-14T09:02:21.938315+0000","last_became_active":"2024-03-14T09:02:20.360169+0000","last_became_peered":"2024-03-14T09:02:20.360169+0000","last_unstale":"2024-03-14T09:02:21.938315+0000","last_undegraded":"2024-03-14T09:02:21.938315+0000","last_fullsized":"2024-03-14T09:02:21.938315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:36:26.889712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936426+0000","last_change":"2024-03-14T09:02:20.778720+0000","last_active":"2024-03-14T09:02:21.936426+0000","last_peered":"2024-03-14T09:02:21.936426+0000","last_clean":"2024-03-14T09:02:21.936426+0000","last_became_active":"2024-03-14T09:02:20.778481+0000","last_became_peered":"2024-03-14T09:02:20.778481+0000","last_unstale":"2024-03-14T09:02:21.936426+0000","last_undegraded":"2024-03-14T09:02:21.936426+0000","last_fullsized":"2024-03-14T09:02:21.936426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:38:18.200010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767724+0000","last_change":"2024-03-14T09:02:03.331128+0000","last_active":"2024-03-14T09:02:20.767724+0000","last_peered":"2024-03-14T09:02:20.767724+0000","last_clean":"2024-03-14T09:02:20.767724+0000","last_became_active":"2024-03-14T09:02:03.330898+0000","last_became_peered":"2024-03-14T09:02:03.330898+0000","last_unstale":"2024-03-14T09:02:20.767724+0000","last_undegraded":"2024-03-14T09:02:20.767724+0000","last_fullsized":"2024-03-14T09:02:20.767724+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765250+0000","last_change":"2024-03-14T09:02:03.757204+0000","last_active":"2024-03-14T09:02:20.765250+0000","last_peered":"2024-03-14T09:02:20.765250+0000","last_clean":"2024-03-14T09:02:20.765250+0000","last_became_active":"2024-03-14T09:02:03.752933+0000","last_became_peered":"2024-03-14T09:02:03.752933+0000","last_unstale":"2024-03-14T09:02:20.765250+0000","last_undegraded":"2024-03-14T09:02:20.765250+0000","last_fullsized":"2024-03-14T09:02:20.765250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937274+0000","last_change":"2024-03-14T09:02:20.360708+0000","last_active":"2024-03-14T09:02:21.937274+0000","last_peered":"2024-03-14T09:02:21.937274+0000","last_clean":"2024-03-14T09:02:21.937274+0000","last_became_active":"2024-03-14T09:02:20.360433+0000","last_became_peered":"2024-03-14T09:02:20.360433+0000","last_unstale":"2024-03-14T09:02:21.937274+0000","last_undegraded":"2024-03-14T09:02:21.937274+0000","last_fullsized":"2024-03-14T09:02:21.937274+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:47.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763604+0000","last_change":"2024-03-14T09:02:20.769767+0000","last_active":"2024-03-14T09:02:21.763604+0000","last_peered":"2024-03-14T09:02:21.763604+0000","last_clean":"2024-03-14T09:02:21.763604+0000","last_became_active":"2024-03-14T09:02:20.769481+0000","last_became_peered":"2024-03-14T09:02:20.769481+0000","last_unstale":"2024-03-14T09:02:21.763604+0000","last_undegraded":"2024-03-14T09:02:21.763604+0000","last_fullsized":"2024-03-14T09:02:21.763604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:43:05.371202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765573+0000","last_change":"2024-03-14T09:02:03.331945+0000","last_active":"2024-03-14T09:02:20.765573+0000","last_peered":"2024-03-14T09:02:20.765573+0000","last_clean":"2024-03-14T09:02:20.765573+0000","last_became_active":"2024-03-14T09:02:03.331686+0000","last_became_peered":"2024-03-14T09:02:03.331686+0000","last_unstale":"2024-03-14T09:02:20.765573+0000","last_undegraded":"2024-03-14T09:02:20.765573+0000","last_fullsized":"2024-03-14T09:02:20.765573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766160+0000","last_change":"2024-03-14T09:02:03.750990+0000","last_active":"2024-03-14T09:02:20.766160+0000","last_peered":"2024-03-14T09:02:20.766160+0000","last_clean":"2024-03-14T09:02:20.766160+0000","last_became_active":"2024-03-14T09:02:03.750742+0000","last_became_peered":"2024-03-14T09:02:03.750742+0000","last_unstale":"2024-03-14T09:02:20.766160+0000","last_undegraded":"2024-03-14T09:02:20.766160+0000","last_fullsized":"2024-03-14T09:02:20.766160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764639+0000","last_change":"2024-03-14T09:02:20.360529+0000","last_active":"2024-03-14T09:02:21.764639+0000","last_peered":"2024-03-14T09:02:21.764639+0000","last_clean":"2024-03-14T09:02:21.764639+0000","last_became_active":"2024-03-14T09:02:20.360276+0000","last_became_peered":"2024-03-14T09:02:20.360276+0000","last_unstale":"2024-03-14T09:02:21.764639+0000","last_undegraded":"2024-03-14T09:02:21.764639+0000","last_fullsized":"2024-03-14T09:02:21.764639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:42.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936629+0000","last_change":"2024-03-14T09:02:20.778652+0000","last_active":"2024-03-14T09:02:21.936629+0000","last_peered":"2024-03-14T09:02:21.936629+0000","last_clean":"2024-03-14T09:02:21.936629+0000","last_became_active":"2024-03-14T09:02:20.778405+0000","last_became_peered":"2024-03-14T09:02:20.778405+0000","last_unstale":"2024-03-14T09:02:21.936629+0000","last_undegraded":"2024-03-14T09:02:21.936629+0000","last_fullsized":"2024-03-14T09:02:21.936629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:20.338489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761692+0000","last_change":"2024-03-14T09:02:03.335358+0000","last_active":"2024-03-14T09:02:20.761692+0000","last_peered":"2024-03-14T09:02:20.761692+0000","last_clean":"2024-03-14T09:02:20.761692+0000","last_became_active":"2024-03-14T09:02:03.335031+0000","last_became_peered":"2024-03-14T09:02:03.335031+0000","last_unstale":"2024-03-14T09:02:20.761692+0000","last_undegraded":"2024-03-14T09:02:20.761692+0000","last_fullsized":"2024-03-14T09:02:20.761692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759873+0000","last_change":"2024-03-14T09:02:03.750291+0000","last_active":"2024-03-14T09:02:20.759873+0000","last_peered":"2024-03-14T09:02:20.759873+0000","last_clean":"2024-03-14T09:02:20.759873+0000","last_became_active":"2024-03-14T09:02:03.749738+0000","last_became_peered":"2024-03-14T09:02:03.749738+0000","last_unstale":"2024-03-14T09:02:20.759873+0000","last_undegraded":"2024-03-14T09:02:20.759873+0000","last_fullsized":"2024-03-14T09:02:20.759873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938293+0000","last_change":"2024-03-14T09:02:20.358703+0000","last_active":"2024-03-14T09:02:21.938293+0000","last_peered":"2024-03-14T09:02:21.938293+0000","last_clean":"2024-03-14T09:02:21.938293+0000","last_became_active":"2024-03-14T09:02:20.358330+0000","last_became_peered":"2024-03-14T09:02:20.358330+0000","last_unstale":"2024-03-14T09:02:21.938293+0000","last_undegraded":"2024-03-14T09:02:21.938293+0000","last_fullsized":"2024-03-14T09:02:21.938293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:00.991900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936390+0000","last_change":"2024-03-14T09:02:20.769652+0000","last_active":"2024-03-14T09:02:21.936390+0000","last_peered":"2024-03-14T09:02:21.936390+0000","last_clean":"2024-03-14T09:02:21.936390+0000","last_became_active":"2024-03-14T09:02:20.769412+0000","last_became_peered":"2024-03-14T09:02:20.769412+0000","last_unstale":"2024-03-14T09:02:21.936390+0000","last_undegraded":"2024-03-14T09:02:21.936390+0000","last_fullsized":"2024-03-14T09:02:21.936390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:12:16.436058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759792+0000","last_change":"2024-03-14T09:02:03.331160+0000","last_active":"2024-03-14T09:02:20.759792+0000","last_peered":"2024-03-14T09:02:20.759792+0000","last_clean":"2024-03-14T09:02:20.759792+0000","last_became_active":"2024-03-14T09:02:03.330643+0000","last_became_peered":"2024-03-14T09:02:03.330643+0000","last_unstale":"2024-03-14T09:02:20.759792+0000","last_undegraded":"2024-03-14T09:02:20.759792+0000","last_fullsized":"2024-03-14T09:02:20.759792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761770+0000","last_change":"2024-03-14T09:02:03.744443+0000","last_active":"2024-03-14T09:02:20.761770+0000","last_peered":"2024-03-14T09:02:20.761770+0000","last_clean":"2024-03-14T09:02:20.761770+0000","last_became_active":"2024-03-14T09:02:03.744149+0000","last_became_peered":"2024-03-14T09:02:03.744149+0000","last_unstale":"2024-03-14T09:02:20.761770+0000","last_undegraded":"2024-03-14T09:02:20.761770+0000","last_fullsized":"2024-03-14T09:02:20.761770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936613+0000","last_change":"2024-03-14T09:02:20.360511+0000","last_active":"2024-03-14T09:02:21.936613+0000","last_peered":"2024-03-14T09:02:21.936613+0000","last_clean":"2024-03-14T09:02:21.936613+0000","last_became_active":"2024-03-14T09:02:20.360257+0000","last_became_peered":"2024-03-14T09:02:20.360257+0000","last_unstale":"2024-03-14T09:02:21.936613+0000","last_undegraded":"2024-03-14T09:02:21.936613+0000","last_fullsized":"2024-03-14T09:02:21.936613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:40.867971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939059+0000","last_change":"2024-03-14T09:02:20.779931+0000","last_active":"2024-03-14T09:02:21.939059+0000","last_peered":"2024-03-14T09:02:21.939059+0000","last_clean":"2024-03-14T09:02:21.939059+0000","last_became_active":"2024-03-14T09:02:20.779669+0000","last_became_peered":"2024-03-14T09:02:20.779669+0000","last_unstale":"2024-03-14T09:02:21.939059+0000","last_undegraded":"2024-03-14T09:02:21.939059+0000","last_fullsized":"2024-03-14T09:02:21.939059+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767479+0000","last_change":"2024-03-14T09:02:03.747713+0000","last_active":"2024-03-14T09:02:20.767479+0000","last_peered":"2024-03-14T09:02:20.767479+0000","last_clean":"2024-03-14T09:02:20.767479+0000","last_became_active":"2024-03-14T09:02:03.747113+0000","last_became_peered":"2024-03-14T09:02:03.747113+0000","last_unstale":"2024-03-14T09:02:20.767479+0000","last_undegraded":"2024-03-14T09:02:20.767479+0000","last_fullsized":"2024-03-14T09:02:20.767479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939001+0000","last_change":"2024-03-14T09:02:20.773464+0000","last_active":"2024-03-14T09:02:21.939001+0000","last_peered":"2024-03-14T09:02:21.939001+0000","last_clean":"2024-03-14T09:02:21.939001+0000","last_became_active":"2024-03-14T09:02:20.773138+0000","last_became_peered":"2024-03-14T09:02:20.773138+0000","last_unstale":"2024-03-14T09:02:21.939001+0000","last_undegraded":"2024-03-14T09:02:21.939001+0000","last_fullsized":"2024-03-14T09:02:21.939001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759647+0000","last_change":"2024-03-14T09:02:03.754647+0000","last_active":"2024-03-14T09:02:20.759647+0000","last_peered":"2024-03-14T09:02:20.759647+0000","last_clean":"2024-03-14T09:02:20.759647+0000","last_became_active":"2024-03-14T09:02:03.750909+0000","last_became_peered":"2024-03-14T09:02:03.750909+0000","last_unstale":"2024-03-14T09:02:20.759647+0000","last_undegraded":"2024-03-14T09:02:20.759647+0000","last_fullsized":"2024-03-14T09:02:20.759647+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763173+0000","last_change":"2024-03-14T09:02:20.771229+0000","last_active":"2024-03-14T09:02:21.763173+0000","last_peered":"2024-03-14T09:02:21.763173+0000","last_clean":"2024-03-14T09:02:21.763173+0000","last_became_active":"2024-03-14T09:02:20.770913+0000","last_became_peered":"2024-03-14T09:02:20.770913+0000","last_unstale":"2024-03-14T09:02:21.763173+0000","last_undegraded":"2024-03-14T09:02:21.763173+0000","last_fullsized":"2024-03-14T09:02:21.763173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:31:49.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766290+0000","last_change":"2024-03-14T09:02:03.754829+0000","last_active":"2024-03-14T09:02:20.766290+0000","last_peered":"2024-03-14T09:02:20.766290+0000","last_clean":"2024-03-14T09:02:20.766290+0000","last_became_active":"2024-03-14T09:02:03.754553+0000","last_became_peered":"2024-03-14T09:02:03.754553+0000","last_unstale":"2024-03-14T09:02:20.766290+0000","last_undegraded":"2024-03-14T09:02:20.766290+0000","last_fullsized":"2024-03-14T09:02:20.766290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758117+0000","last_change":"2024-03-14T09:02:20.783186+0000","last_active":"2024-03-14T09:02:21.758117+0000","last_peered":"2024-03-14T09:02:21.758117+0000","last_clean":"2024-03-14T09:02:21.758117+0000","last_became_active":"2024-03-14T09:02:20.782885+0000","last_became_peered":"2024-03-14T09:02:20.782885+0000","last_unstale":"2024-03-14T09:02:21.758117+0000","last_undegraded":"2024-03-14T09:02:21.758117+0000","last_fullsized":"2024-03-14T09:02:21.758117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:40.300461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765885+0000","last_change":"2024-03-14T09:02:03.751534+0000","last_active":"2024-03-14T09:02:20.765885+0000","last_peered":"2024-03-14T09:02:20.765885+0000","last_clean":"2024-03-14T09:02:20.765885+0000","last_became_active":"2024-03-14T09:02:03.751265+0000","last_became_peered":"2024-03-14T09:02:03.751265+0000","last_unstale":"2024-03-14T09:02:20.765885+0000","last_undegraded":"2024-03-14T09:02:20.765885+0000","last_fullsized":"2024-03-14T09:02:20.765885+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764144+0000","last_change":"2024-03-14T09:02:20.781982+0000","last_active":"2024-03-14T09:02:21.764144+0000","last_peered":"2024-03-14T09:02:21.764144+0000","last_clean":"2024-03-14T09:02:21.764144+0000","last_became_active":"2024-03-14T09:02:20.781688+0000","last_became_peered":"2024-03-14T09:02:20.781688+0000","last_unstale":"2024-03-14T09:02:21.764144+0000","last_undegraded":"2024-03-14T09:02:21.764144+0000","last_fullsized":"2024-03-14T09:02:21.764144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:42.685345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758533+0000","last_change":"2024-03-14T09:02:20.783091+0000","last_active":"2024-03-14T09:02:21.758533+0000","last_peered":"2024-03-14T09:02:21.758533+0000","last_clean":"2024-03-14T09:02:21.758533+0000","last_became_active":"2024-03-14T09:02:20.782795+0000","last_became_peered":"2024-03-14T09:02:20.782795+0000","last_unstale":"2024-03-14T09:02:21.758533+0000","last_undegraded":"2024-03-14T09:02:21.758533+0000","last_fullsized":"2024-03-14T09:02:21.758533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:56:11.975080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768271+0000","last_change":"2024-03-14T09:02:03.737708+0000","last_active":"2024-03-14T09:02:20.768271+0000","last_peered":"2024-03-14T09:02:20.768271+0000","last_clean":"2024-03-14T09:02:20.768271+0000","last_became_active":"2024-03-14T09:02:03.737433+0000","last_became_peered":"2024-03-14T09:02:03.737433+0000","last_unstale":"2024-03-14T09:02:20.768271+0000","last_undegraded":"2024-03-14T09:02:20.768271+0000","last_fullsized":"2024-03-14T09:02:20.768271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763489+0000","last_change":"2024-03-14T09:02:20.764694+0000","last_active":"2024-03-14T09:02:21.763489+0000","last_peered":"2024-03-14T09:02:21.763489+0000","last_clean":"2024-03-14T09:02:21.763489+0000","last_became_active":"2024-03-14T09:02:20.764405+0000","last_became_peered":"2024-03-14T09:02:20.764405+0000","last_unstale":"2024-03-14T09:02:21.763489+0000","last_undegraded":"2024-03-14T09:02:21.763489+0000","last_fullsized":"2024-03-14T09:02:21.763489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:32:40.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765323+0000","last_change":"2024-03-14T09:02:03.757244+0000","last_active":"2024-03-14T09:02:20.765323+0000","last_peered":"2024-03-14T09:02:20.765323+0000","last_clean":"2024-03-14T09:02:20.765323+0000","last_became_active":"2024-03-14T09:02:03.753248+0000","last_became_peered":"2024-03-14T09:02:03.753248+0000","last_unstale":"2024-03-14T09:02:20.765323+0000","last_undegraded":"2024-03-14T09:02:20.765323+0000","last_fullsized":"2024-03-14T09:02:20.765323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766862+0000","last_change":"2024-03-14T09:02:03.753751+0000","last_active":"2024-03-14T09:02:20.766862+0000","last_peered":"2024-03-14T09:02:20.766862+0000","last_clean":"2024-03-14T09:02:20.766862+0000","last_became_active":"2024-03-14T09:02:03.753485+0000","last_became_peered":"2024-03-14T09:02:03.753485+0000","last_unstale":"2024-03-14T09:02:20.766862+0000","last_undegraded":"2024-03-14T09:02:20.766862+0000","last_fullsized":"2024-03-14T09:02:20.766862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:54.950201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938630+0000","last_change":"2024-03-14T09:02:20.778873+0000","last_active":"2024-03-14T09:02:21.938630+0000","last_peered":"2024-03-14T09:02:21.938630+0000","last_clean":"2024-03-14T09:02:21.938630+0000","last_became_active":"2024-03-14T09:02:20.778235+0000","last_became_peered":"2024-03-14T09:02:20.778235+0000","last_unstale":"2024-03-14T09:02:21.938630+0000","last_undegraded":"2024-03-14T09:02:21.938630+0000","last_fullsized":"2024-03-14T09:02:21.938630+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936869+0000","last_change":"2024-03-14T09:02:20.774761+0000","last_active":"2024-03-14T09:02:21.936869+0000","last_peered":"2024-03-14T09:02:21.936869+0000","last_clean":"2024-03-14T09:02:21.936869+0000","last_became_active":"2024-03-14T09:02:20.774462+0000","last_became_peered":"2024-03-14T09:02:20.774462+0000","last_unstale":"2024-03-14T09:02:21.936869+0000","last_undegraded":"2024-03-14T09:02:21.936869+0000","last_fullsized":"2024-03-14T09:02:21.936869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:54.719492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765449+0000","last_change":"2024-03-14T09:02:03.746056+0000","last_active":"2024-03-14T09:02:20.765449+0000","last_peered":"2024-03-14T09:02:20.765449+0000","last_clean":"2024-03-14T09:02:20.765449+0000","last_became_active":"2024-03-14T09:02:03.745200+0000","last_became_peered":"2024-03-14T09:02:03.745200+0000","last_unstale":"2024-03-14T09:02:20.765449+0000","last_undegraded":"2024-03-14T09:02:20.765449+0000","last_fullsized":"2024-03-14T09:02:20.765449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937137+0000","last_change":"2024-03-14T09:02:20.774448+0000","last_active":"2024-03-14T09:02:21.937137+0000","last_peered":"2024-03-14T09:02:21.937137+0000","last_clean":"2024-03-14T09:02:21.937137+0000","last_became_active":"2024-03-14T09:02:20.774210+0000","last_became_peered":"2024-03-14T09:02:20.774210+0000","last_unstale":"2024-03-14T09:02:21.937137+0000","last_undegraded":"2024-03-14T09:02:21.937137+0000","last_fullsized":"2024-03-14T09:02:21.937137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766292+0000","last_change":"2024-03-14T09:02:03.757604+0000","last_active":"2024-03-14T09:02:20.766292+0000","last_peered":"2024-03-14T09:02:20.766292+0000","last_clean":"2024-03-14T09:02:20.766292+0000","last_became_active":"2024-03-14T09:02:03.757387+0000","last_became_peered":"2024-03-14T09:02:03.757387+0000","last_unstale":"2024-03-14T09:02:20.766292+0000","last_undegraded":"2024-03-14T09:02:20.766292+0000","last_fullsized":"2024-03-14T09:02:20.766292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764134+0000","last_change":"2024-03-14T09:02:20.784598+0000","last_active":"2024-03-14T09:02:21.764134+0000","last_peered":"2024-03-14T09:02:21.764134+0000","last_clean":"2024-03-14T09:02:21.764134+0000","last_became_active":"2024-03-14T09:02:20.784352+0000","last_became_peered":"2024-03-14T09:02:20.784352+0000","last_unstale":"2024-03-14T09:02:21.764134+0000","last_undegraded":"2024-03-14T09:02:21.764134+0000","last_fullsized":"2024-03-14T09:02:21.764134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:02:31.536153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.758604+0000","last_change":"2024-03-14T09:02:03.741584+0000","last_active":"2024-03-14T09:02:20.758604+0000","last_peered":"2024-03-14T09:02:20.758604+0000","last_clean":"2024-03-14T09:02:20.758604+0000","last_became_active":"2024-03-14T09:02:03.741367+0000","last_became_peered":"2024-03-14T09:02:03.741367+0000","last_unstale":"2024-03-14T09:02:20.758604+0000","last_undegraded":"2024-03-14T09:02:20.758604+0000","last_fullsized":"2024-03-14T09:02:20.758604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763037+0000","last_change":"2024-03-14T09:02:20.771790+0000","last_active":"2024-03-14T09:02:21.763037+0000","last_peered":"2024-03-14T09:02:21.763037+0000","last_clean":"2024-03-14T09:02:21.763037+0000","last_became_active":"2024-03-14T09:02:20.771470+0000","last_became_peered":"2024-03-14T09:02:20.771470+0000","last_unstale":"2024-03-14T09:02:21.763037+0000","last_undegraded":"2024-03-14T09:02:21.763037+0000","last_fullsized":"2024-03-14T09:02:21.763037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:48.311747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767538+0000","last_change":"2024-03-14T09:02:03.749275+0000","last_active":"2024-03-14T09:02:20.767538+0000","last_peered":"2024-03-14T09:02:20.767538+0000","last_clean":"2024-03-14T09:02:20.767538+0000","last_became_active":"2024-03-14T09:02:03.748874+0000","last_became_peered":"2024-03-14T09:02:03.748874+0000","last_unstale":"2024-03-14T09:02:20.767538+0000","last_undegraded":"2024-03-14T09:02:20.767538+0000","last_fullsized":"2024-03-14T09:02:20.767538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763591+0000","last_change":"2024-03-14T09:02:20.771541+0000","last_active":"2024-03-14T09:02:21.763591+0000","last_peered":"2024-03-14T09:02:21.763591+0000","last_clean":"2024-03-14T09:02:21.763591+0000","last_became_active":"2024-03-14T09:02:20.770773+0000","last_became_peered":"2024-03-14T09:02:20.770773+0000","last_unstale":"2024-03-14T09:02:21.763591+0000","last_undegraded":"2024-03-14T09:02:21.763591+0000","last_fullsized":"2024-03-14T09:02:21.763591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:20.338489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765790+0000","last_change":"2024-03-14T09:02:03.743740+0000","last_active":"2024-03-14T09:02:20.765790+0000","last_peered":"2024-03-14T09:02:20.765790+0000","last_clean":"2024-03-14T09:02:20.765790+0000","last_became_active":"2024-03-14T09:02:03.742714+0000","last_became_peered":"2024-03-14T09:02:03.742714+0000","last_unstale":"2024-03-14T09:02:20.765790+0000","last_undegraded":"2024-03-14T09:02:20.765790+0000","last_fullsized":"2024-03-14T09:02:20.765790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937088+0000","last_change":"2024-03-14T09:02:20.776124+0000","last_active":"2024-03-14T09:02:21.937088+0000","last_peered":"2024-03-14T09:02:21.937088+0000","last_clean":"2024-03-14T09:02:21.937088+0000","last_became_active":"2024-03-14T09:02:20.775420+0000","last_became_peered":"2024-03-14T09:02:20.775420+0000","last_unstale":"2024-03-14T09:02:21.937088+0000","last_undegraded":"2024-03-14T09:02:21.937088+0000","last_fullsized":"2024-03-14T09:02:21.937088+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:00:51.224503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767795+0000","last_change":"2024-03-14T09:02:03.757453+0000","last_active":"2024-03-14T09:02:20.767795+0000","last_peered":"2024-03-14T09:02:20.767795+0000","last_clean":"2024-03-14T09:02:20.767795+0000","last_became_active":"2024-03-14T09:02:03.757182+0000","last_became_peered":"2024-03-14T09:02:03.757182+0000","last_unstale":"2024-03-14T09:02:20.767795+0000","last_undegraded":"2024-03-14T09:02:20.767795+0000","last_fullsized":"2024-03-14T09:02:20.767795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:09.988215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937455+0000","last_change":"2024-03-14T09:02:20.779878+0000","last_active":"2024-03-14T09:02:21.937455+0000","last_peered":"2024-03-14T09:02:21.937455+0000","last_clean":"2024-03-14T09:02:21.937455+0000","last_became_active":"2024-03-14T09:02:20.779625+0000","last_became_peered":"2024-03-14T09:02:20.779625+0000","last_unstale":"2024-03-14T09:02:21.937455+0000","last_undegraded":"2024-03-14T09:02:21.937455+0000","last_fullsized":"2024-03-14T09:02:21.937455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:03.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765504+0000","last_change":"2024-03-14T09:02:03.765489+0000","last_active":"2024-03-14T09:02:20.765504+0000","last_peered":"2024-03-14T09:02:20.765504+0000","last_clean":"2024-03-14T09:02:20.765504+0000","last_became_active":"2024-03-14T09:02:03.765228+0000","last_became_peered":"2024-03-14T09:02:03.765228+0000","last_unstale":"2024-03-14T09:02:20.765504+0000","last_undegraded":"2024-03-14T09:02:20.765504+0000","last_fullsized":"2024-03-14T09:02:20.765504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:39:59.816967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758557+0000","last_change":"2024-03-14T09:02:20.776749+0000","last_active":"2024-03-14T09:02:21.758557+0000","last_peered":"2024-03-14T09:02:21.758557+0000","last_clean":"2024-03-14T09:02:21.758557+0000","last_became_active":"2024-03-14T09:02:20.776512+0000","last_became_peered":"2024-03-14T09:02:20.776512+0000","last_unstale":"2024-03-14T09:02:21.758557+0000","last_undegraded":"2024-03-14T09:02:21.758557+0000","last_fullsized":"2024-03-14T09:02:21.758557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:41.157569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766494+0000","last_change":"2024-03-14T09:02:03.760516+0000","last_active":"2024-03-14T09:02:20.766494+0000","last_peered":"2024-03-14T09:02:20.766494+0000","last_clean":"2024-03-14T09:02:20.766494+0000","last_became_active":"2024-03-14T09:02:03.760292+0000","last_became_peered":"2024-03-14T09:02:03.760292+0000","last_unstale":"2024-03-14T09:02:20.766494+0000","last_undegraded":"2024-03-14T09:02:20.766494+0000","last_fullsized":"2024-03-14T09:02:20.766494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:42:45.106158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763387+0000","last_change":"2024-03-14T09:02:20.781172+0000","last_active":"2024-03-14T09:02:21.763387+0000","last_peered":"2024-03-14T09:02:21.763387+0000","last_clean":"2024-03-14T09:02:21.763387+0000","last_became_active":"2024-03-14T09:02:20.780307+0000","last_became_peered":"2024-03-14T09:02:20.780307+0000","last_unstale":"2024-03-14T09:02:21.763387+0000","last_undegraded":"2024-03-14T09:02:21.763387+0000","last_fullsized":"2024-03-14T09:02:21.763387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:48:43.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.770355+0000","last_change":"2024-03-14T09:02:03.756745+0000","last_active":"2024-03-14T09:02:20.770355+0000","last_peered":"2024-03-14T09:02:20.770355+0000","last_clean":"2024-03-14T09:02:20.770355+0000","last_became_active":"2024-03-14T09:02:03.756499+0000","last_became_peered":"2024-03-14T09:02:03.756499+0000","last_unstale":"2024-03-14T09:02:20.770355+0000","last_undegraded":"2024-03-14T09:02:20.770355+0000","last_fullsized":"2024-03-14T09:02:20.770355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937052+0000","last_change":"2024-03-14T09:02:20.771393+0000","last_active":"2024-03-14T09:02:21.937052+0000","last_peered":"2024-03-14T09:02:21.937052+0000","last_clean":"2024-03-14T09:02:21.937052+0000","last_became_active":"2024-03-14T09:02:20.770865+0000","last_became_peered":"2024-03-14T09:02:20.770865+0000","last_unstale":"2024-03-14T09:02:21.937052+0000","last_undegraded":"2024-03-14T09:02:21.937052+0000","last_fullsized":"2024-03-14T09:02:21.937052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:43.032823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767198+0000","last_change":"2024-03-14T09:02:03.752051+0000","last_active":"2024-03-14T09:02:20.767198+0000","last_peered":"2024-03-14T09:02:20.767198+0000","last_clean":"2024-03-14T09:02:20.767198+0000","last_became_active":"2024-03-14T09:02:03.751771+0000","last_became_peered":"2024-03-14T09:02:03.751771+0000","last_unstale":"2024-03-14T09:02:20.767198+0000","last_undegraded":"2024-03-14T09:02:20.767198+0000","last_fullsized":"2024-03-14T09:02:20.767198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758117+0000","last_change":"2024-03-14T09:02:20.785050+0000","last_active":"2024-03-14T09:02:21.758117+0000","last_peered":"2024-03-14T09:02:21.758117+0000","last_clean":"2024-03-14T09:02:21.758117+0000","last_became_active":"2024-03-14T09:02:20.784824+0000","last_became_peered":"2024-03-14T09:02:20.784824+0000","last_unstale":"2024-03-14T09:02:21.758117+0000","last_undegraded":"2024-03-14T09:02:21.758117+0000","last_fullsized":"2024-03-14T09:02:21.758117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:02.131037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767943+0000","last_change":"2024-03-14T09:02:03.751964+0000","last_active":"2024-03-14T09:02:20.767943+0000","last_peered":"2024-03-14T09:02:20.767943+0000","last_clean":"2024-03-14T09:02:20.767943+0000","last_became_active":"2024-03-14T09:02:03.751703+0000","last_became_peered":"2024-03-14T09:02:03.751703+0000","last_unstale":"2024-03-14T09:02:20.767943+0000","last_undegraded":"2024-03-14T09:02:20.767943+0000","last_fullsized":"2024-03-14T09:02:20.767943+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.762891+0000","last_change":"2024-03-14T09:02:20.782589+0000","last_active":"2024-03-14T09:02:21.762891+0000","last_peered":"2024-03-14T09:02:21.762891+0000","last_clean":"2024-03-14T09:02:21.762891+0000","last_became_active":"2024-03-14T09:02:20.782261+0000","last_became_peered":"2024-03-14T09:02:20.782261+0000","last_unstale":"2024-03-14T09:02:21.762891+0000","last_undegraded":"2024-03-14T09:02:21.762891+0000","last_fullsized":"2024-03-14T09:02:21.762891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:09.186009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764620+0000","last_change":"2024-03-14T09:02:03.752408+0000","last_active":"2024-03-14T09:02:20.764620+0000","last_peered":"2024-03-14T09:02:20.764620+0000","last_clean":"2024-03-14T09:02:20.764620+0000","last_became_active":"2024-03-14T09:02:03.752136+0000","last_became_peered":"2024-03-14T09:02:03.752136+0000","last_unstale":"2024-03-14T09:02:20.764620+0000","last_undegraded":"2024-03-14T09:02:20.764620+0000","last_fullsized":"2024-03-14T09:02:20.764620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764128+0000","last_change":"2024-03-14T09:02:20.774892+0000","last_active":"2024-03-14T09:02:21.764128+0000","last_peered":"2024-03-14T09:02:21.764128+0000","last_clean":"2024-03-14T09:02:21.764128+0000","last_became_active":"2024-03-14T09:02:20.774574+0000","last_became_peered":"2024-03-14T09:02:20.774574+0000","last_unstale":"2024-03-14T09:02:21.764128+0000","last_undegraded":"2024-03-14T09:02:21.764128+0000","last_fullsized":"2024-03-14T09:02:21.764128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:52:28.330150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767440+0000","last_change":"2024-03-14T09:02:03.751100+0000","last_active":"2024-03-14T09:02:20.767440+0000","last_peered":"2024-03-14T09:02:20.767440+0000","last_clean":"2024-03-14T09:02:20.767440+0000","last_became_active":"2024-03-14T09:02:03.750866+0000","last_became_peered":"2024-03-14T09:02:03.750866+0000","last_unstale":"2024-03-14T09:02:20.767440+0000","last_undegraded":"2024-03-14T09:02:20.767440+0000","last_fullsized":"2024-03-14T09:02:20.767440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:07.275479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937198+0000","last_change":"2024-03-14T09:02:20.771963+0000","last_active":"2024-03-14T09:02:21.937198+0000","last_peered":"2024-03-14T09:02:21.937198+0000","last_clean":"2024-03-14T09:02:21.937198+0000","last_became_active":"2024-03-14T09:02:20.771333+0000","last_became_peered":"2024-03-14T09:02:20.771333+0000","last_unstale":"2024-03-14T09:02:21.937198+0000","last_undegraded":"2024-03-14T09:02:21.937198+0000","last_fullsized":"2024-03-14T09:02:21.937198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:23.832986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768428+0000","last_change":"2024-03-14T09:02:03.750480+0000","last_active":"2024-03-14T09:02:20.768428+0000","last_peered":"2024-03-14T09:02:20.768428+0000","last_clean":"2024-03-14T09:02:20.768428+0000","last_became_active":"2024-03-14T09:02:03.750198+0000","last_became_peered":"2024-03-14T09:02:03.750198+0000","last_unstale":"2024-03-14T09:02:20.768428+0000","last_undegraded":"2024-03-14T09:02:20.768428+0000","last_fullsized":"2024-03-14T09:02:20.768428+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936590+0000","last_change":"2024-03-14T09:02:20.772425+0000","last_active":"2024-03-14T09:02:21.936590+0000","last_peered":"2024-03-14T09:02:21.936590+0000","last_clean":"2024-03-14T09:02:21.936590+0000","last_became_active":"2024-03-14T09:02:20.771116+0000","last_became_peered":"2024-03-14T09:02:20.771116+0000","last_unstale":"2024-03-14T09:02:21.936590+0000","last_undegraded":"2024-03-14T09:02:21.936590+0000","last_fullsized":"2024-03-14T09:02:21.936590+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:00:51.224503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768811+0000","last_change":"2024-03-14T09:02:03.758860+0000","last_active":"2024-03-14T09:02:20.768811+0000","last_peered":"2024-03-14T09:02:20.768811+0000","last_clean":"2024-03-14T09:02:20.768811+0000","last_became_active":"2024-03-14T09:02:03.758547+0000","last_became_peered":"2024-03-14T09:02:03.758547+0000","last_unstale":"2024-03-14T09:02:20.768811+0000","last_undegraded":"2024-03-14T09:02:20.768811+0000","last_fullsized":"2024-03-14T09:02:20.768811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938606+0000","last_change":"2024-03-14T09:02:20.779057+0000","last_active":"2024-03-14T09:02:21.938606+0000","last_peered":"2024-03-14T09:02:21.938606+0000","last_clean":"2024-03-14T09:02:21.938606+0000","last_became_active":"2024-03-14T09:02:20.778597+0000","last_became_peered":"2024-03-14T09:02:20.778597+0000","last_unstale":"2024-03-14T09:02:21.938606+0000","last_undegraded":"2024-03-14T09:02:21.938606+0000","last_fullsized":"2024-03-14T09:02:21.938606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:33.018684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765127+0000","last_change":"2024-03-14T09:02:03.757773+0000","last_active":"2024-03-14T09:02:20.765127+0000","last_peered":"2024-03-14T09:02:20.765127+0000","last_clean":"2024-03-14T09:02:20.765127+0000","last_became_active":"2024-03-14T09:02:03.757426+0000","last_became_peered":"2024-03-14T09:02:03.757426+0000","last_unstale":"2024-03-14T09:02:20.765127+0000","last_undegraded":"2024-03-14T09:02:20.765127+0000","last_fullsized":"2024-03-14T09:02:20.765127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936879+0000","last_change":"2024-03-14T09:02:20.773017+0000","last_active":"2024-03-14T09:02:21.936879+0000","last_peered":"2024-03-14T09:02:21.936879+0000","last_clean":"2024-03-14T09:02:21.936879+0000","last_became_active":"2024-03-14T09:02:20.772060+0000","last_became_peered":"2024-03-14T09:02:20.772060+0000","last_unstale":"2024-03-14T09:02:21.936879+0000","last_undegraded":"2024-03-14T09:02:21.936879+0000","last_fullsized":"2024-03-14T09:02:21.936879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:03.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767353+0000","last_change":"2024-03-14T09:02:03.741974+0000","last_active":"2024-03-14T09:02:20.767353+0000","last_peered":"2024-03-14T09:02:20.767353+0000","last_clean":"2024-03-14T09:02:20.767353+0000","last_became_active":"2024-03-14T09:02:03.741657+0000","last_became_peered":"2024-03-14T09:02:03.741657+0000","last_unstale":"2024-03-14T09:02:20.767353+0000","last_undegraded":"2024-03-14T09:02:20.767353+0000","last_fullsized":"2024-03-14T09:02:20.767353+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764044+0000","last_change":"2024-03-14T09:02:20.778499+0000","last_active":"2024-03-14T09:02:21.764044+0000","last_peered":"2024-03-14T09:02:21.764044+0000","last_clean":"2024-03-14T09:02:21.764044+0000","last_became_active":"2024-03-14T09:02:20.778076+0000","last_became_peered":"2024-03-14T09:02:20.778076+0000","last_unstale":"2024-03-14T09:02:21.764044+0000","last_undegraded":"2024-03-14T09:02:21.764044+0000","last_fullsized":"2024-03-14T09:02:21.764044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:38:51.778569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767007+0000","last_change":"2024-03-14T09:02:03.753172+0000","last_active":"2024-03-14T09:02:20.767007+0000","last_peered":"2024-03-14T09:02:20.767007+0000","last_clean":"2024-03-14T09:02:20.767007+0000","last_became_active":"2024-03-14T09:02:03.752882+0000","last_became_peered":"2024-03-14T09:02:03.752882+0000","last_unstale":"2024-03-14T09:02:20.767007+0000","last_undegraded":"2024-03-14T09:02:20.767007+0000","last_fullsized":"2024-03-14T09:02:20.767007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937986+0000","last_change":"2024-03-14T09:02:20.781181+0000","last_active":"2024-03-14T09:02:21.937986+0000","last_peered":"2024-03-14T09:02:21.937986+0000","last_clean":"2024-03-14T09:02:21.937986+0000","last_became_active":"2024-03-14T09:02:20.780863+0000","last_became_peered":"2024-03-14T09:02:20.780863+0000","last_unstale":"2024-03-14T09:02:21.937986+0000","last_undegraded":"2024-03-14T09:02:21.937986+0000","last_fullsized":"2024-03-14T09:02:21.937986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:35.201864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766951+0000","last_change":"2024-03-14T09:02:03.751248+0000","last_active":"2024-03-14T09:02:20.766951+0000","last_peered":"2024-03-14T09:02:20.766951+0000","last_clean":"2024-03-14T09:02:20.766951+0000","last_became_active":"2024-03-14T09:02:03.750977+0000","last_became_peered":"2024-03-14T09:02:03.750977+0000","last_unstale":"2024-03-14T09:02:20.766951+0000","last_undegraded":"2024-03-14T09:02:20.766951+0000","last_fullsized":"2024-03-14T09:02:20.766951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936087+0000","last_change":"2024-03-14T09:02:20.769707+0000","last_active":"2024-03-14T09:02:21.936087+0000","last_peered":"2024-03-14T09:02:21.936087+0000","last_clean":"2024-03-14T09:02:21.936087+0000","last_became_active":"2024-03-14T09:02:20.769411+0000","last_became_peered":"2024-03-14T09:02:20.769411+0000","last_unstale":"2024-03-14T09:02:21.936087+0000","last_undegraded":"2024-03-14T09:02:21.936087+0000","last_fullsized":"2024-03-14T09:02:21.936087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:01:55.985257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764075+0000","last_change":"2024-03-14T09:02:03.747083+0000","last_active":"2024-03-14T09:02:20.764075+0000","last_peered":"2024-03-14T09:02:20.764075+0000","last_clean":"2024-03-14T09:02:20.764075+0000","last_became_active":"2024-03-14T09:02:03.743458+0000","last_became_peered":"2024-03-14T09:02:03.743458+0000","last_unstale":"2024-03-14T09:02:20.764075+0000","last_undegraded":"2024-03-14T09:02:20.764075+0000","last_fullsized":"2024-03-14T09:02:20.764075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936677+0000","last_change":"2024-03-14T09:02:20.781501+0000","last_active":"2024-03-14T09:02:21.936677+0000","last_peered":"2024-03-14T09:02:21.936677+0000","last_clean":"2024-03-14T09:02:21.936677+0000","last_became_active":"2024-03-14T09:02:20.781094+0000","last_became_peered":"2024-03-14T09:02:20.781094+0000","last_unstale":"2024-03-14T09:02:21.936677+0000","last_undegraded":"2024-03-14T09:02:21.936677+0000","last_fullsized":"2024-03-14T09:02:21.936677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:09:25.674123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767275+0000","last_change":"2024-03-14T09:02:03.751416+0000","last_active":"2024-03-14T09:02:20.767275+0000","last_peered":"2024-03-14T09:02:20.767275+0000","last_clean":"2024-03-14T09:02:20.767275+0000","last_became_active":"2024-03-14T09:02:03.751119+0000","last_became_peered":"2024-03-14T09:02:03.751119+0000","last_unstale":"2024-03-14T09:02:20.767275+0000","last_undegraded":"2024-03-14T09:02:20.767275+0000","last_fullsized":"2024-03-14T09:02:20.767275+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:06.397586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938863+0000","last_change":"2024-03-14T09:02:20.781357+0000","last_active":"2024-03-14T09:02:21.938863+0000","last_peered":"2024-03-14T09:02:21.938863+0000","last_clean":"2024-03-14T09:02:21.938863+0000","last_became_active":"2024-03-14T09:02:20.781031+0000","last_became_peered":"2024-03-14T09:02:20.781031+0000","last_unstale":"2024-03-14T09:02:21.938863+0000","last_undegraded":"2024-03-14T09:02:21.938863+0000","last_fullsized":"2024-03-14T09:02:21.938863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:23.832986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.762513+0000","last_change":"2024-03-14T09:02:03.747476+0000","last_active":"2024-03-14T09:02:20.762513+0000","last_peered":"2024-03-14T09:02:20.762513+0000","last_clean":"2024-03-14T09:02:20.762513+0000","last_became_active":"2024-03-14T09:02:03.747104+0000","last_became_peered":"2024-03-14T09:02:03.747104+0000","last_unstale":"2024-03-14T09:02:20.762513+0000","last_undegraded":"2024-03-14T09:02:20.762513+0000","last_fullsized":"2024-03-14T09:02:20.762513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937837+0000","last_change":"2024-03-14T09:02:20.783389+0000","last_active":"2024-03-14T09:02:21.937837+0000","last_peered":"2024-03-14T09:02:21.937837+0000","last_clean":"2024-03-14T09:02:21.937837+0000","last_became_active":"2024-03-14T09:02:20.783105+0000","last_became_peered":"2024-03-14T09:02:20.783105+0000","last_unstale":"2024-03-14T09:02:21.937837+0000","last_undegraded":"2024-03-14T09:02:21.937837+0000","last_fullsized":"2024-03-14T09:02:21.937837+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:22.386770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768186+0000","last_change":"2024-03-14T09:02:03.741721+0000","last_active":"2024-03-14T09:02:20.768186+0000","last_peered":"2024-03-14T09:02:20.768186+0000","last_clean":"2024-03-14T09:02:20.768186+0000","last_became_active":"2024-03-14T09:02:03.741350+0000","last_became_peered":"2024-03-14T09:02:03.741350+0000","last_unstale":"2024-03-14T09:02:20.768186+0000","last_undegraded":"2024-03-14T09:02:20.768186+0000","last_fullsized":"2024-03-14T09:02:20.768186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937222+0000","last_change":"2024-03-14T09:02:20.774158+0000","last_active":"2024-03-14T09:02:21.937222+0000","last_peered":"2024-03-14T09:02:21.937222+0000","last_clean":"2024-03-14T09:02:21.937222+0000","last_became_active":"2024-03-14T09:02:20.773930+0000","last_became_peered":"2024-03-14T09:02:20.773930+0000","last_unstale":"2024-03-14T09:02:21.937222+0000","last_undegraded":"2024-03-14T09:02:21.937222+0000","last_fullsized":"2024-03-14T09:02:21.937222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:33.018684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766577+0000","last_change":"2024-03-14T09:02:03.757259+0000","last_active":"2024-03-14T09:02:20.766577+0000","last_peered":"2024-03-14T09:02:20.766577+0000","last_clean":"2024-03-14T09:02:20.766577+0000","last_became_active":"2024-03-14T09:02:03.757050+0000","last_became_peered":"2024-03-14T09:02:03.757050+0000","last_unstale":"2024-03-14T09:02:20.766577+0000","last_undegraded":"2024-03-14T09:02:20.766577+0000","last_fullsized":"2024-03-14T09:02:20.766577+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937116+0000","last_change":"2024-03-14T09:02:20.771784+0000","last_active":"2024-03-14T09:02:21.937116+0000","last_peered":"2024-03-14T09:02:21.937116+0000","last_clean":"2024-03-14T09:02:21.937116+0000","last_became_active":"2024-03-14T09:02:20.771497+0000","last_became_peered":"2024-03-14T09:02:20.771497+0000","last_unstale":"2024-03-14T09:02:21.937116+0000","last_undegraded":"2024-03-14T09:02:21.937116+0000","last_fullsized":"2024-03-14T09:02:21.937116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:05:42.439216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759931+0000","last_change":"2024-03-14T09:02:03.755819+0000","last_active":"2024-03-14T09:02:20.759931+0000","last_peered":"2024-03-14T09:02:20.759931+0000","last_clean":"2024-03-14T09:02:20.759931+0000","last_became_active":"2024-03-14T09:02:03.750027+0000","last_became_peered":"2024-03-14T09:02:03.750027+0000","last_unstale":"2024-03-14T09:02:20.759931+0000","last_undegraded":"2024-03-14T09:02:20.759931+0000","last_fullsized":"2024-03-14T09:02:20.759931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":758536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.622},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.78800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":425984,"data_stored":169276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":171155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.22900000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.37}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":758390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.31}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.755},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":169644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":438272,"data_stored":171723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.314},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.374},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.504}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":169656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.311},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.505},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":168730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.622}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:24.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:02:25.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.029+0000 7f9027351700 1 Processor -- start 2024-03-14T09:02:25.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.029+0000 7f9027351700 1 -- start start 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9027351700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f902014eaf0 0x7f902006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9027351700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200ac8e0 con 0x7f902006acb0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9027351700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902006de30 con 0x7f9020156500 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9027351700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902006b310 con 0x7f902014eaf0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f90248ec700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f902006acb0 0x7f9020155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33752/0 (socket says 172.21.15.154:33752) 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f90248ec700 1 -- 172.21.15.154:0/1338358407 learned_addr learned my addr 172.21.15.154:0/1338358407 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405524839 0 0) 0x7f90200ac8e0 con 0x7f902006acb0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c0035b0 con 0x7f902006acb0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f90250ed700 1 --2- 172.21.15.154:0/1338358407 >> v2:172.21.15.191:3300/0 conn(0x7f902014eaf0 0x7f902006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187130715 0 0) 0x7f900c0035b0 con 0x7f902006acb0 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 >> v2:172.21.15.191:3300/0 conn(0x7f902014eaf0 msgr2=0x7f902006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:25.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 --2- 172.21.15.154:0/1338358407 >> v2:172.21.15.191:3300/0 conn(0x7f902014eaf0 0x7f902006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 >> v1:172.21.15.191:6789/0 conn(0x7f9020156500 legacy=0x7f902015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020156910 con 0x7f902006acb0 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9010001670 con 0x7f902006acb0 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 >> v1:172.21.15.154:6789/0 conn(0x7f902006acb0 legacy=0x7f9020155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c003a00 0x7f900c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f900c004850 0x7f900c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 0x7f900c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902006b310 con 0x7f900c005330 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00c200 con 0x7f900c004850 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.030+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00c3c0 con 0x7f900c003a00 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 0x7f900c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90248ec700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f900c004850 0x7f900c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90250ed700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c003a00 0x7f900c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 -- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c003a00 msgr2=0x7f900c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c003a00 0x7f900c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 -- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f900c004850 msgr2=0x7f900c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f900c004850 0x7f900c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 -- 172.21.15.154:0/1338358407 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0035b0 con 0x7f900c005330 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 -- 172.21.15.154:0/1338358407 reap_dead start 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f90258ee700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 0x7f900c00ba10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f901800c160 tx=0x7f900c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baac522a23ea59f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018001db0 con 0x7f900c005330 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901800cca0 con 0x7f900c005330 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.031+0000 7f9017fff700 1 -- 172.21.15.154:0/1338358407 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018003c00 con 0x7f900c005330 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 -- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 msgr2=0x7f900c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 0x7f900c00ba10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f901800c160 tx=0x7f900c005880 comp rx=0 tx=0).stop 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 -- 172.21.15.154:0/1338358407 shutdown_connections 2024-03-14T09:02:25.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 --2- 172.21.15.154:0/1338358407 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900c005330 0x7f900c00ba10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 -- 172.21.15.154:0/1338358407 >> 172.21.15.154:0/1338358407 conn(0x7f90200c3880 msgr2=0x7f90200c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 -- 172.21.15.154:0/1338358407 shutdown_connections 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.032+0000 7f9027351700 1 -- 172.21.15.154:0/1338358407 wait complete. 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 Processor -- start 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 -- start start 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 0x7f90201ed720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 0x7f902013f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020140b50 con 0x7f902014f1f0 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90201443d0 con 0x7f90201406c0 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.033+0000 7f9027351700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90201403e0 con 0x7f900c005330 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47810/0 (socket says 172.21.15.154:47810) 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 -- 172.21.15.154:0/429135415 learned_addr learned my addr 172.21.15.154:0/429135415 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90248ec700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 0x7f902013f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90250ed700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 0x7f90201ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 -- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 msgr2=0x7f90201ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 0x7f90201ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 -- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 msgr2=0x7f902013f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 0x7f902013f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 -- 172.21.15.154:0/429135415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020144650 con 0x7f902014f1f0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90248ec700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 0x7f902013f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90250ed700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 0x7f90201ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f90258ee700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f902006d840 tx=0x7f90180014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018005070 con 0x7f902014f1f0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90180051d0 con 0x7f902014f1f0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90201502b0 con 0x7f902014f1f0 2024-03-14T09:02:25.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018005540 con 0x7f902014f1f0 2024-03-14T09:02:25.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.034+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201ee280 con 0x7f902014f1f0 2024-03-14T09:02:25.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.035+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90201447e0 con 0x7f902014f1f0 2024-03-14T09:02:25.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.036+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f90180056a0 con 0x7f902014f1f0 2024-03-14T09:02:25.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.036+0000 7f9015ffb700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 0x7f900406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.036+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f9018089f50 con 0x7f902014f1f0 2024-03-14T09:02:25.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.036+0000 7f90250ed700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 0x7f900406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.037+0000 7f90250ed700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 0x7f900406cd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f901c009be0 tx=0x7f901c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.041+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f90180550a0 con 0x7f902014f1f0 2024-03-14T09:02:25.230 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.231+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9020155700 con 0x7f900406a860 2024-03-14T09:02:25.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.245+0000 7f9015ffb700 1 -- 172.21.15.154:0/429135415 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527878 (secure 0 0 0) 0x7f901c009830 con 0x7f900406a860 2024-03-14T09:02:25.251 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:25.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 msgr2=0x7f900406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 0x7f900406cd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f901c009be0 tx=0x7f901c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:25.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 msgr2=0x7f902013f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f902006d840 tx=0x7f90180014a0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 shutdown_connections 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f900406a860 0x7f900406cd20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f902014f1f0 0x7f902013f670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f900c005330 0x7f90201ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 --2- 172.21.15.154:0/429135415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90201406c0 0x7f902013f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 >> 172.21.15.154:0/429135415 conn(0x7f90200c3880 msgr2=0x7f9020155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 shutdown_connections 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.250+0000 7f9027351700 1 -- 172.21.15.154:0/429135415 wait complete. 2024-03-14T09:02:25.257 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:02:25.262 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-14T09:02:23.532378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":310,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220716,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4595712,"data_stored":2537110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.194186"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767107+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:02:20.767107+0000","last_peered":"2024-03-14T09:02:20.767107+0000","last_clean":"2024-03-14T09:02:20.767107+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:02:20.767107+0000","last_undegraded":"2024-03-14T09:02:20.767107+0000","last_fullsized":"2024-03-14T09:02:20.767107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763302+0000","last_change":"2024-03-14T09:02:03.333342+0000","last_active":"2024-03-14T09:02:20.763302+0000","last_peered":"2024-03-14T09:02:20.763302+0000","last_clean":"2024-03-14T09:02:20.763302+0000","last_became_active":"2024-03-14T09:02:03.333078+0000","last_became_peered":"2024-03-14T09:02:03.333078+0000","last_unstale":"2024-03-14T09:02:20.763302+0000","last_undegraded":"2024-03-14T09:02:20.763302+0000","last_fullsized":"2024-03-14T09:02:20.763302+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767560+0000","last_change":"2024-03-14T09:02:03.758083+0000","last_active":"2024-03-14T09:02:20.767560+0000","last_peered":"2024-03-14T09:02:20.767560+0000","last_clean":"2024-03-14T09:02:20.767560+0000","last_became_active":"2024-03-14T09:02:03.757724+0000","last_became_peered":"2024-03-14T09:02:03.757724+0000","last_unstale":"2024-03-14T09:02:20.767560+0000","last_undegraded":"2024-03-14T09:02:20.767560+0000","last_fullsized":"2024-03-14T09:02:20.767560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939200+0000","last_change":"2024-03-14T09:02:20.357631+0000","last_active":"2024-03-14T09:02:21.939200+0000","last_peered":"2024-03-14T09:02:21.939200+0000","last_clean":"2024-03-14T09:02:21.939200+0000","last_became_active":"2024-03-14T09:02:20.356947+0000","last_became_peered":"2024-03-14T09:02:20.356947+0000","last_unstale":"2024-03-14T09:02:21.939200+0000","last_undegraded":"2024-03-14T09:02:21.939200+0000","last_fullsized":"2024-03-14T09:02:21.939200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:19:06.899281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937444+0000","last_change":"2024-03-14T09:02:20.780469+0000","last_active":"2024-03-14T09:02:21.937444+0000","last_peered":"2024-03-14T09:02:21.937444+0000","last_clean":"2024-03-14T09:02:21.937444+0000","last_became_active":"2024-03-14T09:02:20.780078+0000","last_became_peered":"2024-03-14T09:02:20.780078+0000","last_unstale":"2024-03-14T09:02:21.937444+0000","last_undegraded":"2024-03-14T09:02:21.937444+0000","last_fullsized":"2024-03-14T09:02:21.937444+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:35.201864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763756+0000","last_change":"2024-03-14T09:02:03.335789+0000","last_active":"2024-03-14T09:02:20.763756+0000","last_peered":"2024-03-14T09:02:20.763756+0000","last_clean":"2024-03-14T09:02:20.763756+0000","last_became_active":"2024-03-14T09:02:03.335514+0000","last_became_peered":"2024-03-14T09:02:03.335514+0000","last_unstale":"2024-03-14T09:02:20.763756+0000","last_undegraded":"2024-03-14T09:02:20.763756+0000","last_fullsized":"2024-03-14T09:02:20.763756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:56.556179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764073+0000","last_change":"2024-03-14T09:02:03.749871+0000","last_active":"2024-03-14T09:02:20.764073+0000","last_peered":"2024-03-14T09:02:20.764073+0000","last_clean":"2024-03-14T09:02:20.764073+0000","last_became_active":"2024-03-14T09:02:03.748885+0000","last_became_peered":"2024-03-14T09:02:03.748885+0000","last_unstale":"2024-03-14T09:02:20.764073+0000","last_undegraded":"2024-03-14T09:02:20.764073+0000","last_fullsized":"2024-03-14T09:02:20.764073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939256+0000","last_change":"2024-03-14T09:02:20.357221+0000","last_active":"2024-03-14T09:02:21.939256+0000","last_peered":"2024-03-14T09:02:21.939256+0000","last_clean":"2024-03-14T09:02:21.939256+0000","last_became_active":"2024-03-14T09:02:20.356857+0000","last_became_peered":"2024-03-14T09:02:20.356857+0000","last_unstale":"2024-03-14T09:02:21.939256+0000","last_undegraded":"2024-03-14T09:02:21.939256+0000","last_fullsized":"2024-03-14T09:02:21.939256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:40.867971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764405+0000","last_change":"2024-03-14T09:02:20.784094+0000","last_active":"2024-03-14T09:02:21.764405+0000","last_peered":"2024-03-14T09:02:21.764405+0000","last_clean":"2024-03-14T09:02:21.764405+0000","last_became_active":"2024-03-14T09:02:20.783828+0000","last_became_peered":"2024-03-14T09:02:20.783828+0000","last_unstale":"2024-03-14T09:02:21.764405+0000","last_undegraded":"2024-03-14T09:02:21.764405+0000","last_fullsized":"2024-03-14T09:02:21.764405+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.686979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764341+0000","last_change":"2024-03-14T09:02:03.332151+0000","last_active":"2024-03-14T09:02:20.764341+0000","last_peered":"2024-03-14T09:02:20.764341+0000","last_clean":"2024-03-14T09:02:20.764341+0000","last_became_active":"2024-03-14T09:02:03.331768+0000","last_became_peered":"2024-03-14T09:02:03.331768+0000","last_unstale":"2024-03-14T09:02:20.764341+0000","last_undegraded":"2024-03-14T09:02:20.764341+0000","last_fullsized":"2024-03-14T09:02:20.764341+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:11:36.348155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759955+0000","last_change":"2024-03-14T09:02:03.754008+0000","last_active":"2024-03-14T09:02:20.759955+0000","last_peered":"2024-03-14T09:02:20.759955+0000","last_clean":"2024-03-14T09:02:20.759955+0000","last_became_active":"2024-03-14T09:02:03.753707+0000","last_became_peered":"2024-03-14T09:02:03.753707+0000","last_unstale":"2024-03-14T09:02:20.759955+0000","last_undegraded":"2024-03-14T09:02:20.759955+0000","last_fullsized":"2024-03-14T09:02:20.759955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:23.291407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937022+0000","last_change":"2024-03-14T09:02:20.362219+0000","last_active":"2024-03-14T09:02:21.937022+0000","last_peered":"2024-03-14T09:02:21.937022+0000","last_clean":"2024-03-14T09:02:21.937022+0000","last_became_active":"2024-03-14T09:02:20.361957+0000","last_became_peered":"2024-03-14T09:02:20.361957+0000","last_unstale":"2024-03-14T09:02:21.937022+0000","last_undegraded":"2024-03-14T09:02:21.937022+0000","last_fullsized":"2024-03-14T09:02:21.937022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:08.180878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764262+0000","last_change":"2024-03-14T09:02:20.778325+0000","last_active":"2024-03-14T09:02:21.764262+0000","last_peered":"2024-03-14T09:02:21.764262+0000","last_clean":"2024-03-14T09:02:21.764262+0000","last_became_active":"2024-03-14T09:02:20.777176+0000","last_became_peered":"2024-03-14T09:02:20.777176+0000","last_unstale":"2024-03-14T09:02:21.764262+0000","last_undegraded":"2024-03-14T09:02:21.764262+0000","last_fullsized":"2024-03-14T09:02:21.764262+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:36.109417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763578+0000","last_change":"2024-03-14T09:02:03.323982+0000","last_active":"2024-03-14T09:02:20.763578+0000","last_peered":"2024-03-14T09:02:20.763578+0000","last_clean":"2024-03-14T09:02:20.763578+0000","last_became_active":"2024-03-14T09:02:03.323752+0000","last_became_peered":"2024-03-14T09:02:03.323752+0000","last_unstale":"2024-03-14T09:02:20.763578+0000","last_undegraded":"2024-03-14T09:02:20.763578+0000","last_fullsized":"2024-03-14T09:02:20.763578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763454+0000","last_change":"2024-03-14T09:02:03.750281+0000","last_active":"2024-03-14T09:02:20.763454+0000","last_peered":"2024-03-14T09:02:20.763454+0000","last_clean":"2024-03-14T09:02:20.763454+0000","last_became_active":"2024-03-14T09:02:03.750044+0000","last_became_peered":"2024-03-14T09:02:03.750044+0000","last_unstale":"2024-03-14T09:02:20.763454+0000","last_undegraded":"2024-03-14T09:02:20.763454+0000","last_fullsized":"2024-03-14T09:02:20.763454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936203+0000","last_change":"2024-03-14T09:02:20.358027+0000","last_active":"2024-03-14T09:02:21.936203+0000","last_peered":"2024-03-14T09:02:21.936203+0000","last_clean":"2024-03-14T09:02:21.936203+0000","last_became_active":"2024-03-14T09:02:20.357394+0000","last_became_peered":"2024-03-14T09:02:20.357394+0000","last_unstale":"2024-03-14T09:02:21.936203+0000","last_undegraded":"2024-03-14T09:02:21.936203+0000","last_fullsized":"2024-03-14T09:02:21.936203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:14:26.739393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937296+0000","last_change":"2024-03-14T09:02:20.779583+0000","last_active":"2024-03-14T09:02:21.937296+0000","last_peered":"2024-03-14T09:02:21.937296+0000","last_clean":"2024-03-14T09:02:21.937296+0000","last_became_active":"2024-03-14T09:02:20.779312+0000","last_became_peered":"2024-03-14T09:02:20.779312+0000","last_unstale":"2024-03-14T09:02:21.937296+0000","last_undegraded":"2024-03-14T09:02:21.937296+0000","last_fullsized":"2024-03-14T09:02:21.937296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:01.997032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763343+0000","last_change":"2024-03-14T09:02:03.332108+0000","last_active":"2024-03-14T09:02:20.763343+0000","last_peered":"2024-03-14T09:02:20.763343+0000","last_clean":"2024-03-14T09:02:20.763343+0000","last_became_active":"2024-03-14T09:02:03.331581+0000","last_became_peered":"2024-03-14T09:02:03.331581+0000","last_unstale":"2024-03-14T09:02:20.763343+0000","last_undegraded":"2024-03-14T09:02:20.763343+0000","last_fullsized":"2024-03-14T09:02:20.763343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.760070+0000","last_change":"2024-03-14T09:02:03.750999+0000","last_active":"2024-03-14T09:02:20.760070+0000","last_peered":"2024-03-14T09:02:20.760070+0000","last_clean":"2024-03-14T09:02:20.760070+0000","last_became_active":"2024-03-14T09:02:03.750615+0000","last_became_peered":"2024-03-14T09:02:03.750615+0000","last_unstale":"2024-03-14T09:02:20.760070+0000","last_undegraded":"2024-03-14T09:02:20.760070+0000","last_fullsized":"2024-03-14T09:02:20.760070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936526+0000","last_change":"2024-03-14T09:02:20.361477+0000","last_active":"2024-03-14T09:02:21.936526+0000","last_peered":"2024-03-14T09:02:21.936526+0000","last_clean":"2024-03-14T09:02:21.936526+0000","last_became_active":"2024-03-14T09:02:20.361258+0000","last_became_peered":"2024-03-14T09:02:20.361258+0000","last_unstale":"2024-03-14T09:02:21.936526+0000","last_undegraded":"2024-03-14T09:02:21.936526+0000","last_fullsized":"2024-03-14T09:02:21.936526+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:35.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936391+0000","last_change":"2024-03-14T09:02:20.774132+0000","last_active":"2024-03-14T09:02:21.936391+0000","last_peered":"2024-03-14T09:02:21.936391+0000","last_clean":"2024-03-14T09:02:21.936391+0000","last_became_active":"2024-03-14T09:02:20.773888+0000","last_became_peered":"2024-03-14T09:02:20.773888+0000","last_unstale":"2024-03-14T09:02:21.936391+0000","last_undegraded":"2024-03-14T09:02:21.936391+0000","last_fullsized":"2024-03-14T09:02:21.936391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:52.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764208+0000","last_change":"2024-03-14T09:02:03.329704+0000","last_active":"2024-03-14T09:02:20.764208+0000","last_peered":"2024-03-14T09:02:20.764208+0000","last_clean":"2024-03-14T09:02:20.764208+0000","last_became_active":"2024-03-14T09:02:03.328905+0000","last_became_peered":"2024-03-14T09:02:03.328905+0000","last_unstale":"2024-03-14T09:02:20.764208+0000","last_undegraded":"2024-03-14T09:02:20.764208+0000","last_fullsized":"2024-03-14T09:02:20.764208+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761908+0000","last_change":"2024-03-14T09:02:03.745980+0000","last_active":"2024-03-14T09:02:20.761908+0000","last_peered":"2024-03-14T09:02:20.761908+0000","last_clean":"2024-03-14T09:02:20.761908+0000","last_became_active":"2024-03-14T09:02:03.745710+0000","last_became_peered":"2024-03-14T09:02:03.745710+0000","last_unstale":"2024-03-14T09:02:20.761908+0000","last_undegraded":"2024-03-14T09:02:20.761908+0000","last_fullsized":"2024-03-14T09:02:20.761908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764918+0000","last_change":"2024-03-14T09:02:20.357859+0000","last_active":"2024-03-14T09:02:21.764918+0000","last_peered":"2024-03-14T09:02:21.764918+0000","last_clean":"2024-03-14T09:02:21.764918+0000","last_became_active":"2024-03-14T09:02:20.357581+0000","last_became_peered":"2024-03-14T09:02:20.357581+0000","last_unstale":"2024-03-14T09:02:21.764918+0000","last_undegraded":"2024-03-14T09:02:21.764918+0000","last_fullsized":"2024-03-14T09:02:21.764918+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:22.827855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763702+0000","last_change":"2024-03-14T09:02:20.778300+0000","last_active":"2024-03-14T09:02:21.763702+0000","last_peered":"2024-03-14T09:02:21.763702+0000","last_clean":"2024-03-14T09:02:21.763702+0000","last_became_active":"2024-03-14T09:02:20.778021+0000","last_became_peered":"2024-03-14T09:02:20.778021+0000","last_unstale":"2024-03-14T09:02:21.763702+0000","last_undegraded":"2024-03-14T09:02:21.763702+0000","last_fullsized":"2024-03-14T09:02:21.763702+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:12:16.436058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761145+0000","last_change":"2024-03-14T09:02:03.331634+0000","last_active":"2024-03-14T09:02:20.761145+0000","last_peered":"2024-03-14T09:02:20.761145+0000","last_clean":"2024-03-14T09:02:20.761145+0000","last_became_active":"2024-03-14T09:02:03.331213+0000","last_became_peered":"2024-03-14T09:02:03.331213+0000","last_unstale":"2024-03-14T09:02:20.761145+0000","last_undegraded":"2024-03-14T09:02:20.761145+0000","last_fullsized":"2024-03-14T09:02:20.761145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:32:41.250597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768513+0000","last_change":"2024-03-14T09:02:03.750602+0000","last_active":"2024-03-14T09:02:20.768513+0000","last_peered":"2024-03-14T09:02:20.768513+0000","last_clean":"2024-03-14T09:02:20.768513+0000","last_became_active":"2024-03-14T09:02:03.750193+0000","last_became_peered":"2024-03-14T09:02:03.750193+0000","last_unstale":"2024-03-14T09:02:20.768513+0000","last_undegraded":"2024-03-14T09:02:20.768513+0000","last_fullsized":"2024-03-14T09:02:20.768513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936487+0000","last_change":"2024-03-14T09:02:20.360009+0000","last_active":"2024-03-14T09:02:21.936487+0000","last_peered":"2024-03-14T09:02:21.936487+0000","last_clean":"2024-03-14T09:02:21.936487+0000","last_became_active":"2024-03-14T09:02:20.359817+0000","last_became_peered":"2024-03-14T09:02:20.359817+0000","last_unstale":"2024-03-14T09:02:21.936487+0000","last_undegraded":"2024-03-14T09:02:21.936487+0000","last_fullsized":"2024-03-14T09:02:21.936487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:47.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937746+0000","last_change":"2024-03-14T09:02:20.777827+0000","last_active":"2024-03-14T09:02:21.937746+0000","last_peered":"2024-03-14T09:02:21.937746+0000","last_clean":"2024-03-14T09:02:21.937746+0000","last_became_active":"2024-03-14T09:02:20.777510+0000","last_became_peered":"2024-03-14T09:02:20.777510+0000","last_unstale":"2024-03-14T09:02:21.937746+0000","last_undegraded":"2024-03-14T09:02:21.937746+0000","last_fullsized":"2024-03-14T09:02:21.937746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:01:55.985257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765374+0000","last_change":"2024-03-14T09:02:03.324982+0000","last_active":"2024-03-14T09:02:20.765374+0000","last_peered":"2024-03-14T09:02:20.765374+0000","last_clean":"2024-03-14T09:02:20.765374+0000","last_became_active":"2024-03-14T09:02:03.324739+0000","last_became_peered":"2024-03-14T09:02:03.324739+0000","last_unstale":"2024-03-14T09:02:20.765374+0000","last_undegraded":"2024-03-14T09:02:20.765374+0000","last_fullsized":"2024-03-14T09:02:20.765374+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:04:47.596109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767968+0000","last_change":"2024-03-14T09:02:03.751539+0000","last_active":"2024-03-14T09:02:20.767968+0000","last_peered":"2024-03-14T09:02:20.767968+0000","last_clean":"2024-03-14T09:02:20.767968+0000","last_became_active":"2024-03-14T09:02:03.751299+0000","last_became_peered":"2024-03-14T09:02:03.751299+0000","last_unstale":"2024-03-14T09:02:20.767968+0000","last_undegraded":"2024-03-14T09:02:20.767968+0000","last_fullsized":"2024-03-14T09:02:20.767968+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:38.365353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937512+0000","last_change":"2024-03-14T09:02:20.359861+0000","last_active":"2024-03-14T09:02:21.937512+0000","last_peered":"2024-03-14T09:02:21.937512+0000","last_clean":"2024-03-14T09:02:21.937512+0000","last_became_active":"2024-03-14T09:02:20.359576+0000","last_became_peered":"2024-03-14T09:02:20.359576+0000","last_unstale":"2024-03-14T09:02:21.937512+0000","last_undegraded":"2024-03-14T09:02:21.937512+0000","last_fullsized":"2024-03-14T09:02:21.937512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:02.654716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937317+0000","last_change":"2024-03-14T09:02:20.771966+0000","last_active":"2024-03-14T09:02:21.937317+0000","last_peered":"2024-03-14T09:02:21.937317+0000","last_clean":"2024-03-14T09:02:21.937317+0000","last_became_active":"2024-03-14T09:02:20.771325+0000","last_became_peered":"2024-03-14T09:02:20.771325+0000","last_unstale":"2024-03-14T09:02:21.937317+0000","last_undegraded":"2024-03-14T09:02:21.937317+0000","last_fullsized":"2024-03-14T09:02:21.937317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:05:42.439216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766382+0000","last_change":"2024-03-14T09:02:03.336025+0000","last_active":"2024-03-14T09:02:20.766382+0000","last_peered":"2024-03-14T09:02:20.766382+0000","last_clean":"2024-03-14T09:02:20.766382+0000","last_became_active":"2024-03-14T09:02:03.335598+0000","last_became_peered":"2024-03-14T09:02:03.335598+0000","last_unstale":"2024-03-14T09:02:20.766382+0000","last_undegraded":"2024-03-14T09:02:20.766382+0000","last_fullsized":"2024-03-14T09:02:20.766382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:03.148221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766846+0000","last_change":"2024-03-14T09:02:03.751604+0000","last_active":"2024-03-14T09:02:20.766846+0000","last_peered":"2024-03-14T09:02:20.766846+0000","last_clean":"2024-03-14T09:02:20.766846+0000","last_became_active":"2024-03-14T09:02:03.751176+0000","last_became_peered":"2024-03-14T09:02:03.751176+0000","last_unstale":"2024-03-14T09:02:20.766846+0000","last_undegraded":"2024-03-14T09:02:20.766846+0000","last_fullsized":"2024-03-14T09:02:20.766846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938070+0000","last_change":"2024-03-14T09:02:20.359696+0000","last_active":"2024-03-14T09:02:21.938070+0000","last_peered":"2024-03-14T09:02:21.938070+0000","last_clean":"2024-03-14T09:02:21.938070+0000","last_became_active":"2024-03-14T09:02:20.359360+0000","last_became_peered":"2024-03-14T09:02:20.359360+0000","last_unstale":"2024-03-14T09:02:21.938070+0000","last_undegraded":"2024-03-14T09:02:21.938070+0000","last_fullsized":"2024-03-14T09:02:21.938070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:19.333358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937509+0000","last_change":"2024-03-14T09:02:20.778225+0000","last_active":"2024-03-14T09:02:21.937509+0000","last_peered":"2024-03-14T09:02:21.937509+0000","last_clean":"2024-03-14T09:02:21.937509+0000","last_became_active":"2024-03-14T09:02:20.777932+0000","last_became_peered":"2024-03-14T09:02:20.777932+0000","last_unstale":"2024-03-14T09:02:21.937509+0000","last_undegraded":"2024-03-14T09:02:21.937509+0000","last_fullsized":"2024-03-14T09:02:21.937509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:06.064788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764244+0000","last_change":"2024-03-14T09:02:03.331842+0000","last_active":"2024-03-14T09:02:20.764244+0000","last_peered":"2024-03-14T09:02:20.764244+0000","last_clean":"2024-03-14T09:02:20.764244+0000","last_became_active":"2024-03-14T09:02:03.331505+0000","last_became_peered":"2024-03-14T09:02:03.331505+0000","last_unstale":"2024-03-14T09:02:20.764244+0000","last_undegraded":"2024-03-14T09:02:20.764244+0000","last_fullsized":"2024-03-14T09:02:20.764244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766857+0000","last_change":"2024-03-14T09:02:03.752238+0000","last_active":"2024-03-14T09:02:20.766857+0000","last_peered":"2024-03-14T09:02:20.766857+0000","last_clean":"2024-03-14T09:02:20.766857+0000","last_became_active":"2024-03-14T09:02:03.751848+0000","last_became_peered":"2024-03-14T09:02:03.751848+0000","last_unstale":"2024-03-14T09:02:20.766857+0000","last_undegraded":"2024-03-14T09:02:20.766857+0000","last_fullsized":"2024-03-14T09:02:20.766857+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:12:11.855082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764353+0000","last_change":"2024-03-14T09:02:20.359596+0000","last_active":"2024-03-14T09:02:21.764353+0000","last_peered":"2024-03-14T09:02:21.764353+0000","last_clean":"2024-03-14T09:02:21.764353+0000","last_became_active":"2024-03-14T09:02:20.359020+0000","last_became_peered":"2024-03-14T09:02:20.359020+0000","last_unstale":"2024-03-14T09:02:21.764353+0000","last_undegraded":"2024-03-14T09:02:21.764353+0000","last_fullsized":"2024-03-14T09:02:21.764353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:51.977178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764174+0000","last_change":"2024-03-14T09:02:20.779161+0000","last_active":"2024-03-14T09:02:21.764174+0000","last_peered":"2024-03-14T09:02:21.764174+0000","last_clean":"2024-03-14T09:02:21.764174+0000","last_became_active":"2024-03-14T09:02:20.778628+0000","last_became_peered":"2024-03-14T09:02:20.778628+0000","last_unstale":"2024-03-14T09:02:21.764174+0000","last_undegraded":"2024-03-14T09:02:21.764174+0000","last_fullsized":"2024-03-14T09:02:21.764174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:35:40.762593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759569+0000","last_change":"2024-03-14T09:02:03.331651+0000","last_active":"2024-03-14T09:02:20.759569+0000","last_peered":"2024-03-14T09:02:20.759569+0000","last_clean":"2024-03-14T09:02:20.759569+0000","last_became_active":"2024-03-14T09:02:03.331386+0000","last_became_peered":"2024-03-14T09:02:03.331386+0000","last_unstale":"2024-03-14T09:02:20.759569+0000","last_undegraded":"2024-03-14T09:02:20.759569+0000","last_fullsized":"2024-03-14T09:02:20.759569+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767367+0000","last_change":"2024-03-14T09:02:03.754570+0000","last_active":"2024-03-14T09:02:20.767367+0000","last_peered":"2024-03-14T09:02:20.767367+0000","last_clean":"2024-03-14T09:02:20.767367+0000","last_became_active":"2024-03-14T09:02:03.752638+0000","last_became_peered":"2024-03-14T09:02:03.752638+0000","last_unstale":"2024-03-14T09:02:20.767367+0000","last_undegraded":"2024-03-14T09:02:20.767367+0000","last_fullsized":"2024-03-14T09:02:20.767367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:51.536705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936544+0000","last_change":"2024-03-14T09:02:20.355560+0000","last_active":"2024-03-14T09:02:21.936544+0000","last_peered":"2024-03-14T09:02:21.936544+0000","last_clean":"2024-03-14T09:02:21.936544+0000","last_became_active":"2024-03-14T09:02:20.355211+0000","last_became_peered":"2024-03-14T09:02:20.355211+0000","last_unstale":"2024-03-14T09:02:21.936544+0000","last_undegraded":"2024-03-14T09:02:21.936544+0000","last_fullsized":"2024-03-14T09:02:21.936544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:48:42.780123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936724+0000","last_change":"2024-03-14T09:02:20.771692+0000","last_active":"2024-03-14T09:02:21.936724+0000","last_peered":"2024-03-14T09:02:21.936724+0000","last_clean":"2024-03-14T09:02:21.936724+0000","last_became_active":"2024-03-14T09:02:20.771404+0000","last_became_peered":"2024-03-14T09:02:20.771404+0000","last_unstale":"2024-03-14T09:02:21.936724+0000","last_undegraded":"2024-03-14T09:02:21.936724+0000","last_fullsized":"2024-03-14T09:02:21.936724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763409+0000","last_change":"2024-03-14T09:02:03.332262+0000","last_active":"2024-03-14T09:02:20.763409+0000","last_peered":"2024-03-14T09:02:20.763409+0000","last_clean":"2024-03-14T09:02:20.763409+0000","last_became_active":"2024-03-14T09:02:03.331880+0000","last_became_peered":"2024-03-14T09:02:03.331880+0000","last_unstale":"2024-03-14T09:02:20.763409+0000","last_undegraded":"2024-03-14T09:02:20.763409+0000","last_fullsized":"2024-03-14T09:02:20.763409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766928+0000","last_change":"2024-03-14T09:02:03.750636+0000","last_active":"2024-03-14T09:02:20.766928+0000","last_peered":"2024-03-14T09:02:20.766928+0000","last_clean":"2024-03-14T09:02:20.766928+0000","last_became_active":"2024-03-14T09:02:03.750345+0000","last_became_peered":"2024-03-14T09:02:03.750345+0000","last_unstale":"2024-03-14T09:02:20.766928+0000","last_undegraded":"2024-03-14T09:02:20.766928+0000","last_fullsized":"2024-03-14T09:02:20.766928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:21.995078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936944+0000","last_change":"2024-03-14T09:02:20.359348+0000","last_active":"2024-03-14T09:02:21.936944+0000","last_peered":"2024-03-14T09:02:21.936944+0000","last_clean":"2024-03-14T09:02:21.936944+0000","last_became_active":"2024-03-14T09:02:20.359048+0000","last_became_peered":"2024-03-14T09:02:20.359048+0000","last_unstale":"2024-03-14T09:02:21.936944+0000","last_undegraded":"2024-03-14T09:02:21.936944+0000","last_fullsized":"2024-03-14T09:02:21.936944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:39.295329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758086+0000","last_change":"2024-03-14T09:02:20.773469+0000","last_active":"2024-03-14T09:02:21.758086+0000","last_peered":"2024-03-14T09:02:21.758086+0000","last_clean":"2024-03-14T09:02:21.758086+0000","last_became_active":"2024-03-14T09:02:20.773205+0000","last_became_peered":"2024-03-14T09:02:20.773205+0000","last_unstale":"2024-03-14T09:02:21.758086+0000","last_undegraded":"2024-03-14T09:02:21.758086+0000","last_fullsized":"2024-03-14T09:02:21.758086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:39:03.334981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766306+0000","last_change":"2024-03-14T09:02:03.333934+0000","last_active":"2024-03-14T09:02:20.766306+0000","last_peered":"2024-03-14T09:02:20.766306+0000","last_clean":"2024-03-14T09:02:20.766306+0000","last_became_active":"2024-03-14T09:02:03.332400+0000","last_became_peered":"2024-03-14T09:02:03.332400+0000","last_unstale":"2024-03-14T09:02:20.766306+0000","last_undegraded":"2024-03-14T09:02:20.766306+0000","last_fullsized":"2024-03-14T09:02:20.766306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:15:32.332971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767780+0000","last_change":"2024-03-14T09:02:03.750997+0000","last_active":"2024-03-14T09:02:20.767780+0000","last_peered":"2024-03-14T09:02:20.767780+0000","last_clean":"2024-03-14T09:02:20.767780+0000","last_became_active":"2024-03-14T09:02:03.750760+0000","last_became_peered":"2024-03-14T09:02:03.750760+0000","last_unstale":"2024-03-14T09:02:20.767780+0000","last_undegraded":"2024-03-14T09:02:20.767780+0000","last_fullsized":"2024-03-14T09:02:20.767780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:48.882749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937379+0000","last_change":"2024-03-14T09:02:20.357730+0000","last_active":"2024-03-14T09:02:21.937379+0000","last_peered":"2024-03-14T09:02:21.937379+0000","last_clean":"2024-03-14T09:02:21.937379+0000","last_became_active":"2024-03-14T09:02:20.357422+0000","last_became_peered":"2024-03-14T09:02:20.357422+0000","last_unstale":"2024-03-14T09:02:21.937379+0000","last_undegraded":"2024-03-14T09:02:21.937379+0000","last_fullsized":"2024-03-14T09:02:21.937379+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:31:48.692241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936681+0000","last_change":"2024-03-14T09:02:20.782899+0000","last_active":"2024-03-14T09:02:21.936681+0000","last_peered":"2024-03-14T09:02:21.936681+0000","last_clean":"2024-03-14T09:02:21.936681+0000","last_became_active":"2024-03-14T09:02:20.782406+0000","last_became_peered":"2024-03-14T09:02:20.782406+0000","last_unstale":"2024-03-14T09:02:21.936681+0000","last_undegraded":"2024-03-14T09:02:21.936681+0000","last_fullsized":"2024-03-14T09:02:21.936681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:01.997032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766650+0000","last_change":"2024-03-14T09:02:03.333813+0000","last_active":"2024-03-14T09:02:20.766650+0000","last_peered":"2024-03-14T09:02:20.766650+0000","last_clean":"2024-03-14T09:02:20.766650+0000","last_became_active":"2024-03-14T09:02:03.333439+0000","last_became_peered":"2024-03-14T09:02:03.333439+0000","last_unstale":"2024-03-14T09:02:20.766650+0000","last_undegraded":"2024-03-14T09:02:20.766650+0000","last_fullsized":"2024-03-14T09:02:20.766650+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:05.392567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764111+0000","last_change":"2024-03-14T09:02:03.746425+0000","last_active":"2024-03-14T09:02:20.764111+0000","last_peered":"2024-03-14T09:02:20.764111+0000","last_clean":"2024-03-14T09:02:20.764111+0000","last_became_active":"2024-03-14T09:02:03.746141+0000","last_became_peered":"2024-03-14T09:02:03.746141+0000","last_unstale":"2024-03-14T09:02:20.764111+0000","last_undegraded":"2024-03-14T09:02:20.764111+0000","last_fullsized":"2024-03-14T09:02:20.764111+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758801+0000","last_change":"2024-03-14T09:02:20.356289+0000","last_active":"2024-03-14T09:02:21.758801+0000","last_peered":"2024-03-14T09:02:21.758801+0000","last_clean":"2024-03-14T09:02:21.758801+0000","last_became_active":"2024-03-14T09:02:20.355902+0000","last_became_peered":"2024-03-14T09:02:20.355902+0000","last_unstale":"2024-03-14T09:02:21.758801+0000","last_undegraded":"2024-03-14T09:02:21.758801+0000","last_fullsized":"2024-03-14T09:02:21.758801+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:55.212546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758657+0000","last_change":"2024-03-14T09:02:20.778213+0000","last_active":"2024-03-14T09:02:21.758657+0000","last_peered":"2024-03-14T09:02:21.758657+0000","last_clean":"2024-03-14T09:02:21.758657+0000","last_became_active":"2024-03-14T09:02:20.777980+0000","last_became_peered":"2024-03-14T09:02:20.777980+0000","last_unstale":"2024-03-14T09:02:21.758657+0000","last_undegraded":"2024-03-14T09:02:21.758657+0000","last_fullsized":"2024-03-14T09:02:21.758657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:08:40.316285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766955+0000","last_change":"2024-03-14T09:02:03.330088+0000","last_active":"2024-03-14T09:02:20.766955+0000","last_peered":"2024-03-14T09:02:20.766955+0000","last_clean":"2024-03-14T09:02:20.766955+0000","last_became_active":"2024-03-14T09:02:03.329788+0000","last_became_peered":"2024-03-14T09:02:03.329788+0000","last_unstale":"2024-03-14T09:02:20.766955+0000","last_undegraded":"2024-03-14T09:02:20.766955+0000","last_fullsized":"2024-03-14T09:02:20.766955+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:06.270460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768651+0000","last_change":"2024-03-14T09:02:03.746864+0000","last_active":"2024-03-14T09:02:20.768651+0000","last_peered":"2024-03-14T09:02:20.768651+0000","last_clean":"2024-03-14T09:02:20.768651+0000","last_became_active":"2024-03-14T09:02:03.746405+0000","last_became_peered":"2024-03-14T09:02:03.746405+0000","last_unstale":"2024-03-14T09:02:20.768651+0000","last_undegraded":"2024-03-14T09:02:20.768651+0000","last_fullsized":"2024-03-14T09:02:20.768651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764546+0000","last_change":"2024-03-14T09:02:20.353880+0000","last_active":"2024-03-14T09:02:21.764546+0000","last_peered":"2024-03-14T09:02:21.764546+0000","last_clean":"2024-03-14T09:02:21.764546+0000","last_became_active":"2024-03-14T09:02:20.353617+0000","last_became_peered":"2024-03-14T09:02:20.353617+0000","last_unstale":"2024-03-14T09:02:21.764546+0000","last_undegraded":"2024-03-14T09:02:21.764546+0000","last_fullsized":"2024-03-14T09:02:21.764546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.027692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763874+0000","last_change":"2024-03-14T09:02:20.773271+0000","last_active":"2024-03-14T09:02:21.763874+0000","last_peered":"2024-03-14T09:02:21.763874+0000","last_clean":"2024-03-14T09:02:21.763874+0000","last_became_active":"2024-03-14T09:02:20.772993+0000","last_became_peered":"2024-03-14T09:02:20.772993+0000","last_unstale":"2024-03-14T09:02:21.763874+0000","last_undegraded":"2024-03-14T09:02:21.763874+0000","last_fullsized":"2024-03-14T09:02:21.763874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:59:06.064788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765232+0000","last_change":"2024-03-14T09:02:03.328451+0000","last_active":"2024-03-14T09:02:20.765232+0000","last_peered":"2024-03-14T09:02:20.765232+0000","last_clean":"2024-03-14T09:02:20.765232+0000","last_became_active":"2024-03-14T09:02:03.328184+0000","last_became_peered":"2024-03-14T09:02:03.328184+0000","last_unstale":"2024-03-14T09:02:20.765232+0000","last_undegraded":"2024-03-14T09:02:20.765232+0000","last_fullsized":"2024-03-14T09:02:20.765232+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766842+0000","last_change":"2024-03-14T09:02:03.743189+0000","last_active":"2024-03-14T09:02:20.766842+0000","last_peered":"2024-03-14T09:02:20.766842+0000","last_clean":"2024-03-14T09:02:20.766842+0000","last_became_active":"2024-03-14T09:02:03.742865+0000","last_became_peered":"2024-03-14T09:02:03.742865+0000","last_unstale":"2024-03-14T09:02:20.766842+0000","last_undegraded":"2024-03-14T09:02:20.766842+0000","last_fullsized":"2024-03-14T09:02:20.766842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938770+0000","last_change":"2024-03-14T09:02:20.360163+0000","last_active":"2024-03-14T09:02:21.938770+0000","last_peered":"2024-03-14T09:02:21.938770+0000","last_clean":"2024-03-14T09:02:21.938770+0000","last_became_active":"2024-03-14T09:02:20.359889+0000","last_became_peered":"2024-03-14T09:02:20.359889+0000","last_unstale":"2024-03-14T09:02:21.938770+0000","last_undegraded":"2024-03-14T09:02:21.938770+0000","last_fullsized":"2024-03-14T09:02:21.938770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:08.180878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763461+0000","last_change":"2024-03-14T09:02:20.778702+0000","last_active":"2024-03-14T09:02:21.763461+0000","last_peered":"2024-03-14T09:02:21.763461+0000","last_clean":"2024-03-14T09:02:21.763461+0000","last_became_active":"2024-03-14T09:02:20.778394+0000","last_became_peered":"2024-03-14T09:02:20.778394+0000","last_unstale":"2024-03-14T09:02:21.763461+0000","last_undegraded":"2024-03-14T09:02:21.763461+0000","last_fullsized":"2024-03-14T09:02:21.763461+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:46.738348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763940+0000","last_change":"2024-03-14T09:02:03.330619+0000","last_active":"2024-03-14T09:02:20.763940+0000","last_peered":"2024-03-14T09:02:20.763940+0000","last_clean":"2024-03-14T09:02:20.763940+0000","last_became_active":"2024-03-14T09:02:03.330237+0000","last_became_peered":"2024-03-14T09:02:03.330237+0000","last_unstale":"2024-03-14T09:02:20.763940+0000","last_undegraded":"2024-03-14T09:02:20.763940+0000","last_fullsized":"2024-03-14T09:02:20.763940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:49.751001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759749+0000","last_change":"2024-03-14T09:02:03.749453+0000","last_active":"2024-03-14T09:02:20.759749+0000","last_peered":"2024-03-14T09:02:20.759749+0000","last_clean":"2024-03-14T09:02:20.759749+0000","last_became_active":"2024-03-14T09:02:03.749186+0000","last_became_peered":"2024-03-14T09:02:03.749186+0000","last_unstale":"2024-03-14T09:02:20.759749+0000","last_undegraded":"2024-03-14T09:02:20.759749+0000","last_fullsized":"2024-03-14T09:02:20.759749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936590+0000","last_change":"2024-03-14T09:02:20.355286+0000","last_active":"2024-03-14T09:02:21.936590+0000","last_peered":"2024-03-14T09:02:21.936590+0000","last_clean":"2024-03-14T09:02:21.936590+0000","last_became_active":"2024-03-14T09:02:20.355110+0000","last_became_peered":"2024-03-14T09:02:20.355110+0000","last_unstale":"2024-03-14T09:02:21.936590+0000","last_undegraded":"2024-03-14T09:02:21.936590+0000","last_fullsized":"2024-03-14T09:02:21.936590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:53.714361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936438+0000","last_change":"2024-03-14T09:02:20.776834+0000","last_active":"2024-03-14T09:02:21.936438+0000","last_peered":"2024-03-14T09:02:21.936438+0000","last_clean":"2024-03-14T09:02:21.936438+0000","last_became_active":"2024-03-14T09:02:20.776322+0000","last_became_peered":"2024-03-14T09:02:20.776322+0000","last_unstale":"2024-03-14T09:02:21.936438+0000","last_undegraded":"2024-03-14T09:02:21.936438+0000","last_fullsized":"2024-03-14T09:02:21.936438+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:52.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764129+0000","last_change":"2024-03-14T09:02:03.335208+0000","last_active":"2024-03-14T09:02:20.764129+0000","last_peered":"2024-03-14T09:02:20.764129+0000","last_clean":"2024-03-14T09:02:20.764129+0000","last_became_active":"2024-03-14T09:02:03.334929+0000","last_became_peered":"2024-03-14T09:02:03.334929+0000","last_unstale":"2024-03-14T09:02:20.764129+0000","last_undegraded":"2024-03-14T09:02:20.764129+0000","last_fullsized":"2024-03-14T09:02:20.764129+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:03.880357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767394+0000","last_change":"2024-03-14T09:02:03.747948+0000","last_active":"2024-03-14T09:02:20.767394+0000","last_peered":"2024-03-14T09:02:20.767394+0000","last_clean":"2024-03-14T09:02:20.767394+0000","last_became_active":"2024-03-14T09:02:03.747471+0000","last_became_peered":"2024-03-14T09:02:03.747471+0000","last_unstale":"2024-03-14T09:02:20.767394+0000","last_undegraded":"2024-03-14T09:02:20.767394+0000","last_fullsized":"2024-03-14T09:02:20.767394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764495+0000","last_change":"2024-03-14T09:02:20.358769+0000","last_active":"2024-03-14T09:02:21.764495+0000","last_peered":"2024-03-14T09:02:21.764495+0000","last_clean":"2024-03-14T09:02:21.764495+0000","last_became_active":"2024-03-14T09:02:20.358536+0000","last_became_peered":"2024-03-14T09:02:20.358536+0000","last_unstale":"2024-03-14T09:02:21.764495+0000","last_undegraded":"2024-03-14T09:02:21.764495+0000","last_fullsized":"2024-03-14T09:02:21.764495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:32.013553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763435+0000","last_change":"2024-03-14T09:02:20.781046+0000","last_active":"2024-03-14T09:02:21.763435+0000","last_peered":"2024-03-14T09:02:21.763435+0000","last_clean":"2024-03-14T09:02:21.763435+0000","last_became_active":"2024-03-14T09:02:20.780783+0000","last_became_peered":"2024-03-14T09:02:20.780783+0000","last_unstale":"2024-03-14T09:02:21.763435+0000","last_undegraded":"2024-03-14T09:02:21.763435+0000","last_fullsized":"2024-03-14T09:02:21.763435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:36:27.894843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763651+0000","last_change":"2024-03-14T09:02:03.329056+0000","last_active":"2024-03-14T09:02:20.763651+0000","last_peered":"2024-03-14T09:02:20.763651+0000","last_clean":"2024-03-14T09:02:20.763651+0000","last_became_active":"2024-03-14T09:02:03.328772+0000","last_became_peered":"2024-03-14T09:02:03.328772+0000","last_unstale":"2024-03-14T09:02:20.763651+0000","last_undegraded":"2024-03-14T09:02:20.763651+0000","last_fullsized":"2024-03-14T09:02:20.763651+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:47.877730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766053+0000","last_change":"2024-03-14T09:02:03.751429+0000","last_active":"2024-03-14T09:02:20.766053+0000","last_peered":"2024-03-14T09:02:20.766053+0000","last_clean":"2024-03-14T09:02:20.766053+0000","last_became_active":"2024-03-14T09:02:03.751167+0000","last_became_peered":"2024-03-14T09:02:03.751167+0000","last_unstale":"2024-03-14T09:02:20.766053+0000","last_undegraded":"2024-03-14T09:02:20.766053+0000","last_fullsized":"2024-03-14T09:02:20.766053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:36:14.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"35'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936191+0000","last_change":"2024-03-14T09:02:20.358584+0000","last_active":"2024-03-14T09:02:21.936191+0000","last_peered":"2024-03-14T09:02:21.936191+0000","last_clean":"2024-03-14T09:02:21.936191+0000","last_became_active":"2024-03-14T09:02:20.358323+0000","last_became_peered":"2024-03-14T09:02:20.358323+0000","last_unstale":"2024-03-14T09:02:21.936191+0000","last_undegraded":"2024-03-14T09:02:21.936191+0000","last_fullsized":"2024-03-14T09:02:21.936191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:39:02.329849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937396+0000","last_change":"2024-03-14T09:02:20.777813+0000","last_active":"2024-03-14T09:02:21.937396+0000","last_peered":"2024-03-14T09:02:21.937396+0000","last_clean":"2024-03-14T09:02:21.937396+0000","last_became_active":"2024-03-14T09:02:20.775422+0000","last_became_peered":"2024-03-14T09:02:20.775422+0000","last_unstale":"2024-03-14T09:02:21.937396+0000","last_undegraded":"2024-03-14T09:02:21.937396+0000","last_fullsized":"2024-03-14T09:02:21.937396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:57:04.137876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766139+0000","last_change":"2024-03-14T09:02:03.329164+0000","last_active":"2024-03-14T09:02:20.766139+0000","last_peered":"2024-03-14T09:02:20.766139+0000","last_clean":"2024-03-14T09:02:20.766139+0000","last_became_active":"2024-03-14T09:02:03.328895+0000","last_became_peered":"2024-03-14T09:02:03.328895+0000","last_unstale":"2024-03-14T09:02:20.766139+0000","last_undegraded":"2024-03-14T09:02:20.766139+0000","last_fullsized":"2024-03-14T09:02:20.766139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763499+0000","last_change":"2024-03-14T09:02:03.740610+0000","last_active":"2024-03-14T09:02:20.763499+0000","last_peered":"2024-03-14T09:02:20.763499+0000","last_clean":"2024-03-14T09:02:20.763499+0000","last_became_active":"2024-03-14T09:02:03.740348+0000","last_became_peered":"2024-03-14T09:02:03.740348+0000","last_unstale":"2024-03-14T09:02:20.763499+0000","last_undegraded":"2024-03-14T09:02:20.763499+0000","last_fullsized":"2024-03-14T09:02:20.763499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:38.910337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763973+0000","last_change":"2024-03-14T09:02:20.359924+0000","last_active":"2024-03-14T09:02:21.763973+0000","last_peered":"2024-03-14T09:02:21.763973+0000","last_clean":"2024-03-14T09:02:21.763973+0000","last_became_active":"2024-03-14T09:02:20.358221+0000","last_became_peered":"2024-03-14T09:02:20.358221+0000","last_unstale":"2024-03-14T09:02:21.763973+0000","last_undegraded":"2024-03-14T09:02:21.763973+0000","last_fullsized":"2024-03-14T09:02:21.763973+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:34.196733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763855+0000","last_change":"2024-03-14T09:02:20.773647+0000","last_active":"2024-03-14T09:02:21.763855+0000","last_peered":"2024-03-14T09:02:21.763855+0000","last_clean":"2024-03-14T09:02:21.763855+0000","last_became_active":"2024-03-14T09:02:20.773223+0000","last_became_peered":"2024-03-14T09:02:20.773223+0000","last_unstale":"2024-03-14T09:02:21.763855+0000","last_undegraded":"2024-03-14T09:02:21.763855+0000","last_fullsized":"2024-03-14T09:02:21.763855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:04:06.449049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'9","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766593+0000","last_change":"2024-03-14T09:02:03.333899+0000","last_active":"2024-03-14T09:02:20.766593+0000","last_peered":"2024-03-14T09:02:20.766593+0000","last_clean":"2024-03-14T09:02:20.766593+0000","last_became_active":"2024-03-14T09:02:03.333685+0000","last_became_peered":"2024-03-14T09:02:03.333685+0000","last_unstale":"2024-03-14T09:02:20.766593+0000","last_undegraded":"2024-03-14T09:02:20.766593+0000","last_fullsized":"2024-03-14T09:02:20.766593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:05.246679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768264+0000","last_change":"2024-03-14T09:02:03.741618+0000","last_active":"2024-03-14T09:02:20.768264+0000","last_peered":"2024-03-14T09:02:20.768264+0000","last_clean":"2024-03-14T09:02:20.768264+0000","last_became_active":"2024-03-14T09:02:03.740972+0000","last_became_peered":"2024-03-14T09:02:03.740972+0000","last_unstale":"2024-03-14T09:02:20.768264+0000","last_undegraded":"2024-03-14T09:02:20.768264+0000","last_fullsized":"2024-03-14T09:02:20.768264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937355+0000","last_change":"2024-03-14T09:02:20.359395+0000","last_active":"2024-03-14T09:02:21.937355+0000","last_peered":"2024-03-14T09:02:21.937355+0000","last_clean":"2024-03-14T09:02:21.937355+0000","last_became_active":"2024-03-14T09:02:20.359126+0000","last_became_peered":"2024-03-14T09:02:20.359126+0000","last_unstale":"2024-03-14T09:02:21.937355+0000","last_undegraded":"2024-03-14T09:02:21.937355+0000","last_fullsized":"2024-03-14T09:02:21.937355+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:38:51.483975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937871+0000","last_change":"2024-03-14T09:02:20.778281+0000","last_active":"2024-03-14T09:02:21.937871+0000","last_peered":"2024-03-14T09:02:21.937871+0000","last_clean":"2024-03-14T09:02:21.937871+0000","last_became_active":"2024-03-14T09:02:20.778022+0000","last_became_peered":"2024-03-14T09:02:20.778022+0000","last_unstale":"2024-03-14T09:02:21.937871+0000","last_undegraded":"2024-03-14T09:02:21.937871+0000","last_fullsized":"2024-03-14T09:02:21.937871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:00.389998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768351+0000","last_change":"2024-03-14T09:02:03.335856+0000","last_active":"2024-03-14T09:02:20.768351+0000","last_peered":"2024-03-14T09:02:20.768351+0000","last_clean":"2024-03-14T09:02:20.768351+0000","last_became_active":"2024-03-14T09:02:03.335559+0000","last_became_peered":"2024-03-14T09:02:03.335559+0000","last_unstale":"2024-03-14T09:02:20.768351+0000","last_undegraded":"2024-03-14T09:02:20.768351+0000","last_fullsized":"2024-03-14T09:02:20.768351+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:14.041430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766674+0000","last_change":"2024-03-14T09:02:03.748711+0000","last_active":"2024-03-14T09:02:20.766674+0000","last_peered":"2024-03-14T09:02:20.766674+0000","last_clean":"2024-03-14T09:02:20.766674+0000","last_became_active":"2024-03-14T09:02:03.748483+0000","last_became_peered":"2024-03-14T09:02:03.748483+0000","last_unstale":"2024-03-14T09:02:20.766674+0000","last_undegraded":"2024-03-14T09:02:20.766674+0000","last_fullsized":"2024-03-14T09:02:20.766674+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:43.215729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937350+0000","last_change":"2024-03-14T09:02:20.358762+0000","last_active":"2024-03-14T09:02:21.937350+0000","last_peered":"2024-03-14T09:02:21.937350+0000","last_clean":"2024-03-14T09:02:21.937350+0000","last_became_active":"2024-03-14T09:02:20.358405+0000","last_became_peered":"2024-03-14T09:02:20.358405+0000","last_unstale":"2024-03-14T09:02:21.937350+0000","last_undegraded":"2024-03-14T09:02:21.937350+0000","last_fullsized":"2024-03-14T09:02:21.937350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:35.412558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937568+0000","last_change":"2024-03-14T09:02:20.773160+0000","last_active":"2024-03-14T09:02:21.937568+0000","last_peered":"2024-03-14T09:02:21.937568+0000","last_clean":"2024-03-14T09:02:21.937568+0000","last_became_active":"2024-03-14T09:02:20.772180+0000","last_became_peered":"2024-03-14T09:02:20.772180+0000","last_unstale":"2024-03-14T09:02:21.937568+0000","last_undegraded":"2024-03-14T09:02:21.937568+0000","last_fullsized":"2024-03-14T09:02:21.937568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:43.032823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761610+0000","last_change":"2024-03-14T09:02:03.331300+0000","last_active":"2024-03-14T09:02:20.761610+0000","last_peered":"2024-03-14T09:02:20.761610+0000","last_clean":"2024-03-14T09:02:20.761610+0000","last_became_active":"2024-03-14T09:02:03.331033+0000","last_became_peered":"2024-03-14T09:02:03.331033+0000","last_unstale":"2024-03-14T09:02:20.761610+0000","last_undegraded":"2024-03-14T09:02:20.761610+0000","last_fullsized":"2024-03-14T09:02:20.761610+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:08.983196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765781+0000","last_change":"2024-03-14T09:02:03.758670+0000","last_active":"2024-03-14T09:02:20.765781+0000","last_peered":"2024-03-14T09:02:20.765781+0000","last_clean":"2024-03-14T09:02:20.765781+0000","last_became_active":"2024-03-14T09:02:03.758426+0000","last_became_peered":"2024-03-14T09:02:03.758426+0000","last_unstale":"2024-03-14T09:02:20.765781+0000","last_undegraded":"2024-03-14T09:02:20.765781+0000","last_fullsized":"2024-03-14T09:02:20.765781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:37:44.347728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"35'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758655+0000","last_change":"2024-03-14T09:02:20.350124+0000","last_active":"2024-03-14T09:02:21.758655+0000","last_peered":"2024-03-14T09:02:21.758655+0000","last_clean":"2024-03-14T09:02:21.758655+0000","last_became_active":"2024-03-14T09:02:20.349815+0000","last_became_peered":"2024-03-14T09:02:20.349815+0000","last_unstale":"2024-03-14T09:02:21.758655+0000","last_undegraded":"2024-03-14T09:02:21.758655+0000","last_fullsized":"2024-03-14T09:02:21.758655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:04:08.179245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936817+0000","last_change":"2024-03-14T09:02:20.781627+0000","last_active":"2024-03-14T09:02:21.936817+0000","last_peered":"2024-03-14T09:02:21.936817+0000","last_clean":"2024-03-14T09:02:21.936817+0000","last_became_active":"2024-03-14T09:02:20.781053+0000","last_became_peered":"2024-03-14T09:02:20.781053+0000","last_unstale":"2024-03-14T09:02:21.936817+0000","last_undegraded":"2024-03-14T09:02:21.936817+0000","last_fullsized":"2024-03-14T09:02:21.936817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:17:16.470852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765594+0000","last_change":"2024-03-14T09:02:03.335707+0000","last_active":"2024-03-14T09:02:20.765594+0000","last_peered":"2024-03-14T09:02:20.765594+0000","last_clean":"2024-03-14T09:02:20.765594+0000","last_became_active":"2024-03-14T09:02:03.335448+0000","last_became_peered":"2024-03-14T09:02:03.335448+0000","last_unstale":"2024-03-14T09:02:20.765594+0000","last_undegraded":"2024-03-14T09:02:20.765594+0000","last_fullsized":"2024-03-14T09:02:20.765594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:10.980082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.771658+0000","last_change":"2024-03-14T09:02:03.757103+0000","last_active":"2024-03-14T09:02:20.771658+0000","last_peered":"2024-03-14T09:02:20.771658+0000","last_clean":"2024-03-14T09:02:20.771658+0000","last_became_active":"2024-03-14T09:02:03.756572+0000","last_became_peered":"2024-03-14T09:02:03.756572+0000","last_unstale":"2024-03-14T09:02:20.771658+0000","last_undegraded":"2024-03-14T09:02:20.771658+0000","last_fullsized":"2024-03-14T09:02:20.771658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937732+0000","last_change":"2024-03-14T09:02:20.361261+0000","last_active":"2024-03-14T09:02:21.937732+0000","last_peered":"2024-03-14T09:02:21.937732+0000","last_clean":"2024-03-14T09:02:21.937732+0000","last_became_active":"2024-03-14T09:02:20.360979+0000","last_became_peered":"2024-03-14T09:02:20.360979+0000","last_unstale":"2024-03-14T09:02:21.937732+0000","last_undegraded":"2024-03-14T09:02:21.937732+0000","last_fullsized":"2024-03-14T09:02:21.937732+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:42.027692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764350+0000","last_change":"2024-03-14T09:02:20.784002+0000","last_active":"2024-03-14T09:02:21.764350+0000","last_peered":"2024-03-14T09:02:21.764350+0000","last_clean":"2024-03-14T09:02:21.764350+0000","last_became_active":"2024-03-14T09:02:20.783551+0000","last_became_peered":"2024-03-14T09:02:20.783551+0000","last_unstale":"2024-03-14T09:02:21.764350+0000","last_undegraded":"2024-03-14T09:02:21.764350+0000","last_fullsized":"2024-03-14T09:02:21.764350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:14.007036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764576+0000","last_change":"2024-03-14T09:02:03.330919+0000","last_active":"2024-03-14T09:02:20.764576+0000","last_peered":"2024-03-14T09:02:20.764576+0000","last_clean":"2024-03-14T09:02:20.764576+0000","last_became_active":"2024-03-14T09:02:03.330613+0000","last_became_peered":"2024-03-14T09:02:03.330613+0000","last_unstale":"2024-03-14T09:02:20.764576+0000","last_undegraded":"2024-03-14T09:02:20.764576+0000","last_fullsized":"2024-03-14T09:02:20.764576+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:06:37.360334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.762617+0000","last_change":"2024-03-14T09:02:03.740713+0000","last_active":"2024-03-14T09:02:20.762617+0000","last_peered":"2024-03-14T09:02:20.762617+0000","last_clean":"2024-03-14T09:02:20.762617+0000","last_became_active":"2024-03-14T09:02:03.740458+0000","last_became_peered":"2024-03-14T09:02:03.740458+0000","last_unstale":"2024-03-14T09:02:20.762617+0000","last_undegraded":"2024-03-14T09:02:20.762617+0000","last_fullsized":"2024-03-14T09:02:20.762617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939323+0000","last_change":"2024-03-14T09:02:20.358973+0000","last_active":"2024-03-14T09:02:21.939323+0000","last_peered":"2024-03-14T09:02:21.939323+0000","last_clean":"2024-03-14T09:02:21.939323+0000","last_became_active":"2024-03-14T09:02:20.358681+0000","last_became_peered":"2024-03-14T09:02:20.358681+0000","last_unstale":"2024-03-14T09:02:21.939323+0000","last_undegraded":"2024-03-14T09:02:21.939323+0000","last_fullsized":"2024-03-14T09:02:21.939323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:43:04.366071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936343+0000","last_change":"2024-03-14T09:02:20.776985+0000","last_active":"2024-03-14T09:02:21.936343+0000","last_peered":"2024-03-14T09:02:21.936343+0000","last_clean":"2024-03-14T09:02:21.936343+0000","last_became_active":"2024-03-14T09:02:20.776630+0000","last_became_peered":"2024-03-14T09:02:20.776630+0000","last_unstale":"2024-03-14T09:02:21.936343+0000","last_undegraded":"2024-03-14T09:02:21.936343+0000","last_fullsized":"2024-03-14T09:02:21.936343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:54.719492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.763983+0000","last_change":"2024-03-14T09:02:03.327627+0000","last_active":"2024-03-14T09:02:20.763983+0000","last_peered":"2024-03-14T09:02:20.763983+0000","last_clean":"2024-03-14T09:02:20.763983+0000","last_became_active":"2024-03-14T09:02:03.327348+0000","last_became_peered":"2024-03-14T09:02:03.327348+0000","last_unstale":"2024-03-14T09:02:20.763983+0000","last_undegraded":"2024-03-14T09:02:20.763983+0000","last_fullsized":"2024-03-14T09:02:20.763983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:37.905318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764657+0000","last_change":"2024-03-14T09:02:03.743458+0000","last_active":"2024-03-14T09:02:20.764657+0000","last_peered":"2024-03-14T09:02:20.764657+0000","last_clean":"2024-03-14T09:02:20.764657+0000","last_became_active":"2024-03-14T09:02:03.742694+0000","last_became_peered":"2024-03-14T09:02:03.742694+0000","last_unstale":"2024-03-14T09:02:20.764657+0000","last_undegraded":"2024-03-14T09:02:20.764657+0000","last_fullsized":"2024-03-14T09:02:20.764657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:59:04.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764183+0000","last_change":"2024-03-14T09:02:20.359756+0000","last_active":"2024-03-14T09:02:21.764183+0000","last_peered":"2024-03-14T09:02:21.764183+0000","last_clean":"2024-03-14T09:02:21.764183+0000","last_became_active":"2024-03-14T09:02:20.359362+0000","last_became_peered":"2024-03-14T09:02:20.359362+0000","last_unstale":"2024-03-14T09:02:21.764183+0000","last_undegraded":"2024-03-14T09:02:21.764183+0000","last_fullsized":"2024-03-14T09:02:21.764183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:38:17.194878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939056+0000","last_change":"2024-03-14T09:02:20.770488+0000","last_active":"2024-03-14T09:02:21.939056+0000","last_peered":"2024-03-14T09:02:21.939056+0000","last_clean":"2024-03-14T09:02:21.939056+0000","last_became_active":"2024-03-14T09:02:20.770177+0000","last_became_peered":"2024-03-14T09:02:20.770177+0000","last_unstale":"2024-03-14T09:02:21.939056+0000","last_undegraded":"2024-03-14T09:02:21.939056+0000","last_fullsized":"2024-03-14T09:02:21.939056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767417+0000","last_change":"2024-03-14T09:02:03.330899+0000","last_active":"2024-03-14T09:02:20.767417+0000","last_peered":"2024-03-14T09:02:20.767417+0000","last_clean":"2024-03-14T09:02:20.767417+0000","last_became_active":"2024-03-14T09:02:03.330649+0000","last_became_peered":"2024-03-14T09:02:03.330649+0000","last_unstale":"2024-03-14T09:02:20.767417+0000","last_undegraded":"2024-03-14T09:02:20.767417+0000","last_fullsized":"2024-03-14T09:02:20.767417+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:54:47.779166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.769282+0000","last_change":"2024-03-14T09:02:03.738226+0000","last_active":"2024-03-14T09:02:20.769282+0000","last_peered":"2024-03-14T09:02:20.769282+0000","last_clean":"2024-03-14T09:02:20.769282+0000","last_became_active":"2024-03-14T09:02:03.737960+0000","last_became_peered":"2024-03-14T09:02:03.737960+0000","last_unstale":"2024-03-14T09:02:20.769282+0000","last_undegraded":"2024-03-14T09:02:20.769282+0000","last_fullsized":"2024-03-14T09:02:20.769282+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936337+0000","last_change":"2024-03-14T09:02:20.355279+0000","last_active":"2024-03-14T09:02:21.936337+0000","last_peered":"2024-03-14T09:02:21.936337+0000","last_clean":"2024-03-14T09:02:21.936337+0000","last_became_active":"2024-03-14T09:02:20.355030+0000","last_became_peered":"2024-03-14T09:02:20.355030+0000","last_unstale":"2024-03-14T09:02:21.936337+0000","last_undegraded":"2024-03-14T09:02:21.936337+0000","last_fullsized":"2024-03-14T09:02:21.936337+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:05.269571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763053+0000","last_change":"2024-03-14T09:02:20.771327+0000","last_active":"2024-03-14T09:02:21.763053+0000","last_peered":"2024-03-14T09:02:21.763053+0000","last_clean":"2024-03-14T09:02:21.763053+0000","last_became_active":"2024-03-14T09:02:20.770890+0000","last_became_peered":"2024-03-14T09:02:20.770890+0000","last_unstale":"2024-03-14T09:02:21.763053+0000","last_undegraded":"2024-03-14T09:02:21.763053+0000","last_fullsized":"2024-03-14T09:02:21.763053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:19:07.904413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766027+0000","last_change":"2024-03-14T09:02:03.331030+0000","last_active":"2024-03-14T09:02:20.766027+0000","last_peered":"2024-03-14T09:02:20.766027+0000","last_clean":"2024-03-14T09:02:20.766027+0000","last_became_active":"2024-03-14T09:02:03.330772+0000","last_became_peered":"2024-03-14T09:02:03.330772+0000","last_unstale":"2024-03-14T09:02:20.766027+0000","last_undegraded":"2024-03-14T09:02:20.766027+0000","last_fullsized":"2024-03-14T09:02:20.766027+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:46:37.654603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766207+0000","last_change":"2024-03-14T09:02:03.747325+0000","last_active":"2024-03-14T09:02:20.766207+0000","last_peered":"2024-03-14T09:02:20.766207+0000","last_clean":"2024-03-14T09:02:20.766207+0000","last_became_active":"2024-03-14T09:02:03.747029+0000","last_became_peered":"2024-03-14T09:02:03.747029+0000","last_unstale":"2024-03-14T09:02:20.766207+0000","last_undegraded":"2024-03-14T09:02:20.766207+0000","last_fullsized":"2024-03-14T09:02:20.766207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:22.133444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938315+0000","last_change":"2024-03-14T09:02:20.360468+0000","last_active":"2024-03-14T09:02:21.938315+0000","last_peered":"2024-03-14T09:02:21.938315+0000","last_clean":"2024-03-14T09:02:21.938315+0000","last_became_active":"2024-03-14T09:02:20.360169+0000","last_became_peered":"2024-03-14T09:02:20.360169+0000","last_unstale":"2024-03-14T09:02:21.938315+0000","last_undegraded":"2024-03-14T09:02:21.938315+0000","last_fullsized":"2024-03-14T09:02:21.938315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:36:26.889712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936426+0000","last_change":"2024-03-14T09:02:20.778720+0000","last_active":"2024-03-14T09:02:21.936426+0000","last_peered":"2024-03-14T09:02:21.936426+0000","last_clean":"2024-03-14T09:02:21.936426+0000","last_became_active":"2024-03-14T09:02:20.778481+0000","last_became_peered":"2024-03-14T09:02:20.778481+0000","last_unstale":"2024-03-14T09:02:21.936426+0000","last_undegraded":"2024-03-14T09:02:21.936426+0000","last_fullsized":"2024-03-14T09:02:21.936426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:38:18.200010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767724+0000","last_change":"2024-03-14T09:02:03.331128+0000","last_active":"2024-03-14T09:02:20.767724+0000","last_peered":"2024-03-14T09:02:20.767724+0000","last_clean":"2024-03-14T09:02:20.767724+0000","last_became_active":"2024-03-14T09:02:03.330898+0000","last_became_peered":"2024-03-14T09:02:03.330898+0000","last_unstale":"2024-03-14T09:02:20.767724+0000","last_undegraded":"2024-03-14T09:02:20.767724+0000","last_fullsized":"2024-03-14T09:02:20.767724+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:23:09.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765250+0000","last_change":"2024-03-14T09:02:03.757204+0000","last_active":"2024-03-14T09:02:20.765250+0000","last_peered":"2024-03-14T09:02:20.765250+0000","last_clean":"2024-03-14T09:02:20.765250+0000","last_became_active":"2024-03-14T09:02:03.752933+0000","last_became_peered":"2024-03-14T09:02:03.752933+0000","last_unstale":"2024-03-14T09:02:20.765250+0000","last_undegraded":"2024-03-14T09:02:20.765250+0000","last_fullsized":"2024-03-14T09:02:20.765250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:07:32.265470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937274+0000","last_change":"2024-03-14T09:02:20.360708+0000","last_active":"2024-03-14T09:02:21.937274+0000","last_peered":"2024-03-14T09:02:21.937274+0000","last_clean":"2024-03-14T09:02:21.937274+0000","last_became_active":"2024-03-14T09:02:20.360433+0000","last_became_peered":"2024-03-14T09:02:20.360433+0000","last_unstale":"2024-03-14T09:02:21.937274+0000","last_undegraded":"2024-03-14T09:02:21.937274+0000","last_fullsized":"2024-03-14T09:02:21.937274+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:47.306616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763604+0000","last_change":"2024-03-14T09:02:20.769767+0000","last_active":"2024-03-14T09:02:21.763604+0000","last_peered":"2024-03-14T09:02:21.763604+0000","last_clean":"2024-03-14T09:02:21.763604+0000","last_became_active":"2024-03-14T09:02:20.769481+0000","last_became_peered":"2024-03-14T09:02:20.769481+0000","last_unstale":"2024-03-14T09:02:21.763604+0000","last_undegraded":"2024-03-14T09:02:21.763604+0000","last_fullsized":"2024-03-14T09:02:21.763604+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:43:05.371202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765573+0000","last_change":"2024-03-14T09:02:03.331945+0000","last_active":"2024-03-14T09:02:20.765573+0000","last_peered":"2024-03-14T09:02:20.765573+0000","last_clean":"2024-03-14T09:02:20.765573+0000","last_became_active":"2024-03-14T09:02:03.331686+0000","last_became_peered":"2024-03-14T09:02:03.331686+0000","last_unstale":"2024-03-14T09:02:20.765573+0000","last_undegraded":"2024-03-14T09:02:20.765573+0000","last_fullsized":"2024-03-14T09:02:20.765573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:13.638496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766160+0000","last_change":"2024-03-14T09:02:03.750990+0000","last_active":"2024-03-14T09:02:20.766160+0000","last_peered":"2024-03-14T09:02:20.766160+0000","last_clean":"2024-03-14T09:02:20.766160+0000","last_became_active":"2024-03-14T09:02:03.750742+0000","last_became_peered":"2024-03-14T09:02:03.750742+0000","last_unstale":"2024-03-14T09:02:20.766160+0000","last_undegraded":"2024-03-14T09:02:20.766160+0000","last_fullsized":"2024-03-14T09:02:20.766160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764639+0000","last_change":"2024-03-14T09:02:20.360529+0000","last_active":"2024-03-14T09:02:21.764639+0000","last_peered":"2024-03-14T09:02:21.764639+0000","last_clean":"2024-03-14T09:02:21.764639+0000","last_became_active":"2024-03-14T09:02:20.360276+0000","last_became_peered":"2024-03-14T09:02:20.360276+0000","last_unstale":"2024-03-14T09:02:21.764639+0000","last_undegraded":"2024-03-14T09:02:21.764639+0000","last_fullsized":"2024-03-14T09:02:21.764639+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:42.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936629+0000","last_change":"2024-03-14T09:02:20.778652+0000","last_active":"2024-03-14T09:02:21.936629+0000","last_peered":"2024-03-14T09:02:21.936629+0000","last_clean":"2024-03-14T09:02:21.936629+0000","last_became_active":"2024-03-14T09:02:20.778405+0000","last_became_peered":"2024-03-14T09:02:20.778405+0000","last_unstale":"2024-03-14T09:02:21.936629+0000","last_undegraded":"2024-03-14T09:02:21.936629+0000","last_fullsized":"2024-03-14T09:02:21.936629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:20.338489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761692+0000","last_change":"2024-03-14T09:02:03.335358+0000","last_active":"2024-03-14T09:02:20.761692+0000","last_peered":"2024-03-14T09:02:20.761692+0000","last_clean":"2024-03-14T09:02:20.761692+0000","last_became_active":"2024-03-14T09:02:03.335031+0000","last_became_peered":"2024-03-14T09:02:03.335031+0000","last_unstale":"2024-03-14T09:02:20.761692+0000","last_undegraded":"2024-03-14T09:02:20.761692+0000","last_fullsized":"2024-03-14T09:02:20.761692+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:39:20.990059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759873+0000","last_change":"2024-03-14T09:02:03.750291+0000","last_active":"2024-03-14T09:02:20.759873+0000","last_peered":"2024-03-14T09:02:20.759873+0000","last_clean":"2024-03-14T09:02:20.759873+0000","last_became_active":"2024-03-14T09:02:03.749738+0000","last_became_peered":"2024-03-14T09:02:03.749738+0000","last_unstale":"2024-03-14T09:02:20.759873+0000","last_undegraded":"2024-03-14T09:02:20.759873+0000","last_fullsized":"2024-03-14T09:02:20.759873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938293+0000","last_change":"2024-03-14T09:02:20.358703+0000","last_active":"2024-03-14T09:02:21.938293+0000","last_peered":"2024-03-14T09:02:21.938293+0000","last_clean":"2024-03-14T09:02:21.938293+0000","last_became_active":"2024-03-14T09:02:20.358330+0000","last_became_peered":"2024-03-14T09:02:20.358330+0000","last_unstale":"2024-03-14T09:02:21.938293+0000","last_undegraded":"2024-03-14T09:02:21.938293+0000","last_fullsized":"2024-03-14T09:02:21.938293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:28:00.991900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936390+0000","last_change":"2024-03-14T09:02:20.769652+0000","last_active":"2024-03-14T09:02:21.936390+0000","last_peered":"2024-03-14T09:02:21.936390+0000","last_clean":"2024-03-14T09:02:21.936390+0000","last_became_active":"2024-03-14T09:02:20.769412+0000","last_became_peered":"2024-03-14T09:02:20.769412+0000","last_unstale":"2024-03-14T09:02:21.936390+0000","last_undegraded":"2024-03-14T09:02:21.936390+0000","last_fullsized":"2024-03-14T09:02:21.936390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:12:16.436058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759792+0000","last_change":"2024-03-14T09:02:03.331160+0000","last_active":"2024-03-14T09:02:20.759792+0000","last_peered":"2024-03-14T09:02:20.759792+0000","last_clean":"2024-03-14T09:02:20.759792+0000","last_became_active":"2024-03-14T09:02:03.330643+0000","last_became_peered":"2024-03-14T09:02:03.330643+0000","last_unstale":"2024-03-14T09:02:20.759792+0000","last_undegraded":"2024-03-14T09:02:20.759792+0000","last_fullsized":"2024-03-14T09:02:20.759792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:02.306938+0000","last_clean_scrub_stamp":"2024-03-14T09:02:02.306938+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:10:50.531686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.761770+0000","last_change":"2024-03-14T09:02:03.744443+0000","last_active":"2024-03-14T09:02:20.761770+0000","last_peered":"2024-03-14T09:02:20.761770+0000","last_clean":"2024-03-14T09:02:20.761770+0000","last_became_active":"2024-03-14T09:02:03.744149+0000","last_became_peered":"2024-03-14T09:02:03.744149+0000","last_unstale":"2024-03-14T09:02:20.761770+0000","last_undegraded":"2024-03-14T09:02:20.761770+0000","last_fullsized":"2024-03-14T09:02:20.761770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:06.251698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936613+0000","last_change":"2024-03-14T09:02:20.360511+0000","last_active":"2024-03-14T09:02:21.936613+0000","last_peered":"2024-03-14T09:02:21.936613+0000","last_clean":"2024-03-14T09:02:21.936613+0000","last_became_active":"2024-03-14T09:02:20.360257+0000","last_became_peered":"2024-03-14T09:02:20.360257+0000","last_unstale":"2024-03-14T09:02:21.936613+0000","last_undegraded":"2024-03-14T09:02:21.936613+0000","last_fullsized":"2024-03-14T09:02:21.936613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:19.331704+0000","last_clean_scrub_stamp":"2024-03-14T09:02:19.331704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:40.867971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939059+0000","last_change":"2024-03-14T09:02:20.779931+0000","last_active":"2024-03-14T09:02:21.939059+0000","last_peered":"2024-03-14T09:02:21.939059+0000","last_clean":"2024-03-14T09:02:21.939059+0000","last_became_active":"2024-03-14T09:02:20.779669+0000","last_became_peered":"2024-03-14T09:02:20.779669+0000","last_unstale":"2024-03-14T09:02:21.939059+0000","last_undegraded":"2024-03-14T09:02:21.939059+0000","last_fullsized":"2024-03-14T09:02:21.939059+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767479+0000","last_change":"2024-03-14T09:02:03.747713+0000","last_active":"2024-03-14T09:02:20.767479+0000","last_peered":"2024-03-14T09:02:20.767479+0000","last_clean":"2024-03-14T09:02:20.767479+0000","last_became_active":"2024-03-14T09:02:03.747113+0000","last_became_peered":"2024-03-14T09:02:03.747113+0000","last_unstale":"2024-03-14T09:02:20.767479+0000","last_undegraded":"2024-03-14T09:02:20.767479+0000","last_fullsized":"2024-03-14T09:02:20.767479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.939001+0000","last_change":"2024-03-14T09:02:20.773464+0000","last_active":"2024-03-14T09:02:21.939001+0000","last_peered":"2024-03-14T09:02:21.939001+0000","last_clean":"2024-03-14T09:02:21.939001+0000","last_became_active":"2024-03-14T09:02:20.773138+0000","last_became_peered":"2024-03-14T09:02:20.773138+0000","last_unstale":"2024-03-14T09:02:21.939001+0000","last_undegraded":"2024-03-14T09:02:21.939001+0000","last_fullsized":"2024-03-14T09:02:21.939001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:07:43.563619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759647+0000","last_change":"2024-03-14T09:02:03.754647+0000","last_active":"2024-03-14T09:02:20.759647+0000","last_peered":"2024-03-14T09:02:20.759647+0000","last_clean":"2024-03-14T09:02:20.759647+0000","last_became_active":"2024-03-14T09:02:03.750909+0000","last_became_peered":"2024-03-14T09:02:03.750909+0000","last_unstale":"2024-03-14T09:02:20.759647+0000","last_undegraded":"2024-03-14T09:02:20.759647+0000","last_fullsized":"2024-03-14T09:02:20.759647+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:25:50.756020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763173+0000","last_change":"2024-03-14T09:02:20.771229+0000","last_active":"2024-03-14T09:02:21.763173+0000","last_peered":"2024-03-14T09:02:21.763173+0000","last_clean":"2024-03-14T09:02:21.763173+0000","last_became_active":"2024-03-14T09:02:20.770913+0000","last_became_peered":"2024-03-14T09:02:20.770913+0000","last_unstale":"2024-03-14T09:02:21.763173+0000","last_undegraded":"2024-03-14T09:02:21.763173+0000","last_fullsized":"2024-03-14T09:02:21.763173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:31:49.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766290+0000","last_change":"2024-03-14T09:02:03.754829+0000","last_active":"2024-03-14T09:02:20.766290+0000","last_peered":"2024-03-14T09:02:20.766290+0000","last_clean":"2024-03-14T09:02:20.766290+0000","last_became_active":"2024-03-14T09:02:03.754553+0000","last_became_peered":"2024-03-14T09:02:03.754553+0000","last_unstale":"2024-03-14T09:02:20.766290+0000","last_undegraded":"2024-03-14T09:02:20.766290+0000","last_fullsized":"2024-03-14T09:02:20.766290+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758117+0000","last_change":"2024-03-14T09:02:20.783186+0000","last_active":"2024-03-14T09:02:21.758117+0000","last_peered":"2024-03-14T09:02:21.758117+0000","last_clean":"2024-03-14T09:02:21.758117+0000","last_became_active":"2024-03-14T09:02:20.782885+0000","last_became_peered":"2024-03-14T09:02:20.782885+0000","last_unstale":"2024-03-14T09:02:21.758117+0000","last_undegraded":"2024-03-14T09:02:21.758117+0000","last_fullsized":"2024-03-14T09:02:21.758117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:50:40.300461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765885+0000","last_change":"2024-03-14T09:02:03.751534+0000","last_active":"2024-03-14T09:02:20.765885+0000","last_peered":"2024-03-14T09:02:20.765885+0000","last_clean":"2024-03-14T09:02:20.765885+0000","last_became_active":"2024-03-14T09:02:03.751265+0000","last_became_peered":"2024-03-14T09:02:03.751265+0000","last_unstale":"2024-03-14T09:02:20.765885+0000","last_undegraded":"2024-03-14T09:02:20.765885+0000","last_fullsized":"2024-03-14T09:02:20.765885+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764144+0000","last_change":"2024-03-14T09:02:20.781982+0000","last_active":"2024-03-14T09:02:21.764144+0000","last_peered":"2024-03-14T09:02:21.764144+0000","last_clean":"2024-03-14T09:02:21.764144+0000","last_became_active":"2024-03-14T09:02:20.781688+0000","last_became_peered":"2024-03-14T09:02:20.781688+0000","last_unstale":"2024-03-14T09:02:21.764144+0000","last_undegraded":"2024-03-14T09:02:21.764144+0000","last_fullsized":"2024-03-14T09:02:21.764144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:42.685345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758533+0000","last_change":"2024-03-14T09:02:20.783091+0000","last_active":"2024-03-14T09:02:21.758533+0000","last_peered":"2024-03-14T09:02:21.758533+0000","last_clean":"2024-03-14T09:02:21.758533+0000","last_became_active":"2024-03-14T09:02:20.782795+0000","last_became_peered":"2024-03-14T09:02:20.782795+0000","last_unstale":"2024-03-14T09:02:21.758533+0000","last_undegraded":"2024-03-14T09:02:21.758533+0000","last_fullsized":"2024-03-14T09:02:21.758533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:56:11.975080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768271+0000","last_change":"2024-03-14T09:02:03.737708+0000","last_active":"2024-03-14T09:02:20.768271+0000","last_peered":"2024-03-14T09:02:20.768271+0000","last_clean":"2024-03-14T09:02:20.768271+0000","last_became_active":"2024-03-14T09:02:03.737433+0000","last_became_peered":"2024-03-14T09:02:03.737433+0000","last_unstale":"2024-03-14T09:02:20.768271+0000","last_undegraded":"2024-03-14T09:02:20.768271+0000","last_fullsized":"2024-03-14T09:02:20.768271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763489+0000","last_change":"2024-03-14T09:02:20.764694+0000","last_active":"2024-03-14T09:02:21.763489+0000","last_peered":"2024-03-14T09:02:21.763489+0000","last_clean":"2024-03-14T09:02:21.763489+0000","last_became_active":"2024-03-14T09:02:20.764405+0000","last_became_peered":"2024-03-14T09:02:20.764405+0000","last_unstale":"2024-03-14T09:02:21.763489+0000","last_undegraded":"2024-03-14T09:02:21.763489+0000","last_fullsized":"2024-03-14T09:02:21.763489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:32:40.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765323+0000","last_change":"2024-03-14T09:02:03.757244+0000","last_active":"2024-03-14T09:02:20.765323+0000","last_peered":"2024-03-14T09:02:20.765323+0000","last_clean":"2024-03-14T09:02:20.765323+0000","last_became_active":"2024-03-14T09:02:03.753248+0000","last_became_peered":"2024-03-14T09:02:03.753248+0000","last_unstale":"2024-03-14T09:02:20.765323+0000","last_undegraded":"2024-03-14T09:02:20.765323+0000","last_fullsized":"2024-03-14T09:02:20.765323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:19:11.136247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766862+0000","last_change":"2024-03-14T09:02:03.753751+0000","last_active":"2024-03-14T09:02:20.766862+0000","last_peered":"2024-03-14T09:02:20.766862+0000","last_clean":"2024-03-14T09:02:20.766862+0000","last_became_active":"2024-03-14T09:02:03.753485+0000","last_became_peered":"2024-03-14T09:02:03.753485+0000","last_unstale":"2024-03-14T09:02:20.766862+0000","last_undegraded":"2024-03-14T09:02:20.766862+0000","last_fullsized":"2024-03-14T09:02:20.766862+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:55:54.950201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938630+0000","last_change":"2024-03-14T09:02:20.778873+0000","last_active":"2024-03-14T09:02:21.938630+0000","last_peered":"2024-03-14T09:02:21.938630+0000","last_clean":"2024-03-14T09:02:21.938630+0000","last_became_active":"2024-03-14T09:02:20.778235+0000","last_became_peered":"2024-03-14T09:02:20.778235+0000","last_unstale":"2024-03-14T09:02:21.938630+0000","last_undegraded":"2024-03-14T09:02:21.938630+0000","last_fullsized":"2024-03-14T09:02:21.938630+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936869+0000","last_change":"2024-03-14T09:02:20.774761+0000","last_active":"2024-03-14T09:02:21.936869+0000","last_peered":"2024-03-14T09:02:21.936869+0000","last_clean":"2024-03-14T09:02:21.936869+0000","last_became_active":"2024-03-14T09:02:20.774462+0000","last_became_peered":"2024-03-14T09:02:20.774462+0000","last_unstale":"2024-03-14T09:02:21.936869+0000","last_undegraded":"2024-03-14T09:02:21.936869+0000","last_fullsized":"2024-03-14T09:02:21.936869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:22:54.719492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765449+0000","last_change":"2024-03-14T09:02:03.746056+0000","last_active":"2024-03-14T09:02:20.765449+0000","last_peered":"2024-03-14T09:02:20.765449+0000","last_clean":"2024-03-14T09:02:20.765449+0000","last_became_active":"2024-03-14T09:02:03.745200+0000","last_became_peered":"2024-03-14T09:02:03.745200+0000","last_unstale":"2024-03-14T09:02:20.765449+0000","last_undegraded":"2024-03-14T09:02:20.765449+0000","last_fullsized":"2024-03-14T09:02:20.765449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:41:39.423076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937137+0000","last_change":"2024-03-14T09:02:20.774448+0000","last_active":"2024-03-14T09:02:21.937137+0000","last_peered":"2024-03-14T09:02:21.937137+0000","last_clean":"2024-03-14T09:02:21.937137+0000","last_became_active":"2024-03-14T09:02:20.774210+0000","last_became_peered":"2024-03-14T09:02:20.774210+0000","last_unstale":"2024-03-14T09:02:21.937137+0000","last_undegraded":"2024-03-14T09:02:21.937137+0000","last_fullsized":"2024-03-14T09:02:21.937137+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:10:03.012273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766292+0000","last_change":"2024-03-14T09:02:03.757604+0000","last_active":"2024-03-14T09:02:20.766292+0000","last_peered":"2024-03-14T09:02:20.766292+0000","last_clean":"2024-03-14T09:02:20.766292+0000","last_became_active":"2024-03-14T09:02:03.757387+0000","last_became_peered":"2024-03-14T09:02:03.757387+0000","last_unstale":"2024-03-14T09:02:20.766292+0000","last_undegraded":"2024-03-14T09:02:20.766292+0000","last_fullsized":"2024-03-14T09:02:20.766292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:31:39.192798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764134+0000","last_change":"2024-03-14T09:02:20.784598+0000","last_active":"2024-03-14T09:02:21.764134+0000","last_peered":"2024-03-14T09:02:21.764134+0000","last_clean":"2024-03-14T09:02:21.764134+0000","last_became_active":"2024-03-14T09:02:20.784352+0000","last_became_peered":"2024-03-14T09:02:20.784352+0000","last_unstale":"2024-03-14T09:02:21.764134+0000","last_undegraded":"2024-03-14T09:02:21.764134+0000","last_fullsized":"2024-03-14T09:02:21.764134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:02:31.536153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.758604+0000","last_change":"2024-03-14T09:02:03.741584+0000","last_active":"2024-03-14T09:02:20.758604+0000","last_peered":"2024-03-14T09:02:20.758604+0000","last_clean":"2024-03-14T09:02:20.758604+0000","last_became_active":"2024-03-14T09:02:03.741367+0000","last_became_peered":"2024-03-14T09:02:03.741367+0000","last_unstale":"2024-03-14T09:02:20.758604+0000","last_undegraded":"2024-03-14T09:02:20.758604+0000","last_fullsized":"2024-03-14T09:02:20.758604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763037+0000","last_change":"2024-03-14T09:02:20.771790+0000","last_active":"2024-03-14T09:02:21.763037+0000","last_peered":"2024-03-14T09:02:21.763037+0000","last_clean":"2024-03-14T09:02:21.763037+0000","last_became_active":"2024-03-14T09:02:20.771470+0000","last_became_peered":"2024-03-14T09:02:20.771470+0000","last_unstale":"2024-03-14T09:02:21.763037+0000","last_undegraded":"2024-03-14T09:02:21.763037+0000","last_fullsized":"2024-03-14T09:02:21.763037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:48.311747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767538+0000","last_change":"2024-03-14T09:02:03.749275+0000","last_active":"2024-03-14T09:02:20.767538+0000","last_peered":"2024-03-14T09:02:20.767538+0000","last_clean":"2024-03-14T09:02:20.767538+0000","last_became_active":"2024-03-14T09:02:03.748874+0000","last_became_peered":"2024-03-14T09:02:03.748874+0000","last_unstale":"2024-03-14T09:02:20.767538+0000","last_undegraded":"2024-03-14T09:02:20.767538+0000","last_fullsized":"2024-03-14T09:02:20.767538+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:37:57.561198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763591+0000","last_change":"2024-03-14T09:02:20.771541+0000","last_active":"2024-03-14T09:02:21.763591+0000","last_peered":"2024-03-14T09:02:21.763591+0000","last_clean":"2024-03-14T09:02:21.763591+0000","last_became_active":"2024-03-14T09:02:20.770773+0000","last_became_peered":"2024-03-14T09:02:20.770773+0000","last_unstale":"2024-03-14T09:02:21.763591+0000","last_undegraded":"2024-03-14T09:02:21.763591+0000","last_fullsized":"2024-03-14T09:02:21.763591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:59:20.338489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765790+0000","last_change":"2024-03-14T09:02:03.743740+0000","last_active":"2024-03-14T09:02:20.765790+0000","last_peered":"2024-03-14T09:02:20.765790+0000","last_clean":"2024-03-14T09:02:20.765790+0000","last_became_active":"2024-03-14T09:02:03.742714+0000","last_became_peered":"2024-03-14T09:02:03.742714+0000","last_unstale":"2024-03-14T09:02:20.765790+0000","last_undegraded":"2024-03-14T09:02:20.765790+0000","last_fullsized":"2024-03-14T09:02:20.765790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937088+0000","last_change":"2024-03-14T09:02:20.776124+0000","last_active":"2024-03-14T09:02:21.937088+0000","last_peered":"2024-03-14T09:02:21.937088+0000","last_clean":"2024-03-14T09:02:21.937088+0000","last_became_active":"2024-03-14T09:02:20.775420+0000","last_became_peered":"2024-03-14T09:02:20.775420+0000","last_unstale":"2024-03-14T09:02:21.937088+0000","last_undegraded":"2024-03-14T09:02:21.937088+0000","last_fullsized":"2024-03-14T09:02:21.937088+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:00:51.224503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767795+0000","last_change":"2024-03-14T09:02:03.757453+0000","last_active":"2024-03-14T09:02:20.767795+0000","last_peered":"2024-03-14T09:02:20.767795+0000","last_clean":"2024-03-14T09:02:20.767795+0000","last_became_active":"2024-03-14T09:02:03.757182+0000","last_became_peered":"2024-03-14T09:02:03.757182+0000","last_unstale":"2024-03-14T09:02:20.767795+0000","last_undegraded":"2024-03-14T09:02:20.767795+0000","last_fullsized":"2024-03-14T09:02:20.767795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:45:09.988215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937455+0000","last_change":"2024-03-14T09:02:20.779878+0000","last_active":"2024-03-14T09:02:21.937455+0000","last_peered":"2024-03-14T09:02:21.937455+0000","last_clean":"2024-03-14T09:02:21.937455+0000","last_became_active":"2024-03-14T09:02:20.779625+0000","last_became_peered":"2024-03-14T09:02:20.779625+0000","last_unstale":"2024-03-14T09:02:21.937455+0000","last_undegraded":"2024-03-14T09:02:21.937455+0000","last_fullsized":"2024-03-14T09:02:21.937455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:03.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765504+0000","last_change":"2024-03-14T09:02:03.765489+0000","last_active":"2024-03-14T09:02:20.765504+0000","last_peered":"2024-03-14T09:02:20.765504+0000","last_clean":"2024-03-14T09:02:20.765504+0000","last_became_active":"2024-03-14T09:02:03.765228+0000","last_became_peered":"2024-03-14T09:02:03.765228+0000","last_unstale":"2024-03-14T09:02:20.765504+0000","last_undegraded":"2024-03-14T09:02:20.765504+0000","last_fullsized":"2024-03-14T09:02:20.765504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:39:59.816967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758557+0000","last_change":"2024-03-14T09:02:20.776749+0000","last_active":"2024-03-14T09:02:21.758557+0000","last_peered":"2024-03-14T09:02:21.758557+0000","last_clean":"2024-03-14T09:02:21.758557+0000","last_became_active":"2024-03-14T09:02:20.776512+0000","last_became_peered":"2024-03-14T09:02:20.776512+0000","last_unstale":"2024-03-14T09:02:21.758557+0000","last_undegraded":"2024-03-14T09:02:21.758557+0000","last_fullsized":"2024-03-14T09:02:21.758557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:41.157569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766494+0000","last_change":"2024-03-14T09:02:03.760516+0000","last_active":"2024-03-14T09:02:20.766494+0000","last_peered":"2024-03-14T09:02:20.766494+0000","last_clean":"2024-03-14T09:02:20.766494+0000","last_became_active":"2024-03-14T09:02:03.760292+0000","last_became_peered":"2024-03-14T09:02:03.760292+0000","last_unstale":"2024-03-14T09:02:20.766494+0000","last_undegraded":"2024-03-14T09:02:20.766494+0000","last_fullsized":"2024-03-14T09:02:20.766494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:42:45.106158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.763387+0000","last_change":"2024-03-14T09:02:20.781172+0000","last_active":"2024-03-14T09:02:21.763387+0000","last_peered":"2024-03-14T09:02:21.763387+0000","last_clean":"2024-03-14T09:02:21.763387+0000","last_became_active":"2024-03-14T09:02:20.780307+0000","last_became_peered":"2024-03-14T09:02:20.780307+0000","last_unstale":"2024-03-14T09:02:21.763387+0000","last_undegraded":"2024-03-14T09:02:21.763387+0000","last_fullsized":"2024-03-14T09:02:21.763387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:48:43.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.770355+0000","last_change":"2024-03-14T09:02:03.756745+0000","last_active":"2024-03-14T09:02:20.770355+0000","last_peered":"2024-03-14T09:02:20.770355+0000","last_clean":"2024-03-14T09:02:20.770355+0000","last_became_active":"2024-03-14T09:02:03.756499+0000","last_became_peered":"2024-03-14T09:02:03.756499+0000","last_unstale":"2024-03-14T09:02:20.770355+0000","last_undegraded":"2024-03-14T09:02:20.770355+0000","last_fullsized":"2024-03-14T09:02:20.770355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:03:52.159497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937052+0000","last_change":"2024-03-14T09:02:20.771393+0000","last_active":"2024-03-14T09:02:21.937052+0000","last_peered":"2024-03-14T09:02:21.937052+0000","last_clean":"2024-03-14T09:02:21.937052+0000","last_became_active":"2024-03-14T09:02:20.770865+0000","last_became_peered":"2024-03-14T09:02:20.770865+0000","last_unstale":"2024-03-14T09:02:21.937052+0000","last_undegraded":"2024-03-14T09:02:21.937052+0000","last_fullsized":"2024-03-14T09:02:21.937052+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:43.032823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767198+0000","last_change":"2024-03-14T09:02:03.752051+0000","last_active":"2024-03-14T09:02:20.767198+0000","last_peered":"2024-03-14T09:02:20.767198+0000","last_clean":"2024-03-14T09:02:20.767198+0000","last_became_active":"2024-03-14T09:02:03.751771+0000","last_became_peered":"2024-03-14T09:02:03.751771+0000","last_unstale":"2024-03-14T09:02:20.767198+0000","last_undegraded":"2024-03-14T09:02:20.767198+0000","last_fullsized":"2024-03-14T09:02:20.767198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.758117+0000","last_change":"2024-03-14T09:02:20.785050+0000","last_active":"2024-03-14T09:02:21.758117+0000","last_peered":"2024-03-14T09:02:21.758117+0000","last_clean":"2024-03-14T09:02:21.758117+0000","last_became_active":"2024-03-14T09:02:20.784824+0000","last_became_peered":"2024-03-14T09:02:20.784824+0000","last_unstale":"2024-03-14T09:02:21.758117+0000","last_undegraded":"2024-03-14T09:02:21.758117+0000","last_fullsized":"2024-03-14T09:02:21.758117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:02.131037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767943+0000","last_change":"2024-03-14T09:02:03.751964+0000","last_active":"2024-03-14T09:02:20.767943+0000","last_peered":"2024-03-14T09:02:20.767943+0000","last_clean":"2024-03-14T09:02:20.767943+0000","last_became_active":"2024-03-14T09:02:03.751703+0000","last_became_peered":"2024-03-14T09:02:03.751703+0000","last_unstale":"2024-03-14T09:02:20.767943+0000","last_undegraded":"2024-03-14T09:02:20.767943+0000","last_fullsized":"2024-03-14T09:02:20.767943+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.762891+0000","last_change":"2024-03-14T09:02:20.782589+0000","last_active":"2024-03-14T09:02:21.762891+0000","last_peered":"2024-03-14T09:02:21.762891+0000","last_clean":"2024-03-14T09:02:21.762891+0000","last_became_active":"2024-03-14T09:02:20.782261+0000","last_became_peered":"2024-03-14T09:02:20.782261+0000","last_unstale":"2024-03-14T09:02:21.762891+0000","last_undegraded":"2024-03-14T09:02:21.762891+0000","last_fullsized":"2024-03-14T09:02:21.762891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:01:09.186009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764620+0000","last_change":"2024-03-14T09:02:03.752408+0000","last_active":"2024-03-14T09:02:20.764620+0000","last_peered":"2024-03-14T09:02:20.764620+0000","last_clean":"2024-03-14T09:02:20.764620+0000","last_became_active":"2024-03-14T09:02:03.752136+0000","last_became_peered":"2024-03-14T09:02:03.752136+0000","last_unstale":"2024-03-14T09:02:20.764620+0000","last_undegraded":"2024-03-14T09:02:20.764620+0000","last_fullsized":"2024-03-14T09:02:20.764620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764128+0000","last_change":"2024-03-14T09:02:20.774892+0000","last_active":"2024-03-14T09:02:21.764128+0000","last_peered":"2024-03-14T09:02:21.764128+0000","last_clean":"2024-03-14T09:02:21.764128+0000","last_became_active":"2024-03-14T09:02:20.774574+0000","last_became_peered":"2024-03-14T09:02:20.774574+0000","last_unstale":"2024-03-14T09:02:21.764128+0000","last_undegraded":"2024-03-14T09:02:21.764128+0000","last_fullsized":"2024-03-14T09:02:21.764128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:52:28.330150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767440+0000","last_change":"2024-03-14T09:02:03.751100+0000","last_active":"2024-03-14T09:02:20.767440+0000","last_peered":"2024-03-14T09:02:20.767440+0000","last_clean":"2024-03-14T09:02:20.767440+0000","last_became_active":"2024-03-14T09:02:03.750866+0000","last_became_peered":"2024-03-14T09:02:03.750866+0000","last_unstale":"2024-03-14T09:02:20.767440+0000","last_undegraded":"2024-03-14T09:02:20.767440+0000","last_fullsized":"2024-03-14T09:02:20.767440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:27:07.275479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937198+0000","last_change":"2024-03-14T09:02:20.771963+0000","last_active":"2024-03-14T09:02:21.937198+0000","last_peered":"2024-03-14T09:02:21.937198+0000","last_clean":"2024-03-14T09:02:21.937198+0000","last_became_active":"2024-03-14T09:02:20.771333+0000","last_became_peered":"2024-03-14T09:02:20.771333+0000","last_unstale":"2024-03-14T09:02:21.937198+0000","last_undegraded":"2024-03-14T09:02:21.937198+0000","last_fullsized":"2024-03-14T09:02:21.937198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:23.832986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768428+0000","last_change":"2024-03-14T09:02:03.750480+0000","last_active":"2024-03-14T09:02:20.768428+0000","last_peered":"2024-03-14T09:02:20.768428+0000","last_clean":"2024-03-14T09:02:20.768428+0000","last_became_active":"2024-03-14T09:02:03.750198+0000","last_became_peered":"2024-03-14T09:02:03.750198+0000","last_unstale":"2024-03-14T09:02:20.768428+0000","last_undegraded":"2024-03-14T09:02:20.768428+0000","last_fullsized":"2024-03-14T09:02:20.768428+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:14.770721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936590+0000","last_change":"2024-03-14T09:02:20.772425+0000","last_active":"2024-03-14T09:02:21.936590+0000","last_peered":"2024-03-14T09:02:21.936590+0000","last_clean":"2024-03-14T09:02:21.936590+0000","last_became_active":"2024-03-14T09:02:20.771116+0000","last_became_peered":"2024-03-14T09:02:20.771116+0000","last_unstale":"2024-03-14T09:02:21.936590+0000","last_undegraded":"2024-03-14T09:02:21.936590+0000","last_fullsized":"2024-03-14T09:02:21.936590+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:00:51.224503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768811+0000","last_change":"2024-03-14T09:02:03.758860+0000","last_active":"2024-03-14T09:02:20.768811+0000","last_peered":"2024-03-14T09:02:20.768811+0000","last_clean":"2024-03-14T09:02:20.768811+0000","last_became_active":"2024-03-14T09:02:03.758547+0000","last_became_peered":"2024-03-14T09:02:03.758547+0000","last_unstale":"2024-03-14T09:02:20.768811+0000","last_undegraded":"2024-03-14T09:02:20.768811+0000","last_fullsized":"2024-03-14T09:02:20.768811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:22:55.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938606+0000","last_change":"2024-03-14T09:02:20.779057+0000","last_active":"2024-03-14T09:02:21.938606+0000","last_peered":"2024-03-14T09:02:21.938606+0000","last_clean":"2024-03-14T09:02:21.938606+0000","last_became_active":"2024-03-14T09:02:20.778597+0000","last_became_peered":"2024-03-14T09:02:20.778597+0000","last_unstale":"2024-03-14T09:02:21.938606+0000","last_undegraded":"2024-03-14T09:02:21.938606+0000","last_fullsized":"2024-03-14T09:02:21.938606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:33.018684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.765127+0000","last_change":"2024-03-14T09:02:03.757773+0000","last_active":"2024-03-14T09:02:20.765127+0000","last_peered":"2024-03-14T09:02:20.765127+0000","last_clean":"2024-03-14T09:02:20.765127+0000","last_became_active":"2024-03-14T09:02:03.757426+0000","last_became_peered":"2024-03-14T09:02:03.757426+0000","last_unstale":"2024-03-14T09:02:20.765127+0000","last_undegraded":"2024-03-14T09:02:20.765127+0000","last_fullsized":"2024-03-14T09:02:20.765127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:19.679352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936879+0000","last_change":"2024-03-14T09:02:20.773017+0000","last_active":"2024-03-14T09:02:21.936879+0000","last_peered":"2024-03-14T09:02:21.936879+0000","last_clean":"2024-03-14T09:02:21.936879+0000","last_became_active":"2024-03-14T09:02:20.772060+0000","last_became_peered":"2024-03-14T09:02:20.772060+0000","last_unstale":"2024-03-14T09:02:21.936879+0000","last_undegraded":"2024-03-14T09:02:21.936879+0000","last_fullsized":"2024-03-14T09:02:21.936879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:03.659847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767353+0000","last_change":"2024-03-14T09:02:03.741974+0000","last_active":"2024-03-14T09:02:20.767353+0000","last_peered":"2024-03-14T09:02:20.767353+0000","last_clean":"2024-03-14T09:02:20.767353+0000","last_became_active":"2024-03-14T09:02:03.741657+0000","last_became_peered":"2024-03-14T09:02:03.741657+0000","last_unstale":"2024-03-14T09:02:20.767353+0000","last_undegraded":"2024-03-14T09:02:20.767353+0000","last_fullsized":"2024-03-14T09:02:20.767353+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:25:11.985101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.764044+0000","last_change":"2024-03-14T09:02:20.778499+0000","last_active":"2024-03-14T09:02:21.764044+0000","last_peered":"2024-03-14T09:02:21.764044+0000","last_clean":"2024-03-14T09:02:21.764044+0000","last_became_active":"2024-03-14T09:02:20.778076+0000","last_became_peered":"2024-03-14T09:02:20.778076+0000","last_unstale":"2024-03-14T09:02:21.764044+0000","last_undegraded":"2024-03-14T09:02:21.764044+0000","last_fullsized":"2024-03-14T09:02:21.764044+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:38:51.778569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767007+0000","last_change":"2024-03-14T09:02:03.753172+0000","last_active":"2024-03-14T09:02:20.767007+0000","last_peered":"2024-03-14T09:02:20.767007+0000","last_clean":"2024-03-14T09:02:20.767007+0000","last_became_active":"2024-03-14T09:02:03.752882+0000","last_became_peered":"2024-03-14T09:02:03.752882+0000","last_unstale":"2024-03-14T09:02:20.767007+0000","last_undegraded":"2024-03-14T09:02:20.767007+0000","last_fullsized":"2024-03-14T09:02:20.767007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:31:17.856350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937986+0000","last_change":"2024-03-14T09:02:20.781181+0000","last_active":"2024-03-14T09:02:21.937986+0000","last_peered":"2024-03-14T09:02:21.937986+0000","last_clean":"2024-03-14T09:02:21.937986+0000","last_became_active":"2024-03-14T09:02:20.780863+0000","last_became_peered":"2024-03-14T09:02:20.780863+0000","last_unstale":"2024-03-14T09:02:21.937986+0000","last_undegraded":"2024-03-14T09:02:21.937986+0000","last_fullsized":"2024-03-14T09:02:21.937986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:35.201864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766951+0000","last_change":"2024-03-14T09:02:03.751248+0000","last_active":"2024-03-14T09:02:20.766951+0000","last_peered":"2024-03-14T09:02:20.766951+0000","last_clean":"2024-03-14T09:02:20.766951+0000","last_became_active":"2024-03-14T09:02:03.750977+0000","last_became_peered":"2024-03-14T09:02:03.750977+0000","last_unstale":"2024-03-14T09:02:20.766951+0000","last_undegraded":"2024-03-14T09:02:20.766951+0000","last_fullsized":"2024-03-14T09:02:20.766951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936087+0000","last_change":"2024-03-14T09:02:20.769707+0000","last_active":"2024-03-14T09:02:21.936087+0000","last_peered":"2024-03-14T09:02:21.936087+0000","last_clean":"2024-03-14T09:02:21.936087+0000","last_became_active":"2024-03-14T09:02:20.769411+0000","last_became_peered":"2024-03-14T09:02:20.769411+0000","last_unstale":"2024-03-14T09:02:21.936087+0000","last_undegraded":"2024-03-14T09:02:21.936087+0000","last_fullsized":"2024-03-14T09:02:21.936087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:01:55.985257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.764075+0000","last_change":"2024-03-14T09:02:03.747083+0000","last_active":"2024-03-14T09:02:20.764075+0000","last_peered":"2024-03-14T09:02:20.764075+0000","last_clean":"2024-03-14T09:02:20.764075+0000","last_became_active":"2024-03-14T09:02:03.743458+0000","last_became_peered":"2024-03-14T09:02:03.743458+0000","last_unstale":"2024-03-14T09:02:20.764075+0000","last_undegraded":"2024-03-14T09:02:20.764075+0000","last_fullsized":"2024-03-14T09:02:20.764075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:56.122999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.936677+0000","last_change":"2024-03-14T09:02:20.781501+0000","last_active":"2024-03-14T09:02:21.936677+0000","last_peered":"2024-03-14T09:02:21.936677+0000","last_clean":"2024-03-14T09:02:21.936677+0000","last_became_active":"2024-03-14T09:02:20.781094+0000","last_became_peered":"2024-03-14T09:02:20.781094+0000","last_unstale":"2024-03-14T09:02:21.936677+0000","last_undegraded":"2024-03-14T09:02:21.936677+0000","last_fullsized":"2024-03-14T09:02:21.936677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:09:25.674123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.767275+0000","last_change":"2024-03-14T09:02:03.751416+0000","last_active":"2024-03-14T09:02:20.767275+0000","last_peered":"2024-03-14T09:02:20.767275+0000","last_clean":"2024-03-14T09:02:20.767275+0000","last_became_active":"2024-03-14T09:02:03.751119+0000","last_became_peered":"2024-03-14T09:02:03.751119+0000","last_unstale":"2024-03-14T09:02:20.767275+0000","last_undegraded":"2024-03-14T09:02:20.767275+0000","last_fullsized":"2024-03-14T09:02:20.767275+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:07:06.397586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.938863+0000","last_change":"2024-03-14T09:02:20.781357+0000","last_active":"2024-03-14T09:02:21.938863+0000","last_peered":"2024-03-14T09:02:21.938863+0000","last_clean":"2024-03-14T09:02:21.938863+0000","last_became_active":"2024-03-14T09:02:20.781031+0000","last_became_peered":"2024-03-14T09:02:20.781031+0000","last_unstale":"2024-03-14T09:02:21.938863+0000","last_undegraded":"2024-03-14T09:02:21.938863+0000","last_fullsized":"2024-03-14T09:02:21.938863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:25:23.832986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.762513+0000","last_change":"2024-03-14T09:02:03.747476+0000","last_active":"2024-03-14T09:02:20.762513+0000","last_peered":"2024-03-14T09:02:20.762513+0000","last_clean":"2024-03-14T09:02:20.762513+0000","last_became_active":"2024-03-14T09:02:03.747104+0000","last_became_peered":"2024-03-14T09:02:03.747104+0000","last_unstale":"2024-03-14T09:02:20.762513+0000","last_undegraded":"2024-03-14T09:02:20.762513+0000","last_fullsized":"2024-03-14T09:02:20.762513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:59.457776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937837+0000","last_change":"2024-03-14T09:02:20.783389+0000","last_active":"2024-03-14T09:02:21.937837+0000","last_peered":"2024-03-14T09:02:21.937837+0000","last_clean":"2024-03-14T09:02:21.937837+0000","last_became_active":"2024-03-14T09:02:20.783105+0000","last_became_peered":"2024-03-14T09:02:20.783105+0000","last_unstale":"2024-03-14T09:02:21.937837+0000","last_undegraded":"2024-03-14T09:02:21.937837+0000","last_fullsized":"2024-03-14T09:02:21.937837+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:22.386770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.768186+0000","last_change":"2024-03-14T09:02:03.741721+0000","last_active":"2024-03-14T09:02:20.768186+0000","last_peered":"2024-03-14T09:02:20.768186+0000","last_clean":"2024-03-14T09:02:20.768186+0000","last_became_active":"2024-03-14T09:02:03.741350+0000","last_became_peered":"2024-03-14T09:02:03.741350+0000","last_unstale":"2024-03-14T09:02:20.768186+0000","last_undegraded":"2024-03-14T09:02:20.768186+0000","last_fullsized":"2024-03-14T09:02:20.768186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:05.151601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937222+0000","last_change":"2024-03-14T09:02:20.774158+0000","last_active":"2024-03-14T09:02:21.937222+0000","last_peered":"2024-03-14T09:02:21.937222+0000","last_clean":"2024-03-14T09:02:21.937222+0000","last_became_active":"2024-03-14T09:02:20.773930+0000","last_became_peered":"2024-03-14T09:02:20.773930+0000","last_unstale":"2024-03-14T09:02:21.937222+0000","last_undegraded":"2024-03-14T09:02:21.937222+0000","last_fullsized":"2024-03-14T09:02:21.937222+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:41:33.018684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.766577+0000","last_change":"2024-03-14T09:02:03.757259+0000","last_active":"2024-03-14T09:02:20.766577+0000","last_peered":"2024-03-14T09:02:20.766577+0000","last_clean":"2024-03-14T09:02:20.766577+0000","last_became_active":"2024-03-14T09:02:03.757050+0000","last_became_peered":"2024-03-14T09:02:03.757050+0000","last_unstale":"2024-03-14T09:02:20.766577+0000","last_undegraded":"2024-03-14T09:02:20.766577+0000","last_fullsized":"2024-03-14T09:02:20.766577+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:14:24.532588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-14T09:02:21.937116+0000","last_change":"2024-03-14T09:02:20.771784+0000","last_active":"2024-03-14T09:02:21.937116+0000","last_peered":"2024-03-14T09:02:21.937116+0000","last_clean":"2024-03-14T09:02:21.937116+0000","last_became_active":"2024-03-14T09:02:20.771497+0000","last_became_peered":"2024-03-14T09:02:20.771497+0000","last_unstale":"2024-03-14T09:02:21.937116+0000","last_undegraded":"2024-03-14T09:02:21.937116+0000","last_fullsized":"2024-03-14T09:02:21.937116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:20.336835+0000","last_clean_scrub_stamp":"2024-03-14T09:02:20.336835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:05:42.439216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-14T09:02:20.759931+0000","last_change":"2024-03-14T09:02:03.755819+0000","last_active":"2024-03-14T09:02:20.759931+0000","last_peered":"2024-03-14T09:02:20.759931+0000","last_clean":"2024-03-14T09:02:20.759931+0000","last_became_active":"2024-03-14T09:02:03.750027+0000","last_became_peered":"2024-03-14T09:02:03.750027+0000","last_unstale":"2024-03-14T09:02:20.759931+0000","last_undegraded":"2024-03-14T09:02:20.759931+0000","last_fullsized":"2024-03-14T09:02:20.759931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:03.311957+0000","last_clean_scrub_stamp":"2024-03-14T09:02:03.311957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:23:14.643515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":758536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.622},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.78800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":425984,"data_stored":169276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.71699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":171155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.22900000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.37}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":758390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.31}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.755},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":169644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":438272,"data_stored":171723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.314},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.374},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.504}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55800000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.49099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":169656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.311},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.505},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":168730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.622}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.38700000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.435}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:02:25.270 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:02:25.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-03-14T09:02:25.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.432+0000 7f64a1b39700 1 Processor -- start 2024-03-14T09:02:25.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.432+0000 7f64a1b39700 1 -- start start 2024-03-14T09:02:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f64a1b39700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f649c0c3250 0x7f649c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f64a1b39700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f64a1b39700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c13ade0 con 0x7f649c0a8540 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f64a1b39700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c06fb30 con 0x7f649c0c3250 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f64a1b39700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06ec30 con 0x7f649c0c1780 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649affd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f649c0c3250 0x7f649c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f649c0a8540 0x7f649c154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33762/0 (socket says 172.21.15.154:33762) 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649b7fe700 1 -- 172.21.15.154:0/126461723 learned_addr learned my addr 172.21.15.154:0/126461723 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649affd700 1 -- 172.21.15.154:0/126461723 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 msgr2=0x7f649c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649bfff700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649affd700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c0c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649affd700 1 -- 172.21.15.154:0/126461723 >> v1:172.21.15.154:6789/0 conn(0x7f649c0a8540 legacy=0x7f649c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649a7fc700 1 -- 172.21.15.154:0/126461723 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123824350 0 0) 0x7f6484001110 con 0x7f649c0a8540 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649affd700 1 -- 172.21.15.154:0/126461723 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c06ef10 con 0x7f649c0c3250 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.433+0000 7f649bfff700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:25.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f649affd700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c13a8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f648c009840 tx=0x7f648c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36adde4a47031c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f649a7fc700 1 -- 172.21.15.154:0/126461723 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c011830 con 0x7f649c0c3250 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f649a7fc700 1 -- 172.21.15.154:0/126461723 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c011990 con 0x7f649c0c3250 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f649a7fc700 1 -- 172.21.15.154:0/126461723 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c011d20 con 0x7f649c0c3250 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 -- 172.21.15.154:0/126461723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 msgr2=0x7f649c13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c13a8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f648c009840 tx=0x7f648c009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 -- 172.21.15.154:0/126461723 shutdown_connections 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 --2- 172.21.15.154:0/126461723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c13a8a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 -- 172.21.15.154:0/126461723 >> 172.21.15.154:0/126461723 conn(0x7f649c0c3880 msgr2=0x7f649c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 -- 172.21.15.154:0/126461723 shutdown_connections 2024-03-14T09:02:25.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.434+0000 7f64a1b39700 1 -- 172.21.15.154:0/126461723 wait complete. 2024-03-14T09:02:25.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.435+0000 7f64a1b39700 1 Processor -- start 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 -- start start 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f649c0a8540 0x7f649c1ea960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c139580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1db400 con 0x7f649c0c3250 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c1eaea0 con 0x7f649c0a8540 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f64a1b39700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c06a960 con 0x7f649c0c1780 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f649bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f649bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47824/0 (socket says 172.21.15.154:47824) 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f649bfff700 1 -- 172.21.15.154:0/3095690892 learned_addr learned my addr 172.21.15.154:0/3095690892 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f649affd700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.436+0000 7f649b7fe700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f649c0a8540 0x7f649c1ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 -- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 msgr2=0x7f649c139580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c139580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 -- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f649c0a8540 msgr2=0x7f649c1ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f649c0a8540 0x7f649c1ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 -- 172.21.15.154:0/3095690892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648c0094a0 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f649bfff700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f649400cd50 tx=0x7f649400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400cea0 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649400f480 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c1ed680 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649400f830 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.437+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c06ae90 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.438+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c1ed810 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.438+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f649400b040 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.439+0000 7f6493fff700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 0x7f647c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.439+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f64940897c0 con 0x7f649c0c3250 2024-03-14T09:02:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.439+0000 7f649b7fe700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 0x7f647c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.439+0000 7f649b7fe700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 0x7f647c06ccf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6484000f60 tx=0x7f6484007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.443+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6494054880 con 0x7f649c0c3250 2024-03-14T09:02:25.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.718+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f649c0697d0 con 0x7f649c0c3250 2024-03-14T09:02:25.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.721+0000 7f6493fff700 1 -- 172.21.15.154:0/3095690892 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v16) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6494057ea0 con 0x7f649c0c3250 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 msgr2=0x7f647c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 0x7f647c06ccf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6484000f60 tx=0x7f6484007e70 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 msgr2=0x7f649c1edb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f649400cd50 tx=0x7f649400a560 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 shutdown_connections 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f647c06a830 0x7f647c06ccf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f649c0c3250 0x7f649c1edb10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f649c0a8540 0x7f649c1ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 --2- 172.21.15.154:0/3095690892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f649c0c1780 0x7f649c139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.723+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 >> 172.21.15.154:0/3095690892 conn(0x7f649c0c3880 msgr2=0x7f649c06de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:25.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.724+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 shutdown_connections 2024-03-14T09:02:25.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.724+0000 7f64a1b39700 1 -- 172.21.15.154:0/3095690892 wait complete. 2024-03-14T09:02:25.725 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.1 2024-03-14T09:02:25.736 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:25.736 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:25.757 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:25.757 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:25.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:25.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.940+0000 7fcd61b82700 1 Processor -- start 2024-03-14T09:02:25.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.940+0000 7fcd61b82700 1 -- start start 2024-03-14T09:02:25.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd61b82700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd5c1553d0 0x7fcd5c158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd61b82700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5c154f40 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd61b82700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5c158d80 con 0x7fcd5c0bf9c0 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd61b82700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5c0c16a0 con 0x7fcd5c1553d0 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd5c1553d0 0x7fcd5c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5affd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcd5c0c1d50 0x7fcd5c0c0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33768/0 (socket says 172.21.15.154:33768) 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5affd700 1 -- 172.21.15.154:0/2339955403 learned_addr learned my addr 172.21.15.154:0/2339955403 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5b7fe700 1 -- 172.21.15.154:0/2339955403 >> v1:172.21.15.191:6789/0 conn(0x7fcd5c0bf9c0 legacy=0x7fcd5c15f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5b7fe700 1 -- 172.21.15.154:0/2339955403 >> v1:172.21.15.154:6789/0 conn(0x7fcd5c0c1d50 legacy=0x7fcd5c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5b7fe700 1 -- 172.21.15.154:0/2339955403 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c154930 con 0x7fcd5c1553d0 2024-03-14T09:02:25.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5a7fc700 1 -- 172.21.15.154:0/2339955403 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363563982 0 0) 0x7fcd5c154f40 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5b7fe700 1 --2- 172.21.15.154:0/2339955403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c158840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009840 tx=0x7fcd50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8556ec2605fa094d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.941+0000 7fcd5a7fc700 1 -- 172.21.15.154:0/2339955403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50004230 con 0x7fcd5c1553d0 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd5a7fc700 1 -- 172.21.15.154:0/2339955403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50004390 con 0x7fcd5c1553d0 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd5a7fc700 1 -- 172.21.15.154:0/2339955403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500036a0 con 0x7fcd5c1553d0 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 -- 172.21.15.154:0/2339955403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c1553d0 msgr2=0x7fcd5c158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 --2- 172.21.15.154:0/2339955403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c158840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009840 tx=0x7fcd50009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 -- 172.21.15.154:0/2339955403 shutdown_connections 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 --2- 172.21.15.154:0/2339955403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c158840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 -- 172.21.15.154:0/2339955403 >> 172.21.15.154:0/2339955403 conn(0x7fcd5c06a770 msgr2=0x7fcd5c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 -- 172.21.15.154:0/2339955403 shutdown_connections 2024-03-14T09:02:25.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.942+0000 7fcd61b82700 1 -- 172.21.15.154:0/2339955403 wait complete. 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.943+0000 7fcd61b82700 1 Processor -- start 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.943+0000 7fcd61b82700 1 -- start start 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 0x7fcd5c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c09a410 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1ee010 con 0x7fcd5c1553d0 2024-03-14T09:02:25.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd61b82700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c1ee2f0 con 0x7fcd5c0bf9c0 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47850/0 (socket says 172.21.15.154:47850) 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 -- 172.21.15.154:0/1395371746 learned_addr learned my addr 172.21.15.154:0/1395371746 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5bfff700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5b7fe700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 0x7fcd5c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 -- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 msgr2=0x7fcd5c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 0x7fcd5c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 -- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 msgr2=0x7fcd5c14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.944+0000 7fcd5affd700 1 -- 172.21.15.154:0/1395371746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500094a0 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd5bfff700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd5b7fe700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 0x7fcd5c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd5affd700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0011e0 tx=0x7fcd4c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004c80 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c004e00 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c09a660 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c007310 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.945+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c0722a0 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.946+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c0aa5b0 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.946+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd4c007470 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.946+0000 7fcd43fff700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 0x7fcd4806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:25.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.947+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fcd4c088e50 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.947+0000 7fcd5b7fe700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 0x7fcd4806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:25.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.947+0000 7fcd5b7fe700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 0x7fcd4806ccf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd50000c00 tx=0x7fcd5001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:25.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:25.951+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcd5c0aa5b0 con 0x7fcd5c0c1d50 2024-03-14T09:02:25.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:02:25.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:02:26.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.153+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd5c0aa5b0 con 0x7fcd5c0c1d50 2024-03-14T09:02:26.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.154+0000 7fcd43fff700 1 -- 172.21.15.154:0/1395371746 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7fcd5c0aa5b0 con 0x7fcd5c0c1d50 2024-03-14T09:02:26.154 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:26.154 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:26.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 msgr2=0x7fcd4806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 0x7fcd4806ccf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd50000c00 tx=0x7fcd5001c040 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 msgr2=0x7fcd5c071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0011e0 tx=0x7fcd4c00cae0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 shutdown_connections 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c1553d0 0x7fcd5c14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd5c0bf9c0 0x7fcd5c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4806a830 0x7fcd4806ccf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 --2- 172.21.15.154:0/1395371746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd5c0c1d50 0x7fcd5c071d60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 >> 172.21.15.154:0/1395371746 conn(0x7fcd5c06a770 msgr2=0x7fcd5c0c0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.156+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 shutdown_connections 2024-03-14T09:02:26.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.157+0000 7fcd61b82700 1 -- 172.21.15.154:0/1395371746 wait complete. 2024-03-14T09:02:26.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:01.699649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:02.688126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:02:19.001797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:02:19.712147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:26.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:26.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.293+0000 7f7051328700 1 Processor -- start 2024-03-14T09:02:26.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f7051328700 1 -- start start 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f7051328700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f704c0c4af0 0x7f704c0c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f7051328700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0c55e0 con 0x7f704c06b300 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f7051328700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0c8290 con 0x7f704c15d370 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f7051328700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c0c5ac0 con 0x7f704c0c4af0 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f704affd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f704c0c4af0 0x7f704c0c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f704a7fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f704c06b300 0x7f704c06b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33774/0 (socket says 172.21.15.154:33774) 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.294+0000 7f704a7fc700 1 -- 172.21.15.154:0/2932503574 learned_addr learned my addr 172.21.15.154:0/2932503574 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f704affd700 1 -- 172.21.15.154:0/2932503574 >> v1:172.21.15.191:6789/0 conn(0x7f704c15d370 legacy=0x7f704c15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f704affd700 1 -- 172.21.15.154:0/2932503574 >> v1:172.21.15.154:6789/0 conn(0x7f704c06b300 legacy=0x7f704c06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f7049ffb700 1 -- 172.21.15.154:0/2932503574 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117265900 0 0) 0x7f704c0c55e0 con 0x7f704c06b300 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f704affd700 1 -- 172.21.15.154:0/2932503574 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c154bf0 con 0x7f704c0c4af0 2024-03-14T09:02:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f704affd700 1 --2- 172.21.15.154:0/2932503574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c0c4af0 0x7f704c0c4ed0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7034009840 tx=0x7f7034009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdf1296dcff06fe server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f7049ffb700 1 -- 172.21.15.154:0/2932503574 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011830 con 0x7f704c0c4af0 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f7049ffb700 1 -- 172.21.15.154:0/2932503574 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034011990 con 0x7f704c0c4af0 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.295+0000 7f7049ffb700 1 -- 172.21.15.154:0/2932503574 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034011d20 con 0x7f704c0c4af0 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 -- 172.21.15.154:0/2932503574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c0c4af0 msgr2=0x7f704c0c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 --2- 172.21.15.154:0/2932503574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c0c4af0 0x7f704c0c4ed0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7034009840 tx=0x7f7034009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 -- 172.21.15.154:0/2932503574 shutdown_connections 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 --2- 172.21.15.154:0/2932503574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c0c4af0 0x7f704c0c4ed0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 -- 172.21.15.154:0/2932503574 >> 172.21.15.154:0/2932503574 conn(0x7f704c0abc60 msgr2=0x7f704c0ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 -- 172.21.15.154:0/2932503574 shutdown_connections 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.296+0000 7f7051328700 1 -- 172.21.15.154:0/2932503574 wait complete. 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.297+0000 7f7051328700 1 Processor -- start 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.297+0000 7f7051328700 1 -- start start 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f704c0c4af0 0x7f704c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f704c15d370 0x7f704c15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c159d60 con 0x7f704c06b300 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c14f850 con 0x7f704c15d370 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c159f70 con 0x7f704c0c4af0 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47860/0 (socket says 172.21.15.154:47860) 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 -- 172.21.15.154:0/723110437 learned_addr learned my addr 172.21.15.154:0/723110437 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704b7fe700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f704c15d370 0x7f704c15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704a7fc700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f704c0c4af0 0x7f704c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 -- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f704c0c4af0 msgr2=0x7f704c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f704c0c4af0 0x7f704c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 -- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f704c15d370 msgr2=0x7f704c15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f704c15d370 0x7f704c15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 -- 172.21.15.154:0/723110437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340094a0 con 0x7f704c06b300 2024-03-14T09:02:26.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f704affd700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f70340090b0 tx=0x7f7034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703401a980 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703401ab00 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c15a250 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034024420 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.298+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c15acb0 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.299+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c067bd0 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.300+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7034022070 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.300+0000 7f70437fe700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 0x7f702c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.300+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f703408a440 con 0x7f704c06b300 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.300+0000 7f704a7fc700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 0x7f702c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.301+0000 7f704a7fc700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 0x7f702c06ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c007c20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.305+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f70340556b0 con 0x7f704c06b300 2024-03-14T09:02:26.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.498+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f704c0697d0 con 0x7f704c06b300 2024-03-14T09:02:26.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.499+0000 7f70437fe700 1 -- 172.21.15.154:0/723110437 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7034029020 con 0x7f704c06b300 2024-03-14T09:02:26.498 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 msgr2=0x7f702c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 0x7f702c06ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c007c20 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 msgr2=0x7f704c0c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f70340090b0 tx=0x7f7034004750 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 shutdown_connections 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f704c15d370 0x7f704c15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f702c06a830 0x7f702c06ccf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f704c06b300 0x7f704c0c14f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 --2- 172.21.15.154:0/723110437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f704c0c4af0 0x7f704c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 >> 172.21.15.154:0/723110437 conn(0x7f704c0abc60 msgr2=0x7f704c0c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 shutdown_connections 2024-03-14T09:02:26.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.501+0000 7f7051328700 1 -- 172.21.15.154:0/723110437 wait complete. 2024-03-14T09:02:26.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:02:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.637+0000 7fd57f003700 1 Processor -- start 2024-03-14T09:02:26.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.637+0000 7fd57f003700 1 -- start start 2024-03-14T09:02:26.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57f003700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 0x7fd57806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57f003700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd578155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57f003700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780ac8e0 con 0x7fd578156330 2024-03-14T09:02:26.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57f003700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57806b310 con 0x7fd57814eaf0 2024-03-14T09:02:26.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57f003700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57806de30 con 0x7fd57806acb0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57d5a0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd578156330 0x7fd578157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33776/0 (socket says 172.21.15.154:33776) 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57d5a0700 1 -- 172.21.15.154:0/3626805559 learned_addr learned my addr 172.21.15.154:0/3626805559 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd577fff700 1 --2- 172.21.15.154:0/3626805559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd578155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3626805559 >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 0x7fd57806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd5777fe700 1 -- 172.21.15.154:0/3626805559 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583639643 0 0) 0x7fd5780ac8e0 con 0x7fd578156330 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd5777fe700 1 -- 172.21.15.154:0/3626805559 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5680035b0 con 0x7fd578156330 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd577fff700 1 -- 172.21.15.154:0/3626805559 >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 msgr2=0x7fd57806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd577fff700 1 --2- 172.21.15.154:0/3626805559 >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 0x7fd57806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd577fff700 1 -- 172.21.15.154:0/3626805559 >> v1:172.21.15.154:6789/0 conn(0x7fd578156330 legacy=0x7fd578157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd577fff700 1 -- 172.21.15.154:0/3626805559 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780acb30 con 0x7fd57806acb0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.638+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3626805559 >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 0x7fd57806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd577fff700 1 --2- 172.21.15.154:0/3626805559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd578155df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd564009840 tx=0x7fd564009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699cd49ac82238bc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd5777fe700 1 -- 172.21.15.154:0/3626805559 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564011830 con 0x7fd57806acb0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd5777fe700 1 -- 172.21.15.154:0/3626805559 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564011990 con 0x7fd57806acb0 2024-03-14T09:02:26.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd5777fe700 1 -- 172.21.15.154:0/3626805559 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564011d00 con 0x7fd57806acb0 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 -- 172.21.15.154:0/3626805559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 msgr2=0x7fd578155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 --2- 172.21.15.154:0/3626805559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd578155df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd564009840 tx=0x7fd564009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 -- 172.21.15.154:0/3626805559 shutdown_connections 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 --2- 172.21.15.154:0/3626805559 >> v2:172.21.15.191:3300/0 conn(0x7fd57814eaf0 0x7fd57806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 --2- 172.21.15.154:0/3626805559 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd578155df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 -- 172.21.15.154:0/3626805559 >> 172.21.15.154:0/3626805559 conn(0x7fd5780c3880 msgr2=0x7fd5780c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.639+0000 7fd57f003700 1 -- 172.21.15.154:0/3626805559 shutdown_connections 2024-03-14T09:02:26.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.640+0000 7fd57f003700 1 -- 172.21.15.154:0/3626805559 wait complete. 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 Processor -- start 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 -- start start 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd5781534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 0x7fd57814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781ee120 con 0x7fd578156330 2024-03-14T09:02:26.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781529e0 con 0x7fd57814eaf0 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.641+0000 7fd57f003700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd578152cc0 con 0x7fd57806acb0 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47876/0 (socket says 172.21.15.154:47876) 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 -- 172.21.15.154:0/3559873635 learned_addr learned my addr 172.21.15.154:0/3559873635 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd577fff700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 0x7fd57814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd5781534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 -- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 msgr2=0x7fd5781534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd5781534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 -- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 msgr2=0x7fd57814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 0x7fd57814a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 -- 172.21.15.154:0/3559873635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0094a0 con 0x7fd578156330 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd5781534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd577fff700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 0x7fd57814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:26.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57d5a0700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd57000cb40 tx=0x7fd57000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000ab50 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57000acb0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5640094a0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570012620 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.642+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57814ace0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.643+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5781531e0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.644+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd57001b3b0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.644+0000 7fd5757fa700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 0x7fd55c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.644+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd570088de0 con 0x7fd578156330 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.644+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 0x7fd55c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.645+0000 7fd57cd9f700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 0x7fd55c06ccf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5781449d0 tx=0x7fd56c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.649+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd5781531e0 con 0x7fd578156330 2024-03-14T09:02:26.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.842+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5781531e0 con 0x7fd578156330 2024-03-14T09:02:26.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.842+0000 7fd5757fa700 1 -- 172.21.15.154:0/3559873635 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd570017020 con 0x7fd578156330 2024-03-14T09:02:26.842 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:26.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 msgr2=0x7fd55c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 0x7fd55c06ccf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd5781449d0 tx=0x7fd56c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:26.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 msgr2=0x7fd5780c1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd57000cb40 tx=0x7fd57000cf00 comp rx=0 tx=0).stop 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 shutdown_connections 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd55c06a830 0x7fd55c06ccf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd578156330 0x7fd5780c1860 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd57806acb0 0x7fd5781534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 --2- 172.21.15.154:0/3559873635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd57814eaf0 0x7fd57814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 >> 172.21.15.154:0/3559873635 conn(0x7fd5780c3880 msgr2=0x7fd578071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 shutdown_connections 2024-03-14T09:02:26.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.844+0000 7fd57f003700 1 -- 172.21.15.154:0/3559873635 wait complete. 2024-03-14T09:02:26.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:02:26.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.981+0000 7fa71b3d4700 1 Processor -- start 2024-03-14T09:02:26.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.981+0000 7fa71b3d4700 1 -- start start 2024-03-14T09:02:26.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa71b3d4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa71414e980 0x7fa7141552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa71b3d4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa714158a80 0x7fa7140c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa71b3d4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71414ed60 con 0x7fa7140c4a80 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa71b3d4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7140c6b10 con 0x7fa714158a80 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa71b3d4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa714155820 con 0x7fa71414e980 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa714158a80 0x7fa7140c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa714158a80 0x7fa7140c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47878/0 (socket says 172.21.15.154:47878) 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 -- 172.21.15.154:0/2669696182 learned_addr learned my addr 172.21.15.154:0/2669696182 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719170700 1 --2- 172.21.15.154:0/2669696182 >> v2:172.21.15.191:3300/0 conn(0x7fa71414e980 0x7fa7141552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 -- 172.21.15.154:0/2669696182 >> v2:172.21.15.191:3300/0 conn(0x7fa71414e980 msgr2=0x7fa7141552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 --2- 172.21.15.154:0/2669696182 >> v2:172.21.15.191:3300/0 conn(0x7fa71414e980 0x7fa7141552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 -- 172.21.15.154:0/2669696182 >> v1:172.21.15.191:6789/0 conn(0x7fa7140c4a80 legacy=0x7fa714158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.982+0000 7fa719971700 1 -- 172.21.15.154:0/2669696182 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7140c6cf0 con 0x7fa714158a80 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa719971700 1 --2- 172.21.15.154:0/2669696182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa714158a80 0x7fa7140c4f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa7140c7870 tx=0x7fa70c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2b3ef3d1b988a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa70bfff700 1 -- 172.21.15.154:0/2669696182 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00e070 con 0x7fa714158a80 2024-03-14T09:02:26.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa70bfff700 1 -- 172.21.15.154:0/2669696182 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c003a20 con 0x7fa714158a80 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa70bfff700 1 -- 172.21.15.154:0/2669696182 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c003dd0 con 0x7fa714158a80 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 -- 172.21.15.154:0/2669696182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa714158a80 msgr2=0x7fa7140c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/2669696182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa714158a80 0x7fa7140c4f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa7140c7870 tx=0x7fa70c00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 -- 172.21.15.154:0/2669696182 shutdown_connections 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/2669696182 >> v2:172.21.15.191:3300/0 conn(0x7fa71414e980 0x7fa7141552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/2669696182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa714158a80 0x7fa7140c4f10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 -- 172.21.15.154:0/2669696182 >> 172.21.15.154:0/2669696182 conn(0x7fa7140abcf0 msgr2=0x7fa7140ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 -- 172.21.15.154:0/2669696182 shutdown_connections 2024-03-14T09:02:26.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.983+0000 7fa71b3d4700 1 -- 172.21.15.154:0/2669696182 wait complete. 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 Processor -- start 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 -- start start 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7140c4a80 0x7fa7140c0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 0x7fa71414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7140be810 con 0x7fa71414e980 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141ee360 con 0x7fa714158a80 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71b3d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa714154af0 con 0x7fa7140c4a80 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71896f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71896f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47880/0 (socket says 172.21.15.154:47880) 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.985+0000 7fa71896f700 1 -- 172.21.15.154:0/3116646293 learned_addr learned my addr 172.21.15.154:0/3116646293 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa719170700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7140c4a80 0x7fa7140c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa719971700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 0x7fa71414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 -- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7140c4a80 msgr2=0x7fa7140c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7140c4a80 0x7fa7140c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 -- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 msgr2=0x7fa71414b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 0x7fa71414b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 -- 172.21.15.154:0/3116646293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c009b30 con 0x7fa71414e980 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa719971700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 0x7fa71414b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71896f700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b4d0 tx=0x7fa71000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710011830 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa710011990 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7140bdc00 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710011d40 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.986+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7140be1c0 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.987+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7140697d0 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.987+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa71001a400 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.988+0000 7fa709ffb700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 0x7fa6f806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.988+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa710088d80 con 0x7fa71414e980 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.988+0000 7fa719170700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 0x7fa6f806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:26.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.988+0000 7fa719170700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 0x7fa6f806ccf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa70400a990 tx=0x7fa704008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:26.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:26.992+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa710053ed0 con 0x7fa71414e980 2024-03-14T09:02:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.182+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7140bd5b0 con 0x7fa71414e980 2024-03-14T09:02:27.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.183+0000 7fa709ffb700 1 -- 172.21.15.154:0/3116646293 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7100150a0 con 0x7fa71414e980 2024-03-14T09:02:27.183 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 msgr2=0x7fa6f806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 0x7fa6f806ccf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa70400a990 tx=0x7fa704008040 comp rx=0 tx=0).stop 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 msgr2=0x7fa7140beca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b4d0 tx=0x7fa71000b890 comp rx=0 tx=0).stop 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 shutdown_connections 2024-03-14T09:02:27.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa714158a80 0x7fa71414b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa7140c4a80 0x7fa7140c0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6f806a830 0x7fa6f806ccf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 --2- 172.21.15.154:0/3116646293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa71414e980 0x7fa7140beca0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 >> 172.21.15.154:0/3116646293 conn(0x7fa7140abcf0 msgr2=0x7fa71406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 shutdown_connections 2024-03-14T09:02:27.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.185+0000 7fa71b3d4700 1 -- 172.21.15.154:0/3116646293 wait complete. 2024-03-14T09:02:27.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:02:27.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.321+0000 7f43dd506700 1 Processor -- start 2024-03-14T09:02:27.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.321+0000 7f43dd506700 1 -- start start 2024-03-14T09:02:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43dd506700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43d80c7ff0 0x7f43d806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43dd506700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43d80c4a70 0x7f43d80c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43dd506700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d80c7550 con 0x7f43d80c6b00 2024-03-14T09:02:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43dd506700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d806c140 con 0x7f43d80c7ff0 2024-03-14T09:02:27.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43dd506700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d814b780 con 0x7f43d80c4a70 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43d80c7ff0 0x7f43d806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d6ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f43d80c6b00 0x7f43d8155190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33792/0 (socket says 172.21.15.154:33792) 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d6ffd700 1 -- 172.21.15.154:0/3459502633 learned_addr learned my addr 172.21.15.154:0/3459502633 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d77fe700 1 --2- 172.21.15.154:0/3459502633 >> v2:172.21.15.191:3300/0 conn(0x7f43d80c4a70 0x7f43d80c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d67fc700 1 -- 172.21.15.154:0/3459502633 >> v2:172.21.15.191:3300/0 conn(0x7f43d80c4a70 msgr2=0x7f43d80c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d67fc700 1 --2- 172.21.15.154:0/3459502633 >> v2:172.21.15.191:3300/0 conn(0x7f43d80c4a70 0x7f43d80c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d67fc700 1 -- 172.21.15.154:0/3459502633 >> v1:172.21.15.154:6789/0 conn(0x7f43d80c6b00 legacy=0x7f43d8155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d67fc700 1 -- 172.21.15.154:0/3459502633 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d80c8450 con 0x7f43d80c7ff0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.322+0000 7f43d5ffb700 1 -- 172.21.15.154:0/3459502633 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353650084 0 0) 0x7f43d80c7550 con 0x7f43d80c6b00 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43d67fc700 1 --2- 172.21.15.154:0/3459502633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d806c5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43d8155980 tx=0x7f43c8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1152e2706bd55971 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43d5ffb700 1 -- 172.21.15.154:0/3459502633 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8011830 con 0x7f43d80c7ff0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43d5ffb700 1 -- 172.21.15.154:0/3459502633 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8011990 con 0x7f43d80c7ff0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43d5ffb700 1 -- 172.21.15.154:0/3459502633 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8011d20 con 0x7f43d80c7ff0 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 -- 172.21.15.154:0/3459502633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 msgr2=0x7f43d806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 --2- 172.21.15.154:0/3459502633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d806c5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f43d8155980 tx=0x7f43c8009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 -- 172.21.15.154:0/3459502633 shutdown_connections 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 --2- 172.21.15.154:0/3459502633 >> v2:172.21.15.191:3300/0 conn(0x7f43d80c4a70 0x7f43d80c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 --2- 172.21.15.154:0/3459502633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d806c5d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 -- 172.21.15.154:0/3459502633 >> 172.21.15.154:0/3459502633 conn(0x7f43d80abc80 msgr2=0x7f43d80ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 -- 172.21.15.154:0/3459502633 shutdown_connections 2024-03-14T09:02:27.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.323+0000 7f43dd506700 1 -- 172.21.15.154:0/3459502633 wait complete. 2024-03-14T09:02:27.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 Processor -- start 2024-03-14T09:02:27.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 -- start start 2024-03-14T09:02:27.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 0x7f43d8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 0x7f43d80aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81530e0 con 0x7f43d80c7ff0 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80afc80 con 0x7f43d814a7c0 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.325+0000 7f43dd506700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8153a40 con 0x7f43d80c4a70 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47900/0 (socket says 172.21.15.154:47900) 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 -- 172.21.15.154:0/2954325096 learned_addr learned my addr 172.21.15.154:0/2954325096 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d77fe700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 0x7f43d80aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d6ffd700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 0x7f43d8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 -- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 msgr2=0x7f43d8153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 0x7f43d8153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 -- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 msgr2=0x7f43d80aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 0x7f43d80aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 -- 172.21.15.154:0/2954325096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80094a0 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d6ffd700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 0x7f43d8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d77fe700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 0x7f43d80aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43d67fc700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f43c8014040 tx=0x7f43c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c80038a0 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c801f070 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d81e9c80 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8003c00 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.326+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81ea240 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.327+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d80697d0 con 0x7f43d80c7ff0 2024-03-14T09:02:27.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.328+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f43c801a430 con 0x7f43d80c7ff0 2024-03-14T09:02:27.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.328+0000 7f43bf7fe700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 0x7f43c406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.328+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f43c8021070 con 0x7f43d80c7ff0 2024-03-14T09:02:27.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.329+0000 7f43d6ffd700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 0x7f43c406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.329+0000 7f43d6ffd700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 0x7f43c406ccf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000f60 tx=0x7f43cc007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.333+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f43c8055a50 con 0x7f43d80c7ff0 2024-03-14T09:02:27.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.528+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43d80c7730 con 0x7f43d80c7ff0 2024-03-14T09:02:27.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.528+0000 7f43bf7fe700 1 -- 172.21.15.154:0/2954325096 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v36) v1 ==== 105+0+11 (secure 0 0 0) 0x7f43c8059070 con 0x7f43d80c7ff0 2024-03-14T09:02:27.528 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:27.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 msgr2=0x7f43c406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 0x7f43c406ccf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000f60 tx=0x7f43cc007cc0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 msgr2=0x7f43d8149eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f43c8014040 tx=0x7f43c8004750 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 shutdown_connections 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43d814a7c0 0x7f43d80aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43d80c4a70 0x7f43d8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f43c406a830 0x7f43c406ccf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 --2- 172.21.15.154:0/2954325096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43d80c7ff0 0x7f43d8149eb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 >> 172.21.15.154:0/2954325096 conn(0x7f43d80abc80 msgr2=0x7f43d80ab200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 shutdown_connections 2024-03-14T09:02:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.530+0000 7f43dd506700 1 -- 172.21.15.154:0/2954325096 wait complete. 2024-03-14T09:02:27.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:02:27.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.667+0000 7f7117b33700 1 Processor -- start 2024-03-14T09:02:27.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.668+0000 7f7117b33700 1 -- start start 2024-03-14T09:02:27.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.668+0000 7f7117b33700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71100a9650 0x7f71100c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.668+0000 7f7117b33700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 0x7f711006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f7117b33700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7110156050 con 0x7f7110156960 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f7117b33700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100a86f0 con 0x7f71100a9650 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f7117b33700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c34b0 con 0x7f71101564e0 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71100a9650 0x7f71100c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71160d0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7110156960 0x7f7110152690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33796/0 (socket says 172.21.15.154:33796) 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71160d0700 1 -- 172.21.15.154:0/3996189292 learned_addr learned my addr 172.21.15.154:0/3996189292 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71150ce700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 0x7f711006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 -- 172.21.15.154:0/3996189292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 msgr2=0x7f711006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 0x7f711006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 -- 172.21.15.154:0/3996189292 >> v1:172.21.15.154:6789/0 conn(0x7f7110156960 legacy=0x7f7110152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 -- 172.21.15.154:0/3996189292 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100a8990 con 0x7f71100a9650 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71148cd700 1 -- 172.21.15.154:0/3996189292 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037884642 0 0) 0x7f7110156050 con 0x7f7110156960 2024-03-14T09:02:27.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71150ce700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 0x7f711006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71158cf700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c1780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f7104009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e400f8a067857a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71148cd700 1 -- 172.21.15.154:0/3996189292 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011830 con 0x7f71100a9650 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71148cd700 1 -- 172.21.15.154:0/3996189292 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104011990 con 0x7f71100a9650 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.669+0000 7f71148cd700 1 -- 172.21.15.154:0/3996189292 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011d20 con 0x7f71100a9650 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 -- 172.21.15.154:0/3996189292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 msgr2=0x7f71100c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c1780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f7104009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 -- 172.21.15.154:0/3996189292 shutdown_connections 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c1780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 --2- 172.21.15.154:0/3996189292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71101564e0 0x7f711006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 -- 172.21.15.154:0/3996189292 >> 172.21.15.154:0/3996189292 conn(0x7f71100c3880 msgr2=0x7f71100c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 -- 172.21.15.154:0/3996189292 shutdown_connections 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.670+0000 7f7117b33700 1 -- 172.21.15.154:0/3996189292 wait complete. 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.671+0000 7f7117b33700 1 Processor -- start 2024-03-14T09:02:27.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.671+0000 7f7117b33700 1 -- start start 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71101564e0 0x7f711014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7110156960 0x7f71101444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c5560 con 0x7f71100a9650 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101ebe80 con 0x7f71101564e0 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c6480 con 0x7f7110156960 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47930/0 (socket says 172.21.15.154:47930) 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 -- 172.21.15.154:0/2721260693 learned_addr learned my addr 172.21.15.154:0/2721260693 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71160d0700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7110156960 0x7f71101444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71150ce700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71101564e0 0x7f711014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 -- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7110156960 msgr2=0x7f71101444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7110156960 0x7f71101444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 -- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71101564e0 msgr2=0x7f711014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71101564e0 0x7f711014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 -- 172.21.15.154:0/2721260693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040094a0 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71158cf700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7104014040 tx=0x7f7104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040045f0 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710401e070 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71100c5280 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104003b00 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.672+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101ebaa0 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.673+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71101eb620 con 0x7f71100a9650 2024-03-14T09:02:27.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.674+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7104003c60 con 0x7f71100a9650 2024-03-14T09:02:27.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.674+0000 7f71027fc700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 0x7f70f406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:27.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.674+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f710408a1c0 con 0x7f71100a9650 2024-03-14T09:02:27.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.674+0000 7f71150ce700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 0x7f70f406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:27.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.675+0000 7f71150ce700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 0x7f70f406cd20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f710c006fd0 tx=0x7f710c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:27.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.679+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f71101eb620 con 0x7f71100a9650 2024-03-14T09:02:27.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.874+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f71101ebfc0 con 0x7f71100a9650 2024-03-14T09:02:27.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.874+0000 7f71027fc700 1 -- 172.21.15.154:0/2721260693 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v36) v1 ==== 101+0+11 (secure 0 0 0) 0x7f7104020020 con 0x7f71100a9650 2024-03-14T09:02:27.874 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 msgr2=0x7f70f406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 0x7f70f406cd20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f710c006fd0 tx=0x7f710c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 msgr2=0x7f71100c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7104014040 tx=0x7f7104004750 comp rx=0 tx=0).stop 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 shutdown_connections 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7110156960 0x7f71101444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f70f406a860 0x7f70f406cd20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100a9650 0x7f71100c4b80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 --2- 172.21.15.154:0/2721260693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71101564e0 0x7f711014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:27.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 >> 172.21.15.154:0/2721260693 conn(0x7f71100c3880 msgr2=0x7f711006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:27.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 shutdown_connections 2024-03-14T09:02:27.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:27.877+0000 7f7117b33700 1 -- 172.21.15.154:0/2721260693 wait complete. 2024-03-14T09:02:27.889 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:02:28.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.012+0000 7f55fc6a5700 1 Processor -- start 2024-03-14T09:02:28.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.012+0000 7f55fc6a5700 1 -- start start 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55fc6a5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f55f40c4a80 0x7f55f40c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55fc6a5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f55f4158370 0x7f55f4158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55fc6a5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f414ed60 con 0x7f55f414e980 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55fc6a5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40c6b10 con 0x7f55f40c4a80 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55fc6a5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40c6cf0 con 0x7f55f4158370 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55f9c40700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f55f40c4a80 0x7f55f40c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55f9c40700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f55f40c4a80 0x7f55f40c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47934/0 (socket says 172.21.15.154:47934) 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.013+0000 7f55f9c40700 1 -- 172.21.15.154:0/3718928715 learned_addr learned my addr 172.21.15.154:0/3718928715 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:28.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55fac42700 1 --2- 172.21.15.154:0/3718928715 >> v2:172.21.15.191:3300/0 conn(0x7f55f4158370 0x7f55f4158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f9c40700 1 -- 172.21.15.154:0/3718928715 >> v2:172.21.15.191:3300/0 conn(0x7f55f4158370 msgr2=0x7f55f4158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f9c40700 1 --2- 172.21.15.154:0/3718928715 >> v2:172.21.15.191:3300/0 conn(0x7f55f4158370 0x7f55f4158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f9c40700 1 -- 172.21.15.154:0/3718928715 >> v1:172.21.15.191:6789/0 conn(0x7f55f414e980 legacy=0x7f55f41552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f9c40700 1 -- 172.21.15.154:0/3718928715 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40c6fd0 con 0x7f55f40c4a80 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f9c40700 1 --2- 172.21.15.154:0/3718928715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f40c4a80 0x7f55f40c4f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55f4152430 tx=0x7f55f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c429697be4fae94 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f943f700 1 -- 172.21.15.154:0/3718928715 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000b870 con 0x7f55f40c4a80 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f943f700 1 -- 172.21.15.154:0/3718928715 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000b9d0 con 0x7f55f40c4a80 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55f943f700 1 -- 172.21.15.154:0/3718928715 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000bd60 con 0x7f55f40c4a80 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3718928715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f40c4a80 msgr2=0x7f55f40c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.014+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3718928715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f40c4a80 0x7f55f40c4f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55f4152430 tx=0x7f55f0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3718928715 shutdown_connections 2024-03-14T09:02:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3718928715 >> v2:172.21.15.191:3300/0 conn(0x7f55f4158370 0x7f55f4158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3718928715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f40c4a80 0x7f55f40c4f10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3718928715 >> 172.21.15.154:0/3718928715 conn(0x7f55f40abcf0 msgr2=0x7f55f40ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3718928715 shutdown_connections 2024-03-14T09:02:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.015+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3718928715 wait complete. 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 Processor -- start 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 -- start start 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 0x7f55f40c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55f414e980 0x7f55f40beca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40bdc00 con 0x7f55f4158370 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40bdee0 con 0x7f55f414e980 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.016+0000 7f55fc6a5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40be1c0 con 0x7f55f40c4a80 2024-03-14T09:02:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47950/0 (socket says 172.21.15.154:47950) 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 -- 172.21.15.154:0/3497603714 learned_addr learned my addr 172.21.15.154:0/3497603714 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fa441700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 0x7f55f40c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55f9c40700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55f414e980 0x7f55f40beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 -- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 msgr2=0x7f55f40c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 0x7f55f40c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 -- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55f414e980 msgr2=0x7f55f40beca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55f414e980 0x7f55f40beca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 -- 172.21.15.154:0/3497603714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009550 con 0x7f55f4158370 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fa441700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 0x7f55f40c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fac42700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f55ec007e60 tx=0x7f55ec00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00ede0 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec014420 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40be4a0 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0147d0 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.017+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40c0680 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.018+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f40be9c0 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.019+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f55ec014930 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.019+0000 7f55eaffd700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 0x7f55d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.019+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f55ec0896e0 con 0x7f55f4158370 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.019+0000 7f55fa441700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 0x7f55d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.020+0000 7f55fa441700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 0x7f55d806ccf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55f4156670 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.023+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f55f40be9c0 con 0x7f55f4158370 2024-03-14T09:02:28.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.232+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f55f40be9c0 con 0x7f55f4158370 2024-03-14T09:02:28.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.233+0000 7f55eaffd700 1 -- 172.21.15.154:0/3497603714 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f55f40be9c0 con 0x7f55f4158370 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 msgr2=0x7f55d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 0x7f55d806ccf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55f4156670 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 msgr2=0x7f55f40bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f55ec007e60 tx=0x7f55ec00e3c0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 shutdown_connections 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f55d806a830 0x7f55d806ccf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55f4158370 0x7f55f40bf1e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55f40c4a80 0x7f55f40c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 --2- 172.21.15.154:0/3497603714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55f414e980 0x7f55f40beca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 >> 172.21.15.154:0/3497603714 conn(0x7f55f40abcf0 msgr2=0x7f55f414cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 shutdown_connections 2024-03-14T09:02:28.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.235+0000 7f55fc6a5700 1 -- 172.21.15.154:0/3497603714 wait complete. 2024-03-14T09:02:28.236 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:02:28.247 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:02:28.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.247 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:28.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:02:28.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.297 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:02:28.315 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:28.316 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:28.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:02:28.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:02:28.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.496+0000 7f165a177700 1 Processor -- start 2024-03-14T09:02:28.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.496+0000 7f165a177700 1 -- start start 2024-03-14T09:02:28.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f165a177700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f165414fbd0 0x7f165415df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f165a177700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540acd10 con 0x7f16540702f0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f165a177700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540ac9c0 con 0x7f16540c4380 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f165a177700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16540ac7b0 con 0x7f165414fbd0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16537fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f16540702f0 0x7f16540c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33804/0 (socket says 172.21.15.154:33804) 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16537fe700 1 -- 172.21.15.154:0/3857119109 learned_addr learned my addr 172.21.15.154:0/3857119109 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f1653fff700 1 --2- 172.21.15.154:0/3857119109 >> v2:172.21.15.191:3300/0 conn(0x7f165414fbd0 0x7f165415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473371854 0 0) 0x7f16540acd10 con 0x7f16540702f0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16400035b0 con 0x7f16540702f0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589988863 0 0) 0x7f16540ac9c0 con 0x7f16540c4380 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.497+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16540acd10 con 0x7f16540c4380 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f1653fff700 1 -- 172.21.15.154:0/3857119109 >> v1:172.21.15.191:6789/0 conn(0x7f16540c4380 legacy=0x7f16540c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f1653fff700 1 -- 172.21.15.154:0/3857119109 >> v1:172.21.15.154:6789/0 conn(0x7f16540702f0 legacy=0x7f16540c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2202426643 0 0) 0x7f16400035b0 con 0x7f16540702f0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f1653fff700 1 -- 172.21.15.154:0/3857119109 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540707f0 con 0x7f165414fbd0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f1653fff700 1 --2- 172.21.15.154:0/3857119109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f165414fbd0 0x7f165415df60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f164c009d90 tx=0x7f164c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3628cd5908e0a194 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c00e070 con 0x7f165414fbd0 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c003a20 con 0x7f165414fbd0 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f16527fc700 1 -- 172.21.15.154:0/3857119109 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c001670 con 0x7f165414fbd0 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f165a177700 1 -- 172.21.15.154:0/3857119109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f165414fbd0 msgr2=0x7f165415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f165a177700 1 --2- 172.21.15.154:0/3857119109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f165414fbd0 0x7f165415df60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f164c009d90 tx=0x7f164c00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f165a177700 1 -- 172.21.15.154:0/3857119109 shutdown_connections 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f165a177700 1 --2- 172.21.15.154:0/3857119109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f165414fbd0 0x7f165415df60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.498+0000 7f165a177700 1 -- 172.21.15.154:0/3857119109 >> 172.21.15.154:0/3857119109 conn(0x7f16540a9bf0 msgr2=0x7f16540a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.499+0000 7f165a177700 1 -- 172.21.15.154:0/3857119109 shutdown_connections 2024-03-14T09:02:28.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.499+0000 7f165a177700 1 -- 172.21.15.154:0/3857119109 wait complete. 2024-03-14T09:02:28.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.500+0000 7f165a177700 1 Processor -- start 2024-03-14T09:02:28.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.500+0000 7f165a177700 1 -- start start 2024-03-14T09:02:28.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.500+0000 7f165a177700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16540702f0 0x7f16541ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16540c4380 0x7f1654152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654152460 con 0x7f165414fbd0 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16541ec2e0 con 0x7f16540702f0 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16541533a0 con 0x7f16540c4380 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47962/0 (socket says 172.21.15.154:47962) 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 -- 172.21.15.154:0/1742311966 learned_addr learned my addr 172.21.15.154:0/1742311966 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f16537fe700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16540702f0 0x7f16541ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 -- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16540c4380 msgr2=0x7f1654152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16540c4380 0x7f1654152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 -- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16540702f0 msgr2=0x7f16541ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16540702f0 0x7f16541ebaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 -- 172.21.15.154:0/1742311966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c009a70 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f1653fff700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f164c015040 tx=0x7f164c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c004600 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c004760 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16541507f0 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c012670 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.501+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654145a20 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.502+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540697d0 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.503+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f164c0127d0 con 0x7f165414fbd0 2024-03-14T09:02:28.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.503+0000 7f164bfff700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 0x7f163406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:28.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.503+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f164c08a730 con 0x7f165414fbd0 2024-03-14T09:02:28.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.503+0000 7f16537fe700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 0x7f163406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:28.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.504+0000 7f16537fe700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 0x7f163406ccf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f163c001860 tx=0x7f163c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:28.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.508+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f164c055800 con 0x7f165414fbd0 2024-03-14T09:02:28.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.748+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f165414ca30 con 0x7f165414fbd0 2024-03-14T09:02:28.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.748+0000 7f164bfff700 1 -- 172.21.15.154:0/1742311966 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+149 (secure 0 0 0) 0x7f164c058e20 con 0x7f165414fbd0 2024-03-14T09:02:28.750 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:02:28.750 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:02:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 msgr2=0x7f163406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 0x7f163406ccf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f163c001860 tx=0x7f163c009040 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 msgr2=0x7f1654152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f164c015040 tx=0x7f164c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 shutdown_connections 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f163406a830 0x7f163406ccf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f165414fbd0 0x7f1654152e60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16540702f0 0x7f16541ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 --2- 172.21.15.154:0/1742311966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16540c4380 0x7f1654152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 >> 172.21.15.154:0/1742311966 conn(0x7f16540a9bf0 msgr2=0x7f165414eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 shutdown_connections 2024-03-14T09:02:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:28.750+0000 7f165a177700 1 -- 172.21.15.154:0/1742311966 wait complete. 2024-03-14T09:02:28.762 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:02:28.762 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:02:28.763 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:28.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.763 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:02:28.784 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:28.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42315sec preferred_lft 42315sec 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:02:28.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:02:28.787 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:02:28.787 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:28.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.787 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:02:28.853 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:02:28.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.855 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:02:28.915 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:02:28.915 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:02:28.915 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:02:28.915 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:02:28.915 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:02:28.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:02:28.916 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:02:29.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:29.026 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:02:29.044 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:02:29.044 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:29.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:29.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:02:29.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:29.111 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:29.170 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:02:29.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:29.170 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:29.226 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-03-14T09:02:29.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:29.227 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:29.313 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:29.313 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:29.332 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:29.332 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:29.332 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:02:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:29.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:29.351 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:29.352 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:29.352 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:02:29.352 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:02:29.352 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:02:29.354 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:02:29.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:29.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:29.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:29.382 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:02:29.471 INFO:teuthology.orchestra.run.smithi191.stdout:client5104 2024-03-14T09:02:29.471 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:29.471 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:29.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs authorize cephfs client.testuser / rw 2024-03-14T09:02:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.626+0000 7f796b402700 1 Processor -- start 2024-03-14T09:02:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 -- start start 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7964158920 0x7f7964155360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79640c1380 0x7f79640c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7964157ed0 con 0x7f796406d4d0 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79641558a0 con 0x7f79640c1380 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.627+0000 7f796b402700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7964158490 con 0x7f7964158920 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79640c1380 0x7f79640c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79640c1380 0x7f79640c17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44568/0 (socket says 172.21.15.154:44568) 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 -- 172.21.15.154:0/4001211661 learned_addr learned my addr 172.21.15.154:0/4001211661 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796899d700 1 --2- 172.21.15.154:0/4001211661 >> v2:172.21.15.191:3300/0 conn(0x7f7964158920 0x7f7964155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 -- 172.21.15.154:0/4001211661 >> v2:172.21.15.191:3300/0 conn(0x7f7964158920 msgr2=0x7f7964155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 --2- 172.21.15.154:0/4001211661 >> v2:172.21.15.191:3300/0 conn(0x7f7964158920 0x7f7964155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 -- 172.21.15.154:0/4001211661 >> v1:172.21.15.191:6789/0 conn(0x7f796406d4d0 legacy=0x7f796406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 -- 172.21.15.154:0/4001211661 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796406d8b0 con 0x7f79640c1380 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f796999f700 1 --2- 172.21.15.154:0/4001211661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79640c1380 0x7f79640c17d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f79640c4a20 tx=0x7f795c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f03360ffbcdba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f795bfff700 1 -- 172.21.15.154:0/4001211661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00e070 con 0x7f79640c1380 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f795bfff700 1 -- 172.21.15.154:0/4001211661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795c00b910 con 0x7f79640c1380 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.628+0000 7f795bfff700 1 -- 172.21.15.154:0/4001211661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c00bcc0 con 0x7f79640c1380 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 -- 172.21.15.154:0/4001211661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79640c1380 msgr2=0x7f79640c17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 --2- 172.21.15.154:0/4001211661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79640c1380 0x7f79640c17d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f79640c4a20 tx=0x7f795c00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 -- 172.21.15.154:0/4001211661 shutdown_connections 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 --2- 172.21.15.154:0/4001211661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79640c1380 0x7f79640c17d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 --2- 172.21.15.154:0/4001211661 >> v2:172.21.15.191:3300/0 conn(0x7f7964158920 0x7f7964155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 -- 172.21.15.154:0/4001211661 >> 172.21.15.154:0/4001211661 conn(0x7f79640a9d00 msgr2=0x7f79640aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 -- 172.21.15.154:0/4001211661 shutdown_connections 2024-03-14T09:02:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.629+0000 7f796b402700 1 -- 172.21.15.154:0/4001211661 wait complete. 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.630+0000 7f796b402700 1 Processor -- start 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.630+0000 7f796b402700 1 -- start start 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79640c1380 0x7f7964151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7964158920 0x7f796406f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964148e50 con 0x7f796406d4d0 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79641ea940 con 0x7f7964158920 2024-03-14T09:02:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796b402700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796414f290 con 0x7f79640c1380 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44578/0 (socket says 172.21.15.154:44578) 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 -- 172.21.15.154:0/2080418866 learned_addr learned my addr 172.21.15.154:0/2080418866 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796999f700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7964158920 0x7f796406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796899d700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79640c1380 0x7f7964151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 -- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79640c1380 msgr2=0x7f7964151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79640c1380 0x7f7964151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 -- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7964158920 msgr2=0x7f796406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7964158920 0x7f796406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 -- 172.21.15.154:0/2080418866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c009b30 con 0x7f796406d4d0 2024-03-14T09:02:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f796919e700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f795400c4e0 tx=0x7f795400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.631+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540052d0 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.632+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954005430 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.632+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79641eabb0 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.632+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540038c0 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.632+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796406f980 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.633+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79640697d0 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.633+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7954003a20 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.633+0000 7f7959ffb700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 0x7f794806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.633+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f79540884e0 con 0x7f796406d4d0 2024-03-14T09:02:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.634+0000 7f796899d700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 0x7f794806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:29.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.634+0000 7f796899d700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 0x7f794806ccf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7960009b40 tx=0x7f7960009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:29.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.638+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7954053520 con 0x7f796406d4d0 2024-03-14T09:02:29.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.878+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]} v 0) v1 -- 0x7f79640c2550 con 0x7f796406d4d0 2024-03-14T09:02:29.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.881+0000 7f7959ffb700 1 -- 172.21.15.154:0/2080418866 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.testuser", "caps": ["/", "rw"]}]=0 v17) v1 ==== 134+0+66 (secure 0 0 0) 0x7f7954056b40 con 0x7f796406d4d0 2024-03-14T09:02:29.881 INFO:teuthology.orchestra.run.smithi154.stdout:[client.testuser] 2024-03-14T09:02:29.881 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg== 2024-03-14T09:02:29.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 msgr2=0x7f794806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 0x7f794806ccf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7960009b40 tx=0x7f7960009210 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 msgr2=0x7f79641eb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f795400c4e0 tx=0x7f795400c7f0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 shutdown_connections 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7964158920 0x7f796406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f794806a830 0x7f794806ccf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f796406d4d0 0x7f79641eb3b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 --2- 172.21.15.154:0/2080418866 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79640c1380 0x7f7964151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 >> 172.21.15.154:0/2080418866 conn(0x7f79640a9d00 msgr2=0x7f79640c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 shutdown_connections 2024-03-14T09:02:29.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:29.883+0000 7f796b402700 1 -- 172.21.15.154:0/2080418866 wait complete. 2024-03-14T09:02:29.895 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:02:29.912 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.neWYTGXbY1 2024-03-14T09:02:29.913 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:29.913 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/tmp/tmp.neWYTGXbY1 2024-03-14T09:02:29.969 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:29.969 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:30.027 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:30.027 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:30.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.247+0000 7f8ae9cd1700 1 Processor -- start 2024-03-14T09:02:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.247+0000 7f8ae9cd1700 1 -- start start 2024-03-14T09:02:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae9cd1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ae4070850 0x7f8ae414fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae9cd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae9cd1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40c1e30 con 0x7f8ae40c22c0 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae9cd1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae4152fc0 con 0x7f8ae4070850 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae9cd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c1820 con 0x7f8ae4072270 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ae4070850 0x7f8ae414fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ae4070850 0x7f8ae414fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47104/0 (socket says 172.21.15.154:47104) 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:43438/0 (socket says 172.21.15.154:43438) 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae37fe700 1 -- 172.21.15.154:0/3893040701 learned_addr learned my addr 172.21.15.154:0/3893040701 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.248+0000 7f8ae3fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8ae40c22c0 0x7f8ae4155b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:47054/0 (socket says 172.21.15.154:47054) 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae37fe700 1 -- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 msgr2=0x7f8ae40726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae37fe700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae37fe700 1 -- 172.21.15.154:0/3893040701 >> v1:172.21.15.191:6789/0 conn(0x7f8ae40c22c0 legacy=0x7f8ae4155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae37fe700 1 -- 172.21.15.154:0/3893040701 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae414fff0 con 0x7f8ae4070850 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae2ffd700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae37fe700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae4070850 0x7f8ae414fab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098f0 tx=0x7f8acc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a0f55760141d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae27fc700 1 -- 172.21.15.154:0/3893040701 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011830 con 0x7f8ae4070850 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae27fc700 1 -- 172.21.15.154:0/3893040701 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc011990 con 0x7f8ae4070850 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.249+0000 7f8ae27fc700 1 -- 172.21.15.154:0/3893040701 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011d20 con 0x7f8ae4070850 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae4070850 msgr2=0x7f8ae414fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae4070850 0x7f8ae414fab0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0098f0 tx=0x7f8acc009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3893040701 shutdown_connections 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae4070850 0x7f8ae414fab0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3893040701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4072270 0x7f8ae40726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3893040701 >> 172.21.15.154:0/3893040701 conn(0x7f8ae406a770 msgr2=0x7f8ae406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3893040701 shutdown_connections 2024-03-14T09:02:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.250+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3893040701 wait complete. 2024-03-14T09:02:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.251+0000 7f8ae9cd1700 1 Processor -- start 2024-03-14T09:02:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.251+0000 7f8ae9cd1700 1 -- start start 2024-03-14T09:02:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4070850 0x7f8ae4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae40c22c0 0x7f8ae40a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae41e8c60 con 0x7f8ae4072270 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae41e8050 con 0x7f8ae40c22c0 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae41e8330 con 0x7f8ae4070850 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44582/0 (socket says 172.21.15.154:44582) 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 -- 172.21.15.154:0/3879650363 learned_addr learned my addr 172.21.15.154:0/3879650363 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae3fff700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae40c22c0 0x7f8ae40a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae37fe700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4070850 0x7f8ae4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 -- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4070850 msgr2=0x7f8ae4155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4070850 0x7f8ae4155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 -- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae40c22c0 msgr2=0x7f8ae40a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae40c22c0 0x7f8ae40a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 -- 172.21.15.154:0/3879650363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40094a0 con 0x7f8ae4072270 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae2ffd700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ebe0 tx=0x7f8ad4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400c910 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400ca70 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc009550 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400cde0 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.252+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae41e88f0 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.253+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae40697d0 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.254+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8ad4013470 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.254+0000 7f8adbfff700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 0x7f8ac406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.254+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8ad4089270 con 0x7f8ae4072270 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.254+0000 7f8ae37fe700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 0x7f8ac406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.255+0000 7f8ae37fe700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 0x7f8ac406ccf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009c40 tx=0x7f8acc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.259+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8ad40543c0 con 0x7f8ae4072270 2024-03-14T09:02:30.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.455+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ae41e8da0 con 0x7f8ae4072270 2024-03-14T09:02:30.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.456+0000 7f8adbfff700 1 -- 172.21.15.154:0/3879650363 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7f8ad40579e0 con 0x7f8ae4072270 2024-03-14T09:02:30.456 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:30.457 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 msgr2=0x7f8ac406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 0x7f8ac406ccf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009c40 tx=0x7f8acc008030 comp rx=0 tx=0).stop 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 msgr2=0x7f8ae4156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400ebe0 tx=0x7f8ad4009ec0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 shutdown_connections 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ae40c22c0 0x7f8ae40a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ae4070850 0x7f8ae4155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ac406a830 0x7f8ac406ccf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 --2- 172.21.15.154:0/3879650363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae4072270 0x7f8ae4156280 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 >> 172.21.15.154:0/3879650363 conn(0x7f8ae406a770 msgr2=0x7f8ae414ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 shutdown_connections 2024-03-14T09:02:30.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.459+0000 7f8ae9cd1700 1 -- 172.21.15.154:0/3879650363 wait complete. 2024-03-14T09:02:30.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:01.699649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:02.688126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:02:19.001797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:02:19.712147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:30.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.596+0000 7f5707b59700 1 Processor -- start 2024-03-14T09:02:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.596+0000 7f5707b59700 1 -- start start 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f5707b59700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57000c6850 0x7f570015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f5707b59700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57000c7bb0 con 0x7f570006c670 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f5707b59700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57000c7e90 con 0x7f57000c31c0 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f5707b59700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57000c3e00 con 0x7f57000c6850 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57060f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57000c6850 0x7f570015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57058f5700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f570006c670 0x7f570006cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58796/0 (socket says 172.21.15.154:58796) 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57058f5700 1 -- 172.21.15.154:0/1358605449 learned_addr learned my addr 172.21.15.154:0/1358605449 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57060f6700 1 -- 172.21.15.154:0/1358605449 >> v1:172.21.15.191:6789/0 conn(0x7f57000c31c0 legacy=0x7f57000c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57060f6700 1 -- 172.21.15.154:0/1358605449 >> v1:172.21.15.154:6789/0 conn(0x7f570006c670 legacy=0x7f570006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57060f6700 1 -- 172.21.15.154:0/1358605449 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57000c8170 con 0x7f57000c6850 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.597+0000 7f57060f6700 1 --2- 172.21.15.154:0/1358605449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57000c6850 0x7f570015fd70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009d90 tx=0x7f56f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15bd5cac2f67764 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f57048f3700 1 -- 172.21.15.154:0/1358605449 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800e070 con 0x7f57000c6850 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f57048f3700 1 -- 172.21.15.154:0/1358605449 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800bb90 con 0x7f57000c6850 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f57048f3700 1 -- 172.21.15.154:0/1358605449 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8003a20 con 0x7f57000c6850 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 -- 172.21.15.154:0/1358605449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57000c6850 msgr2=0x7f570015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 --2- 172.21.15.154:0/1358605449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57000c6850 0x7f570015fd70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009d90 tx=0x7f56f800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 -- 172.21.15.154:0/1358605449 shutdown_connections 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 --2- 172.21.15.154:0/1358605449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57000c6850 0x7f570015fd70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 -- 172.21.15.154:0/1358605449 >> 172.21.15.154:0/1358605449 conn(0x7f57000a9bf0 msgr2=0x7f57000a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 -- 172.21.15.154:0/1358605449 shutdown_connections 2024-03-14T09:02:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.598+0000 7f5707b59700 1 -- 172.21.15.154:0/1358605449 wait complete. 2024-03-14T09:02:30.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.599+0000 7f5707b59700 1 Processor -- start 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 -- start start 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57000c31c0 0x7f57001e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57000c6850 0x7f57001d6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57000abd40 con 0x7f570006c670 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57000a9440 con 0x7f57000c31c0 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f5707b59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57001ea370 con 0x7f57000c6850 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f57058f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f57058f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44600/0 (socket says 172.21.15.154:44600) 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f57058f5700 1 -- 172.21.15.154:0/3781196476 learned_addr learned my addr 172.21.15.154:0/3781196476 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f57060f6700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57000c6850 0x7f57001d6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.600+0000 7f57050f4700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57000c31c0 0x7f57001e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 -- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57000c6850 msgr2=0x7f57001d6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57000c6850 0x7f57001d6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 -- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57000c31c0 msgr2=0x7f57001e9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57000c31c0 0x7f57001e9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 -- 172.21.15.154:0/3781196476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009a70 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f57058f5700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c490 tx=0x7f56f000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0005d70 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00070d0 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57001edf30 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0007460 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.601+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57001d67d0 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.602+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57000a9920 con 0x7f570006c670 2024-03-14T09:02:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.602+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f56f00075c0 con 0x7f570006c670 2024-03-14T09:02:30.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.603+0000 7f56f67fc700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 0x7f56e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.603+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f56f00884e0 con 0x7f570006c670 2024-03-14T09:02:30.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.603+0000 7f57050f4700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 0x7f56e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.603+0000 7f57050f4700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 0x7f56e406ccf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57001ddd10 tx=0x7f56fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.607+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f57000a9920 con 0x7f570006c670 2024-03-14T09:02:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.800+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57000a9920 con 0x7f570006c670 2024-03-14T09:02:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.801+0000 7f56f67fc700 1 -- 172.21.15.154:0/3781196476 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56f0017020 con 0x7f570006c670 2024-03-14T09:02:30.801 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 msgr2=0x7f56e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 0x7f56e406ccf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57001ddd10 tx=0x7f56fc009040 comp rx=0 tx=0).stop 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 msgr2=0x7f57000acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c490 tx=0x7f56f000c850 comp rx=0 tx=0).stop 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 shutdown_connections 2024-03-14T09:02:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57000c6850 0x7f57001d6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f56e406a830 0x7f56e406ccf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f570006c670 0x7f57000acb40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 --2- 172.21.15.154:0/3781196476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57000c31c0 0x7f57001e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 >> 172.21.15.154:0/3781196476 conn(0x7f57000a9bf0 msgr2=0x7f570006f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 shutdown_connections 2024-03-14T09:02:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.803+0000 7f5707b59700 1 -- 172.21.15.154:0/3781196476 wait complete. 2024-03-14T09:02:30.815 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:02:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.940+0000 7f907db85700 1 Processor -- start 2024-03-14T09:02:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.940+0000 7f907db85700 1 -- start start 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f907db85700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9078070a10 0x7f90780c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f907db85700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90780c5f10 con 0x7f907806c6e0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f907db85700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90780c7d50 con 0x7f90780c31c0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f907db85700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90780c8030 con 0x7f9078070a10 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90777fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f907806c6e0 0x7f907806d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58810/0 (socket says 172.21.15.154:58810) 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90777fe700 1 -- 172.21.15.154:0/2201361821 learned_addr learned my addr 172.21.15.154:0/2201361821 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> v2:172.21.15.191:3300/0 conn(0x7f9078070a10 0x7f90780c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97181783 0 0) 0x7f90780c5f10 con 0x7f907806c6e0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90640035b0 con 0x7f907806c6e0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516149583 0 0) 0x7f90640035b0 con 0x7f907806c6e0 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 >> v2:172.21.15.191:3300/0 conn(0x7f9078070a10 msgr2=0x7f90780c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 --2- 172.21.15.154:0/2201361821 >> v2:172.21.15.191:3300/0 conn(0x7f9078070a10 0x7f90780c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 >> v1:172.21.15.191:6789/0 conn(0x7f90780c31c0 legacy=0x7f90780c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780c8310 con 0x7f907806c6e0 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> v2:172.21.15.191:3300/0 conn(0x7f9078070a10 0x7f90780c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9060001670 con 0x7f907806c6e0 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 >> v1:172.21.15.154:6789/0 conn(0x7f907806c6e0 legacy=0x7f907806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9064003a00 0x7f9064003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.941+0000 7f90767fc700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 0x7f9064004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064005210 0x7f906400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780c8030 con 0x7f9064004850 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906400c1b0 con 0x7f9064005210 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906400c370 con 0x7f9064003a00 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 0x7f9064004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9077fff700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064005210 0x7f906400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90777fe700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9064003a00 0x7f9064003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 -- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9064003a00 msgr2=0x7f9064003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9064003a00 0x7f9064003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 -- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064005210 msgr2=0x7f906400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064005210 0x7f906400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 -- 172.21.15.154:0/2201361821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640035b0 con 0x7f9064004850 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9077fff700 1 -- 172.21.15.154:0/2201361821 reap_dead start 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90777fe700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9064003a00 0x7f9064003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f9076ffd700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 0x7f9064004cd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9068009d00 tx=0x7f9064005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2188bd3eaa5e5543 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068009e50 con 0x7f9064004850 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800e9a0 con 0x7f9064004850 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.942+0000 7f90767fc700 1 -- 172.21.15.154:0/2201361821 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800ed30 con 0x7f9064004850 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 -- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 msgr2=0x7f9064004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 0x7f9064004cd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9068009d00 tx=0x7f9064005880 comp rx=0 tx=0).stop 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 -- 172.21.15.154:0/2201361821 shutdown_connections 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 --2- 172.21.15.154:0/2201361821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9064004850 0x7f9064004cd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 -- 172.21.15.154:0/2201361821 >> 172.21.15.154:0/2201361821 conn(0x7f90780a9bf0 msgr2=0x7f90780a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 -- 172.21.15.154:0/2201361821 shutdown_connections 2024-03-14T09:02:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.943+0000 7f907db85700 1 -- 172.21.15.154:0/2201361821 wait complete. 2024-03-14T09:02:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.944+0000 7f907db85700 1 Processor -- start 2024-03-14T09:02:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.944+0000 7f907db85700 1 -- start start 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064004850 0x7f90780abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 0x7f90781db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781dbf10 con 0x7f90780a7140 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781dae10 con 0x7f9064004850 2024-03-14T09:02:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f907db85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780ad160 con 0x7f90781db550 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44610/0 (socket says 172.21.15.154:44610) 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 -- 172.21.15.154:0/3678406540 learned_addr learned my addr 172.21.15.154:0/3678406540 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f90777fe700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064004850 0x7f90780abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9077fff700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 0x7f90781db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 -- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 msgr2=0x7f90781db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 0x7f90781db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 -- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064004850 msgr2=0x7f90780abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064004850 0x7f90780abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 -- 172.21.15.154:0/3678406540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780c8d30 con 0x7f90780a7140 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9077fff700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 0x7f90781db9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f9076ffd700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f90680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.945+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068013e20 con 0x7f90780a7140 2024-03-14T09:02:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.946+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068009e50 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.946+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906801c690 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.946+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780ace80 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.946+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907809aaf0 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.947+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907809ae60 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.947+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9068024070 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.947+0000 7f906ffff700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 0x7f905806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.948+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f906808ab50 con 0x7f90780a7140 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.948+0000 7f90777fe700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 0x7f905806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.948+0000 7f90777fe700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 0x7f905806ccf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9060000c00 tx=0x7f9060007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:30.952+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9068055dc0 con 0x7f90780a7140 2024-03-14T09:02:31.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.147+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90780697d0 con 0x7f90780a7140 2024-03-14T09:02:31.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.147+0000 7f906ffff700 1 -- 172.21.15.154:0/3678406540 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9068017020 con 0x7f90780a7140 2024-03-14T09:02:31.147 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:31.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.149+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 msgr2=0x7f905806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.149+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 0x7f905806ccf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9060000c00 tx=0x7f9060007040 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 msgr2=0x7f90780ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f90680047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 shutdown_connections 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90781db550 0x7f90781db9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9064004850 0x7f90780abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f905806a830 0x7f905806ccf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 --2- 172.21.15.154:0/3678406540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90780a7140 0x7f90780ac1e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 >> 172.21.15.154:0/3678406540 conn(0x7f90780a9bf0 msgr2=0x7f90780c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 shutdown_connections 2024-03-14T09:02:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.150+0000 7f907db85700 1 -- 172.21.15.154:0/3678406540 wait complete. 2024-03-14T09:02:31.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:02:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.287+0000 7f52811b4700 1 Processor -- start 2024-03-14T09:02:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.287+0000 7f52811b4700 1 -- start start 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f52811b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f527c0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f52811b4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c0acd10 con 0x7f527c14fbd0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f52811b4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c150050 con 0x7f527c0c5d60 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f52811b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0ac7b0 con 0x7f527c0702f0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f527a59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f527c14fbd0 0x7f527c0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58822/0 (socket says 172.21.15.154:58822) 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f527a59c700 1 -- 172.21.15.154:0/4202542730 learned_addr learned my addr 172.21.15.154:0/4202542730 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f527ad9d700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f527c0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897503473 0 0) 0x7f527c0acd10 con 0x7f527c14fbd0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.288+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52640035b0 con 0x7f527c14fbd0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411487606 0 0) 0x7f52640035b0 con 0x7f527c14fbd0 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 msgr2=0x7f527c0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f527c0c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 >> v1:172.21.15.191:6789/0 conn(0x7f527c0c5d60 legacy=0x7f527c15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0ac9c0 con 0x7f527c14fbd0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f526c001670 con 0x7f527c14fbd0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 >> v1:172.21.15.154:6789/0 conn(0x7f527c14fbd0 legacy=0x7f527c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f5264003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5264003f70 0x7f5264004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f5264005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0ac7b0 con 0x7f5264004d90 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400ba70 con 0x7f5264003f70 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400bc30 con 0x7f527c0702f0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f5264005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527ad9d700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f5264003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527a59c700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5264003f70 0x7f5264004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 -- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 msgr2=0x7f5264003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c0702f0 0x7f5264003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 -- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5264003f70 msgr2=0x7f5264004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5264003f70 0x7f5264004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 -- 172.21.15.154:0/4202542730 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640035b0 con 0x7f5264004d90 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f527b59e700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f5264005210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f527400c290 tx=0x7f5264005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab326770478b27f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274001e70 con 0x7f5264004d90 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400ce50 con 0x7f5264004d90 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.289+0000 7f5279d9b700 1 -- 172.21.15.154:0/4202542730 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274012660 con 0x7f5264004d90 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 -- 172.21.15.154:0/4202542730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 msgr2=0x7f5264005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 --2- 172.21.15.154:0/4202542730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f5264005210 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f527400c290 tx=0x7f5264005880 comp rx=0 tx=0).stop 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f527b59e700 1 -- 172.21.15.154:0/4202542730 reap_dead start 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 -- 172.21.15.154:0/4202542730 shutdown_connections 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 -- 172.21.15.154:0/4202542730 >> 172.21.15.154:0/4202542730 conn(0x7f527c0a9bf0 msgr2=0x7f527c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 -- 172.21.15.154:0/4202542730 shutdown_connections 2024-03-14T09:02:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.290+0000 7f52811b4700 1 -- 172.21.15.154:0/4202542730 wait complete. 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.291+0000 7f52811b4700 1 Processor -- start 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.291+0000 7f52811b4700 1 -- start start 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f527c1564f0 0x7f527c152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c152a90 0x7f527c1eb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c1507a0 con 0x7f5264004d90 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c153f50 con 0x7f527c1564f0 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f52811b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c14d0c0 con 0x7f527c152a90 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f527ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44626/0 (socket says 172.21.15.154:44626) 2024-03-14T09:02:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f527ad9d700 1 -- 172.21.15.154:0/3520423873 learned_addr learned my addr 172.21.15.154:0/3520423873 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f527a59c700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f527c1564f0 0x7f527c152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.292+0000 7f527b59e700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c152a90 0x7f527c1eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 -- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c152a90 msgr2=0x7f527c1eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c152a90 0x7f527c1eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 -- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f527c1564f0 msgr2=0x7f527c152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f527c1564f0 0x7f527c152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 -- 172.21.15.154:0/3520423873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c151ab0 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f527ad9d700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5270011ad0 tx=0x7f5270011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000f8e0 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527000fa40 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c153c70 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000fdf0 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.293+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c152030 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.294+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c1518f0 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.294+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5270014440 con 0x7f5264004d90 2024-03-14T09:02:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.295+0000 7f52637fe700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 0x7f526806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.295+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f5270089720 con 0x7f5264004d90 2024-03-14T09:02:31.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.295+0000 7f527a59c700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 0x7f526806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.295+0000 7f527a59c700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 0x7f526806ccf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f527c0c2060 tx=0x7f526c000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.299+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5270054820 con 0x7f5264004d90 2024-03-14T09:02:31.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.493+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f527c0697d0 con 0x7f5264004d90 2024-03-14T09:02:31.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.494+0000 7f52637fe700 1 -- 172.21.15.154:0/3520423873 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5270057e40 con 0x7f5264004d90 2024-03-14T09:02:31.494 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 msgr2=0x7f526806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 0x7f526806ccf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f527c0c2060 tx=0x7f526c000f80 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 msgr2=0x7f527c155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5270011ad0 tx=0x7f5270011de0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 shutdown_connections 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f527c152a90 0x7f527c1eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f526806a830 0x7f526806ccf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5264004d90 0x7f527c155fb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 --2- 172.21.15.154:0/3520423873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f527c1564f0 0x7f527c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 >> 172.21.15.154:0/3520423873 conn(0x7f527c0a9bf0 msgr2=0x7f527c0c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 shutdown_connections 2024-03-14T09:02:31.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.496+0000 7f52811b4700 1 -- 172.21.15.154:0/3520423873 wait complete. 2024-03-14T09:02:31.508 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:02:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.632+0000 7f90c17e6700 1 Processor -- start 2024-03-14T09:02:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 -- start start 2024-03-14T09:02:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc158840 0x7f90bc0c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc154f40 con 0x7f90bc1553d0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc154930 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90c17e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c57e0 con 0x7f90bc158840 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90bb7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90bb7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44628/0 (socket says 172.21.15.154:44628) 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.633+0000 7f90bb7fe700 1 -- 172.21.15.154:0/352828681 learned_addr learned my addr 172.21.15.154:0/352828681 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90ba7fc700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc158840 0x7f90bc0c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90bb7fe700 1 -- 172.21.15.154:0/352828681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc158840 msgr2=0x7f90bc0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90bb7fe700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc158840 0x7f90bc0c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90bb7fe700 1 -- 172.21.15.154:0/352828681 >> v1:172.21.15.191:6789/0 conn(0x7f90bc1553d0 legacy=0x7f90bc0c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90bb7fe700 1 -- 172.21.15.154:0/352828681 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc0c5ac0 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90bb7fe700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009e50 tx=0x7f90b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67656dc4ca7efa9a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90b9ffb700 1 -- 172.21.15.154:0/352828681 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b400e070 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90b9ffb700 1 -- 172.21.15.154:0/352828681 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4003a20 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90b9ffb700 1 -- 172.21.15.154:0/352828681 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4003dd0 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90c17e6700 1 -- 172.21.15.154:0/352828681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 msgr2=0x7f90bc0bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.634+0000 7f90c17e6700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009e50 tx=0x7f90b400b330 comp rx=0 tx=0).stop 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 -- 172.21.15.154:0/352828681 shutdown_connections 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc0bfe20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 --2- 172.21.15.154:0/352828681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc158840 0x7f90bc0c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 -- 172.21.15.154:0/352828681 >> 172.21.15.154:0/352828681 conn(0x7f90bc06a770 msgr2=0x7f90bc06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 -- 172.21.15.154:0/352828681 shutdown_connections 2024-03-14T09:02:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.635+0000 7f90c17e6700 1 -- 172.21.15.154:0/352828681 wait complete. 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.636+0000 7f90c17e6700 1 Processor -- start 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.636+0000 7f90c17e6700 1 -- start start 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 0x7f90bc14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 0x7f90bc14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc14e7e0 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc14e3e0 con 0x7f90bc158840 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0a9e80 con 0x7f90bc1553d0 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44642/0 (socket says 172.21.15.154:44642) 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 -- 172.21.15.154:0/3073152786 learned_addr learned my addr 172.21.15.154:0/3073152786 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90ba7fc700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 0x7f90bc14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90bb7fe700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 0x7f90bc14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 -- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 msgr2=0x7f90bc14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 0x7f90bc14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 -- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 msgr2=0x7f90bc14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 0x7f90bc14f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 -- 172.21.15.154:0/3073152786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4009b30 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90ba7fc700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 0x7f90bc14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90bb7fe700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 0x7f90bc14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90baffd700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c4a0 tx=0x7f90b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00054e0 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0003680 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc0aa120 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003a30 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.637+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc06b120 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.638+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0aa360 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.639+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f90b0003b90 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.639+0000 7f90a37fe700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 0x7f90a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.639+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f90b0088570 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.639+0000 7f90ba7fc700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 0x7f90a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.640+0000 7f90ba7fc700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 0x7f90a806ccf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006fd0 tx=0x7f90ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.643+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f90bc0aa360 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.839+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90bc0aa360 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.839+0000 7f90a37fe700 1 -- 172.21.15.154:0/3073152786 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v36) v1 ==== 105+0+11 (secure 0 0 0) 0x7f90bc0aa360 con 0x7f90bc0bf9c0 2024-03-14T09:02:31.839 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 msgr2=0x7f90a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 0x7f90a806ccf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006fd0 tx=0x7f90ac008040 comp rx=0 tx=0).stop 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 msgr2=0x7f90bc150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c4a0 tx=0x7f90b000c860 comp rx=0 tx=0).stop 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 shutdown_connections 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90bc158840 0x7f90bc14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f90a806a830 0x7f90a806ccf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90bc0bf9c0 0x7f90bc150f10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 --2- 172.21.15.154:0/3073152786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90bc1553d0 0x7f90bc14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 >> 172.21.15.154:0/3073152786 conn(0x7f90bc06a770 msgr2=0x7f90bc0c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 shutdown_connections 2024-03-14T09:02:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.841+0000 7f90c17e6700 1 -- 172.21.15.154:0/3073152786 wait complete. 2024-03-14T09:02:31.853 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:02:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.977+0000 7f497ef94700 1 Processor -- start 2024-03-14T09:02:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.977+0000 7f497ef94700 1 -- start start 2024-03-14T09:02:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497ef94700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f49780a9650 0x7f49780c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497ef94700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f49781564e0 0x7f497806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497ef94700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4978156050 con 0x7f4978156960 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497ef94700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780a86f0 con 0x7f49781564e0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497ef94700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780c34b0 con 0x7f49780a9650 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f4977fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f49781564e0 0x7f497806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497d531700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4978156960 0x7f4978152690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58830/0 (socket says 172.21.15.154:58830) 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.978+0000 7f497d531700 1 -- 172.21.15.154:0/827528909 learned_addr learned my addr 172.21.15.154:0/827528909 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f497cd30700 1 --2- 172.21.15.154:0/827528909 >> v2:172.21.15.191:3300/0 conn(0x7f49780a9650 0x7f49780c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f4977fff700 1 -- 172.21.15.154:0/827528909 >> v2:172.21.15.191:3300/0 conn(0x7f49780a9650 msgr2=0x7f49780c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f4977fff700 1 --2- 172.21.15.154:0/827528909 >> v2:172.21.15.191:3300/0 conn(0x7f49780a9650 0x7f49780c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f4977fff700 1 -- 172.21.15.154:0/827528909 >> v1:172.21.15.154:6789/0 conn(0x7f4978156960 legacy=0x7f4978152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f4977fff700 1 -- 172.21.15.154:0/827528909 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780a8990 con 0x7f49781564e0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f49777fe700 1 -- 172.21.15.154:0/827528909 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160811680 0 0) 0x7f4978156050 con 0x7f4978156960 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f4977fff700 1 --2- 172.21.15.154:0/827528909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49781564e0 0x7f497806a770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4978070da0 tx=0x7f4964009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1c35975b965d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f49777fe700 1 -- 172.21.15.154:0/827528909 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011830 con 0x7f49781564e0 2024-03-14T09:02:31.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f49777fe700 1 -- 172.21.15.154:0/827528909 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964011990 con 0x7f49781564e0 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f49777fe700 1 -- 172.21.15.154:0/827528909 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964011d20 con 0x7f49781564e0 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f497ef94700 1 -- 172.21.15.154:0/827528909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49781564e0 msgr2=0x7f497806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.979+0000 7f497ef94700 1 --2- 172.21.15.154:0/827528909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49781564e0 0x7f497806a770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4978070da0 tx=0x7f4964009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 -- 172.21.15.154:0/827528909 shutdown_connections 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 --2- 172.21.15.154:0/827528909 >> v2:172.21.15.191:3300/0 conn(0x7f49780a9650 0x7f49780c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 --2- 172.21.15.154:0/827528909 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49781564e0 0x7f497806a770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 -- 172.21.15.154:0/827528909 >> 172.21.15.154:0/827528909 conn(0x7f49780c3880 msgr2=0x7f49780c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 -- 172.21.15.154:0/827528909 shutdown_connections 2024-03-14T09:02:31.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.980+0000 7f497ef94700 1 -- 172.21.15.154:0/827528909 wait complete. 2024-03-14T09:02:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.981+0000 7f497ef94700 1 Processor -- start 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.981+0000 7f497ef94700 1 -- start start 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.981+0000 7f497ef94700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49781564e0 0x7f4978145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4978156960 0x7f49780c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497813d0c0 con 0x7f49780a9650 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781eb8e0 con 0x7f4978156960 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c6320 con 0x7f49781564e0 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44668/0 (socket says 172.21.15.154:44668) 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 -- 172.21.15.154:0/3481955925 learned_addr learned my addr 172.21.15.154:0/3481955925 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 -- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49781564e0 msgr2=0x7f4978145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49781564e0 0x7f4978145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 -- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4978156960 msgr2=0x7f49780c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4978156960 0x7f49780c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 -- 172.21.15.154:0/3481955925 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640094a0 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497cd30700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d800 tx=0x7f496c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c0125f0 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c012750 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49781428a0 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c012b00 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.982+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781ebb80 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.983+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49780c4500 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.984+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f496c012c60 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.984+0000 7f49757fa700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 0x7f495c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.984+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f496c0893c0 con 0x7f49780a9650 2024-03-14T09:02:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.984+0000 7f4977fff700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 0x7f495c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:31.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.985+0000 7f4977fff700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 0x7f495c06cd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4978140a50 tx=0x7f4964010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:31.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:31.989+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f496c054510 con 0x7f49780a9650 2024-03-14T09:02:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.184+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f49780697d0 con 0x7f49780a9650 2024-03-14T09:02:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.184+0000 7f49757fa700 1 -- 172.21.15.154:0/3481955925 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v36) v1 ==== 101+0+11 (secure 0 0 0) 0x7f496c057b30 con 0x7f49780a9650 2024-03-14T09:02:32.184 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 msgr2=0x7f495c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 0x7f495c06cd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4978140a50 tx=0x7f4964010000 comp rx=0 tx=0).stop 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 msgr2=0x7f497813cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f496c00d800 tx=0x7f496c00dbc0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 shutdown_connections 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4978156960 0x7f49780c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f495c06a860 0x7f495c06cd20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49780a9650 0x7f497813cb80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 --2- 172.21.15.154:0/3481955925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49781564e0 0x7f4978145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 >> 172.21.15.154:0/3481955925 conn(0x7f49780c3880 msgr2=0x7f49780aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 shutdown_connections 2024-03-14T09:02:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.186+0000 7f497ef94700 1 -- 172.21.15.154:0/3481955925 wait complete. 2024-03-14T09:02:32.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:02:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.321+0000 7f4f337fa700 1 Processor -- start 2024-03-14T09:02:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.321+0000 7f4f337fa700 1 -- start start 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f337fa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4f2c0702f0 0x7f4f2c0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f337fa700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c0acd10 con 0x7f4f2c14fbd0 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f337fa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c150050 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f337fa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c0ac7b0 con 0x7f4f2c0702f0 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f30d95700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4f2c14fbd0 0x7f4f2c0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58840/0 (socket says 172.21.15.154:58840) 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f30d95700 1 -- 172.21.15.154:0/1550661243 learned_addr learned my addr 172.21.15.154:0/1550661243 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f31596700 1 --2- 172.21.15.154:0/1550661243 >> v2:172.21.15.191:3300/0 conn(0x7f4f2c0702f0 0x7f4f2c0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f23fff700 1 -- 172.21.15.154:0/1550661243 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857883251 0 0) 0x7f4f2c0acd10 con 0x7f4f2c14fbd0 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f23fff700 1 -- 172.21.15.154:0/1550661243 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f180035b0 con 0x7f4f2c14fbd0 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f31596700 1 -- 172.21.15.154:0/1550661243 >> v1:172.21.15.191:6789/0 conn(0x7f4f2c0c5d60 legacy=0x7f4f2c15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f31596700 1 -- 172.21.15.154:0/1550661243 >> v1:172.21.15.154:6789/0 conn(0x7f4f2c14fbd0 legacy=0x7f4f2c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f31596700 1 -- 172.21.15.154:0/1550661243 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c0ac9c0 con 0x7f4f2c0702f0 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.322+0000 7f4f31596700 1 --2- 172.21.15.154:0/1550661243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c0702f0 0x7f4f2c0c4380 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009840 tx=0x7f4f1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf800f7d62f48466 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f23fff700 1 -- 172.21.15.154:0/1550661243 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011830 con 0x7f4f2c0702f0 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f23fff700 1 -- 172.21.15.154:0/1550661243 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c011990 con 0x7f4f2c0702f0 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f23fff700 1 -- 172.21.15.154:0/1550661243 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011d20 con 0x7f4f2c0702f0 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 -- 172.21.15.154:0/1550661243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c0702f0 msgr2=0x7f4f2c0c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1550661243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c0702f0 0x7f4f2c0c4380 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009840 tx=0x7f4f1c009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 -- 172.21.15.154:0/1550661243 shutdown_connections 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1550661243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c0702f0 0x7f4f2c0c4380 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 -- 172.21.15.154:0/1550661243 >> 172.21.15.154:0/1550661243 conn(0x7f4f2c0a9bf0 msgr2=0x7f4f2c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 -- 172.21.15.154:0/1550661243 shutdown_connections 2024-03-14T09:02:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.323+0000 7f4f337fa700 1 -- 172.21.15.154:0/1550661243 wait complete. 2024-03-14T09:02:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.324+0000 7f4f337fa700 1 Processor -- start 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.324+0000 7f4f337fa700 1 -- start start 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f2c0702f0 0x7f4f2c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 0x7f4f2c1e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c1e9580 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c155ca0 con 0x7f4f2c14fbd0 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f337fa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c1ec400 con 0x7f4f2c0702f0 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44670/0 (socket says 172.21.15.154:44670) 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 -- 172.21.15.154:0/1333920365 learned_addr learned my addr 172.21.15.154:0/1333920365 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f31596700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f2c0702f0 0x7f4f2c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f31d97700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 0x7f4f2c1e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 -- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f2c0702f0 msgr2=0x7f4f2c144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f2c0702f0 0x7f4f2c144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 -- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 msgr2=0x7f4f2c1e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 0x7f4f2c1e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f30d95700 1 -- 172.21.15.154:0/1333920365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0094a0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.325+0000 7f4f31d97700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 0x7f4f2c1e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f30d95700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c500 tx=0x7f4f2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28005d70 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f280070d0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28007460 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c06f870 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.326+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c06fe30 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.327+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c1e96c0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.327+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4f280075c0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.328+0000 7f4f21ffb700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 0x7f4f1006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.328+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4f280886d0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.328+0000 7f4f31596700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 0x7f4f1006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.328+0000 7f4f31596700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 0x7f4f1006cd20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0090b0 tx=0x7f4f1c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.332+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4f2c1e96c0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.546+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f2c1e96c0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.546+0000 7f4f21ffb700 1 -- 172.21.15.154:0/1333920365 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4f2c1e96c0 con 0x7f4f2c0c5d60 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 msgr2=0x7f4f1006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 0x7f4f1006cd20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0090b0 tx=0x7f4f1c008030 comp rx=0 tx=0).stop 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 msgr2=0x7f4f2c1e99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c500 tx=0x7f4f2800c8c0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 shutdown_connections 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f2c14fbd0 0x7f4f2c1e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f2c0702f0 0x7f4f2c144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4f1006a860 0x7f4f1006cd20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 --2- 172.21.15.154:0/1333920365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f2c0c5d60 0x7f4f2c1e99a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 >> 172.21.15.154:0/1333920365 conn(0x7f4f2c0a9bf0 msgr2=0x7f4f2c14ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 shutdown_connections 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.548+0000 7f4f337fa700 1 -- 172.21.15.154:0/1333920365 wait complete. 2024-03-14T09:02:32.549 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:02:32.560 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T09:02:32.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:32.560 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:32.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:02:32.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:32.605 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T09:02:32.622 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:32.622 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:32.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:02:32.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:02:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.841+0000 7f01141d2700 1 Processor -- start 2024-03-14T09:02:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.841+0000 7f01141d2700 1 -- start start 2024-03-14T09:02:32.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f01141d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f010c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f01141d2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f010c0c1780 0x7f010c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f01141d2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010c13ade0 con 0x7f010c0c3250 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f01141d2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c06fb30 con 0x7f010c0c1780 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f01141d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c06ec30 con 0x7f010c0a8540 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f011176d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f010c0c3250 0x7f010c154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58850/0 (socket says 172.21.15.154:58850) 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f011176d700 1 -- 172.21.15.154:0/4137035760 learned_addr learned my addr 172.21.15.154:0/4137035760 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f011276f700 1 --2- 172.21.15.154:0/4137035760 >> v2:172.21.15.191:3300/0 conn(0x7f010c0c1780 0x7f010c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0111f6e700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f010c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130660442 0 0) 0x7f0108001110 con 0x7f010c0c3250 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f80035b0 con 0x7f010c0c3250 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550547839 0 0) 0x7f00f80035b0 con 0x7f010c0c3250 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 msgr2=0x7f010c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f010c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 >> v2:172.21.15.191:3300/0 conn(0x7f010c0c1780 msgr2=0x7f010c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 --2- 172.21.15.154:0/4137035760 >> v2:172.21.15.191:3300/0 conn(0x7f010c0c1780 0x7f010c0c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c06ef10 con 0x7f010c0c3250 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0108001800 con 0x7f010c0c3250 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 >> v1:172.21.15.154:6789/0 conn(0x7f010c0c3250 legacy=0x7f010c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f00f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f00f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c06fb30 con 0x7f00f8003a00 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c06ec30 con 0x7f00f8004850 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f800ba90 con 0x7f010c0a8540 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.842+0000 7f0111f6e700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f00f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f011176d700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f011276f700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f00f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 -- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 msgr2=0x7f00f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c0a8540 0x7f00f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 -- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 msgr2=0x7f00f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00f8004850 0x7f00f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 -- 172.21.15.154:0/4137035760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80035b0 con 0x7f00f8003a00 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0111f6e700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f00f8003e20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0100009460 tx=0x7f00f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b9b7ab7ab9ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e6c0 con 0x7f00f8003a00 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000e820 con 0x7f00f8003a00 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.843+0000 7f0110f6c700 1 -- 172.21.15.154:0/4137035760 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000eb90 con 0x7f00f8003a00 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 -- 172.21.15.154:0/4137035760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 msgr2=0x7f00f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 --2- 172.21.15.154:0/4137035760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f00f8003e20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0100009460 tx=0x7f00f8005880 comp rx=0 tx=0).stop 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f011276f700 1 -- 172.21.15.154:0/4137035760 reap_dead start 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 -- 172.21.15.154:0/4137035760 shutdown_connections 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 -- 172.21.15.154:0/4137035760 >> 172.21.15.154:0/4137035760 conn(0x7f010c0c3880 msgr2=0x7f010c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 -- 172.21.15.154:0/4137035760 shutdown_connections 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.844+0000 7f01141d2700 1 -- 172.21.15.154:0/4137035760 wait complete. 2024-03-14T09:02:32.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 Processor -- start 2024-03-14T09:02:32.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 -- start start 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c134350 0x7f010c136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f010c133120 0x7f010c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c150ff0 con 0x7f00f8003a00 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c14e240 con 0x7f010c133120 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.845+0000 7f01141d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c137400 con 0x7f010c134350 2024-03-14T09:02:32.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44682/0 (socket says 172.21.15.154:44682) 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 -- 172.21.15.154:0/215517582 learned_addr learned my addr 172.21.15.154:0/215517582 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f011176d700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c134350 0x7f010c136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f011276f700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f010c133120 0x7f010c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 -- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c134350 msgr2=0x7f010c136620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c134350 0x7f010c136620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 -- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f010c133120 msgr2=0x7f010c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f010c133120 0x7f010c14d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 -- 172.21.15.154:0/215517582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c14e490 con 0x7f00f8003a00 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f0111f6e700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f010001c040 tx=0x7f01000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e6c0 con 0x7f00f8003a00 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01000038c0 con 0x7f00f8003a00 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c132640 con 0x7f00f8003a00 2024-03-14T09:02:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100003c70 con 0x7f00f8003a00 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.846+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c132c00 con 0x7f00f8003a00 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.847+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c132d90 con 0x7f00f8003a00 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.848+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0100020040 con 0x7f00f8003a00 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.848+0000 7f00fe7fc700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 0x7f00f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.848+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f010008b6f0 con 0x7f00f8003a00 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.849+0000 7f011176d700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 0x7f00f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.849+0000 7f011176d700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 0x7f00f006ccf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f010c154ad0 tx=0x7f0108007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:32.853+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0100056960 con 0x7f00f8003a00 2024-03-14T09:02:33.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.094+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f010c0697d0 con 0x7f00f8003a00 2024-03-14T09:02:33.094 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.095+0000 7f00fe7fc700 1 -- 172.21.15.154:0/215517582 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+149 (secure 0 0 0) 0x7f0100059f80 con 0x7f00f8003a00 2024-03-14T09:02:33.094 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:02:33.095 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 msgr2=0x7f00f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 0x7f00f006ccf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f010c154ad0 tx=0x7f0108007b00 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 msgr2=0x7f010c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f010001c040 tx=0x7f01000047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 shutdown_connections 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f010c133120 0x7f010c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00f006a830 0x7f00f006ccf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00f8003a00 0x7f010c150ab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 --2- 172.21.15.154:0/215517582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f010c134350 0x7f010c136620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 >> 172.21.15.154:0/215517582 conn(0x7f010c0c3880 msgr2=0x7f010c139980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 shutdown_connections 2024-03-14T09:02:33.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:33.097+0000 7f01141d2700 1 -- 172.21.15.154:0/215517582 wait complete. 2024-03-14T09:02:33.108 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:02:33.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:02:33.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:02:33.108 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-03-14T09:02:33.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:33.109 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:02:33.109 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:02:33.109 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.neWYTGXbY1 2024-03-14T09:02:33.109 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:02:33.125 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.4jHBSfQMVk 2024-03-14T09:02:33.125 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /tmp/tmp.neWYTGXbY1 /tmp/tmp.4jHBSfQMVk 2024-03-14T09:02:33.192 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.4jHBSfQMVk 2024-03-14T09:02:33.320 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.4jHBSfQMVk is 66B 2024-03-14T09:02:33.368 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.4jHBSfQMVk 2024-03-14T09:02:33.385 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg== 2024-03-14T09:02:33.385 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:33.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.385 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:02:33.442 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:02:33.443 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42306sec preferred_lft 42306sec 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:02:33.444 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.445 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:02:33.445 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:02:33.445 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:02:33.445 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:02:33.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.445 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:02:33.508 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:02:33.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.510 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:02:33.568 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:02:33.568 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:02:33.568 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:02:33.568 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:02:33.568 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:02:33.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:02:33.569 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:02:33.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.676 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:02:33.692 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:02:33.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:02:33.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:33.760 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:33.816 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:02:33.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:02:33.817 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:02:33.873 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:33.873 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/tmp/tmp.neWYTGXbY1 of=/dev/stdout 2024-03-14T09:02:33.930 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-03-14T09:02:33.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:34.017 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:34.017 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:34.017 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:02:34.038 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:34.038 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:34.038 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:34.039 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:02:34.060 INFO:teuthology.orchestra.run.smithi154.stderr:mount error 2 = No such file or directory 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:02:34.061 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,key=testuser,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:02:34.062 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-14T09:02:34.063 INFO:tasks.cephfs.kernel_mount:mount command failed 2024-03-14T09:02:34.063 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:02:34.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:02:34.102 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:02:34.144 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:02:34.144 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:34.144 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:34.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:02:34.162 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:34.162 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:34.204 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:34.204 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:34.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:34.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.385+0000 7f03fffc5700 1 Processor -- start 2024-03-14T09:02:34.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.385+0000 7f03fffc5700 1 -- start start 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fffc5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03f8160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fffc5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80ac8e0 con 0x7f03f814eaf0 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fffc5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f806de30 con 0x7f03f8155df0 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fffc5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f806b310 con 0x7f03f815e280 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fdd61700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f03f814eaf0 0x7f03f806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58860/0 (socket says 172.21.15.154:58860) 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fdd61700 1 -- 172.21.15.154:0/1353020699 learned_addr learned my addr 172.21.15.154:0/1353020699 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fe562700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03f8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446959096 0 0) 0x7f03f80ac8e0 con 0x7f03f814eaf0 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.386+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e40035b0 con 0x7f03f814eaf0 2024-03-14T09:02:34.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3334357530 0 0) 0x7f03e40035b0 con 0x7f03f814eaf0 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 msgr2=0x7f03f8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03f8160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 >> v1:172.21.15.191:6789/0 conn(0x7f03f8155df0 legacy=0x7f03f8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f8156960 con 0x7f03f814eaf0 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f03e8001670 con 0x7f03f814eaf0 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 >> v1:172.21.15.154:6789/0 conn(0x7f03f814eaf0 legacy=0x7f03f806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03e4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03e4004d90 0x7f03e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f806b310 con 0x7f03e4003f70 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e400bbd0 con 0x7f03e4004d90 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e400bd90 con 0x7f03f815e280 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fdd61700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03e4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fe562700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03e4004d90 0x7f03e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 -- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 msgr2=0x7f03e4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f815e280 0x7f03e4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 -- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03e4004d90 msgr2=0x7f03e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03e4004d90 0x7f03e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 -- 172.21.15.154:0/1353020699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40035b0 con 0x7f03e4003f70 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fe562700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03e4004d90 0x7f03e4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fd560700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03e4004850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f40018d0 tx=0x7f03e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311dc832700e6bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400c690 con 0x7f03e4003f70 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f400c7f0 con 0x7f03e4003f70 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.387+0000 7f03fcd5f700 1 -- 172.21.15.154:0/1353020699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400cba0 con 0x7f03e4003f70 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 -- 172.21.15.154:0/1353020699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 msgr2=0x7f03e4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 --2- 172.21.15.154:0/1353020699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03e4004850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f40018d0 tx=0x7f03e4005880 comp rx=0 tx=0).stop 2024-03-14T09:02:34.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fe562700 1 -- 172.21.15.154:0/1353020699 reap_dead start 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 -- 172.21.15.154:0/1353020699 shutdown_connections 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 -- 172.21.15.154:0/1353020699 >> 172.21.15.154:0/1353020699 conn(0x7f03f80c3880 msgr2=0x7f03f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 -- 172.21.15.154:0/1353020699 shutdown_connections 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.388+0000 7f03fffc5700 1 -- 172.21.15.154:0/1353020699 wait complete. 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.389+0000 7f03fffc5700 1 Processor -- start 2024-03-14T09:02:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.389+0000 7f03fffc5700 1 -- start start 2024-03-14T09:02:34.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 0x7f03f81edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f814a420 0x7f03f814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f813e450 con 0x7f03e4003f70 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f813de80 con 0x7f03f813d680 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fffc5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f813dbe0 con 0x7f03f814a420 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44694/0 (socket says 172.21.15.154:44694) 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 -- 172.21.15.154:0/2545623774 learned_addr learned my addr 172.21.15.154:0/2545623774 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fe562700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f814a420 0x7f03f814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fd560700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 0x7f03f81edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 -- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f814a420 msgr2=0x7f03f814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f814a420 0x7f03f814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 -- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 msgr2=0x7f03f81edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 0x7f03f81edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 -- 172.21.15.154:0/2545623774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f81ee500 con 0x7f03e4003f70 2024-03-14T09:02:34.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fd560700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 0x7f03f81edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.390+0000 7f03fdd61700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007290 tx=0x7f03e80075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800f6c0 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e800f860 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f81ee7a0 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800fc10 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f81524c0 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.391+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f81526e0 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.392+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f03e800fd70 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.392+0000 7f03ee7fc700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 0x7f03dc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.392+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f03e8088ff0 con 0x7f03e4003f70 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.393+0000 7f03fd560700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 0x7f03dc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.393+0000 7f03fd560700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 0x7f03dc06ccf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.397+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f03e80540b0 con 0x7f03e4003f70 2024-03-14T09:02:34.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.592+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03f81ee350 con 0x7f03e4003f70 2024-03-14T09:02:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.593+0000 7f03ee7fc700 1 -- 172.21.15.154:0/2545623774 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7f03e80576d0 con 0x7f03e4003f70 2024-03-14T09:02:34.593 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:34.594 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 msgr2=0x7f03dc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 0x7f03dc06ccf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4005040 comp rx=0 tx=0).stop 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 msgr2=0x7f03f813e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007290 tx=0x7f03e80075a0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 shutdown_connections 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03f814a420 0x7f03f814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f03dc06a830 0x7f03dc06ccf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03e4003f70 0x7f03f813e8e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 --2- 172.21.15.154:0/2545623774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03f813d680 0x7f03f81edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 >> 172.21.15.154:0/2545623774 conn(0x7f03f80c3880 msgr2=0x7f03f8154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 shutdown_connections 2024-03-14T09:02:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.596+0000 7f03fffc5700 1 -- 172.21.15.154:0/2545623774 wait complete. 2024-03-14T09:02:34.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:01.699649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:02.688126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:02:19.001797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:02:19.712147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:34.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:34.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.733+0000 7ff8cb6ca700 1 Processor -- start 2024-03-14T09:02:34.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.733+0000 7ff8cb6ca700 1 -- start start 2024-03-14T09:02:34.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8cb6ca700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 0x7ff8c4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8cb6ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 0x7ff8c40c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8cb6ca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c4070e10 con 0x7ff8c4072290 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8cb6ca700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c40c1bb0 con 0x7ff8c414c5c0 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8cb6ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c40c6170 con 0x7ff8c4072710 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8c9466700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 0x7ff8c4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8c8c65700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff8c4072290 0x7ff8c40c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:47110/0 (socket says 172.21.15.154:47110) 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8c8c65700 1 -- 172.21.15.154:0/2770880732 learned_addr learned my addr 172.21.15.154:0/2770880732 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.734+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/2770880732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 0x7ff8c40c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9c67700 1 -- 172.21.15.154:0/2770880732 >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 msgr2=0x7ff8c4152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/2770880732 >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 0x7ff8c4152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9c67700 1 -- 172.21.15.154:0/2770880732 >> v1:172.21.15.191:6789/0 conn(0x7ff8c4072290 legacy=0x7ff8c40c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9c67700 1 -- 172.21.15.154:0/2770880732 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c414c9a0 con 0x7ff8c4072710 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9466700 1 --2- 172.21.15.154:0/2770880732 >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 0x7ff8c4152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/2770880732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 0x7ff8c40c26e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009d90 tx=0x7ff8bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ffc57415fcd76d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8bbfff700 1 -- 172.21.15.154:0/2770880732 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00e070 con 0x7ff8c4072710 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8bbfff700 1 -- 172.21.15.154:0/2770880732 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc00bb20 con 0x7ff8c4072710 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.735+0000 7ff8bbfff700 1 -- 172.21.15.154:0/2770880732 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc012430 con 0x7ff8c4072710 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/2770880732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 msgr2=0x7ff8c40c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/2770880732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 0x7ff8c40c26e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009d90 tx=0x7ff8bc00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/2770880732 shutdown_connections 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/2770880732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072710 0x7ff8c40c26e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/2770880732 >> v2:172.21.15.191:3300/0 conn(0x7ff8c414c5c0 0x7ff8c4152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/2770880732 >> 172.21.15.154:0/2770880732 conn(0x7ff8c406a770 msgr2=0x7ff8c406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/2770880732 shutdown_connections 2024-03-14T09:02:34.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.736+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/2770880732 wait complete. 2024-03-14T09:02:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.737+0000 7ff8cb6ca700 1 Processor -- start 2024-03-14T09:02:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.737+0000 7ff8cb6ca700 1 -- start start 2024-03-14T09:02:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072290 0x7ff8c41ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8c4072710 0x7ff8c406df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c414fb80 con 0x7ff8c414c5c0 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c414f2e0 con 0x7ff8c4072710 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8cb6ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c414f5c0 con 0x7ff8c4072290 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44706/0 (socket says 172.21.15.154:44706) 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 -- 172.21.15.154:0/3722099379 learned_addr learned my addr 172.21.15.154:0/3722099379 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c8c65700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8c4072710 0x7ff8c406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9466700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072290 0x7ff8c41ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 -- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072290 msgr2=0x7ff8c41ea0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072290 0x7ff8c41ea0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 -- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8c4072710 msgr2=0x7ff8c406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8c4072710 0x7ff8c406df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.738+0000 7ff8c9c67700 1 -- 172.21.15.154:0/3722099379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40094a0 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8c9c67700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc014040 tx=0x7ff8bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc012430 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc020070 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc009a70 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc003ea0 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.739+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41ed7e0 con 0x7ff8c414c5c0 2024-03-14T09:02:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.740+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c41ed640 con 0x7ff8c414c5c0 2024-03-14T09:02:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.740+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff8bc004040 con 0x7ff8c414c5c0 2024-03-14T09:02:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.741+0000 7ff8b9ffb700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 0x7ff8a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.741+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff8bc022070 con 0x7ff8c414c5c0 2024-03-14T09:02:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.741+0000 7ff8c9466700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 0x7ff8a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.741+0000 7ff8c9466700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 0x7ff8a806ccf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8c406e900 tx=0x7ff8b4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:34.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.745+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff8c41ed640 con 0x7ff8c414c5c0 2024-03-14T09:02:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.939+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8c41ed640 con 0x7ff8c414c5c0 2024-03-14T09:02:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.939+0000 7ff8b9ffb700 1 -- 172.21.15.154:0/3722099379 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8c41ed640 con 0x7ff8c414c5c0 2024-03-14T09:02:34.940 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 msgr2=0x7ff8a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 0x7ff8a806ccf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8c406e900 tx=0x7ff8b4009210 comp rx=0 tx=0).stop 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 msgr2=0x7ff8c40abd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc014040 tx=0x7ff8bc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 shutdown_connections 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8a806a830 0x7ff8a806ccf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8c414c5c0 0x7ff8c40abd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8c4072290 0x7ff8c41ea0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 --2- 172.21.15.154:0/3722099379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8c4072710 0x7ff8c406df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 >> 172.21.15.154:0/3722099379 conn(0x7ff8c406a770 msgr2=0x7ff8c40c2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 shutdown_connections 2024-03-14T09:02:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:34.942+0000 7ff8cb6ca700 1 -- 172.21.15.154:0/3722099379 wait complete. 2024-03-14T09:02:34.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:02:35.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.078+0000 7fbf21137700 1 Processor -- start 2024-03-14T09:02:35.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.078+0000 7fbf21137700 1 -- start start 2024-03-14T09:02:35.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf21137700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbf1c14eaf0 0x7fbf1c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf21137700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbf1c156500 0x7fbf1c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf21137700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0ac8e0 con 0x7fbf1c06acb0 2024-03-14T09:02:35.080 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf21137700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c06b310 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf21137700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c06de30 con 0x7fbf1c156500 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbf1c14eaf0 0x7fbf1c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbf1c14eaf0 0x7fbf1c06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44722/0 (socket says 172.21.15.154:44722) 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 -- 172.21.15.154:0/4057797017 learned_addr learned my addr 172.21.15.154:0/4057797017 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1b59e700 1 --2- 172.21.15.154:0/4057797017 >> v2:172.21.15.191:3300/0 conn(0x7fbf1c156500 0x7fbf1c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 -- 172.21.15.154:0/4057797017 >> v2:172.21.15.191:3300/0 conn(0x7fbf1c156500 msgr2=0x7fbf1c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 --2- 172.21.15.154:0/4057797017 >> v2:172.21.15.191:3300/0 conn(0x7fbf1c156500 0x7fbf1c15f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 -- 172.21.15.154:0/4057797017 >> v1:172.21.15.191:6789/0 conn(0x7fbf1c06acb0 legacy=0x7fbf1c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 -- 172.21.15.154:0/4057797017 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c156910 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf1ad9d700 1 --2- 172.21.15.154:0/4057797017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c06a770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0c4920 tx=0x7fbf10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59ae3c4a1605d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf19d9b700 1 -- 172.21.15.154:0/4057797017 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10004230 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf19d9b700 1 -- 172.21.15.154:0/4057797017 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf10004390 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.079+0000 7fbf19d9b700 1 -- 172.21.15.154:0/4057797017 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100036a0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 -- 172.21.15.154:0/4057797017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 msgr2=0x7fbf1c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 --2- 172.21.15.154:0/4057797017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c06a770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0c4920 tx=0x7fbf10009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 -- 172.21.15.154:0/4057797017 shutdown_connections 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 --2- 172.21.15.154:0/4057797017 >> v2:172.21.15.191:3300/0 conn(0x7fbf1c156500 0x7fbf1c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 --2- 172.21.15.154:0/4057797017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c06a770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 -- 172.21.15.154:0/4057797017 >> 172.21.15.154:0/4057797017 conn(0x7fbf1c0c3880 msgr2=0x7fbf1c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 -- 172.21.15.154:0/4057797017 shutdown_connections 2024-03-14T09:02:35.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.080+0000 7fbf21137700 1 -- 172.21.15.154:0/4057797017 wait complete. 2024-03-14T09:02:35.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.081+0000 7fbf21137700 1 Processor -- start 2024-03-14T09:02:35.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 -- start start 2024-03-14T09:02:35.085 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c06acb0 0x7fbf1c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c156500 0x7fbf1c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c152540 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c144b20 con 0x7fbf1c06acb0 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf21137700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1443d0 con 0x7fbf1c156500 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf1a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44730/0 (socket says 172.21.15.154:44730) 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf1a59c700 1 -- 172.21.15.154:0/3027755317 learned_addr learned my addr 172.21.15.154:0/3027755317 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf1b59e700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c156500 0x7fbf1c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.082+0000 7fbf1ad9d700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c06acb0 0x7fbf1c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 -- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c156500 msgr2=0x7fbf1c14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c156500 0x7fbf1c14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 -- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c06acb0 msgr2=0x7fbf1c14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c06acb0 0x7fbf1c14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 -- 172.21.15.154:0/3027755317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009550 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf1a59c700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c4d0 tx=0x7fbf0c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011790 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0118f0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c152260 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011ca0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.083+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c14cf80 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.084+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c1445f0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.084+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbf0c01c460 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.085+0000 7fbf037fe700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 0x7fbf0806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.085+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fbf0c088ce0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.085+0000 7fbf1ad9d700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 0x7fbf0806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.085+0000 7fbf1ad9d700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 0x7fbf0806ccf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009130 tx=0x7fbf10008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.089+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbf0c053e30 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.281+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf1c0697d0 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.281+0000 7fbf037fe700 1 -- 172.21.15.154:0/3027755317 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf0c057450 con 0x7fbf1c14eaf0 2024-03-14T09:02:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 msgr2=0x7fbf0806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 0x7fbf0806ccf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009130 tx=0x7fbf10008030 comp rx=0 tx=0).stop 2024-03-14T09:02:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 msgr2=0x7fbf1c14a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c4d0 tx=0x7fbf0c00c7e0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 shutdown_connections 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbf1c156500 0x7fbf1c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbf1c06acb0 0x7fbf1c14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbf0806a830 0x7fbf0806ccf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 --2- 172.21.15.154:0/3027755317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf1c14eaf0 0x7fbf1c14a8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 >> 172.21.15.154:0/3027755317 conn(0x7fbf1c0c3880 msgr2=0x7fbf1c155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 shutdown_connections 2024-03-14T09:02:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.283+0000 7fbf21137700 1 -- 172.21.15.154:0/3027755317 wait complete. 2024-03-14T09:02:35.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:02:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.421+0000 7f2b4fe59700 1 Processor -- start 2024-03-14T09:02:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.421+0000 7f2b4fe59700 1 -- start start 2024-03-14T09:02:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4fe59700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b480c7f70 0x7f2b480c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4fe59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c4f30 0x7f2b481547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4fe59700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b48157e80 con 0x7f2b480c8350 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4fe59700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b48158ac0 con 0x7f2b480c7f70 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4fe59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48158120 con 0x7f2b480c4f30 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4dbf5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b480c7f70 0x7f2b480c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4dbf5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b480c7f70 0x7f2b480c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44740/0 (socket says 172.21.15.154:44740) 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4e3f6700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2b480c8350 0x7f2b48154d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58868/0 (socket says 172.21.15.154:58868) 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.422+0000 7f2b4dbf5700 1 -- 172.21.15.154:0/490916913 learned_addr learned my addr 172.21.15.154:0/490916913 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4dbf5700 1 -- 172.21.15.154:0/490916913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c4f30 msgr2=0x7f2b481547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4dbf5700 1 --2- 172.21.15.154:0/490916913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c4f30 0x7f2b481547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4dbf5700 1 -- 172.21.15.154:0/490916913 >> v1:172.21.15.154:6789/0 conn(0x7f2b480c8350 legacy=0x7f2b48154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4dbf5700 1 -- 172.21.15.154:0/490916913 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48158400 con 0x7f2b480c7f70 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4dbf5700 1 --2- 172.21.15.154:0/490916913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c7f70 0x7f2b480c49f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2b380098f0 tx=0x7f2b38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1a2e61cf51019d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4cbf3700 1 -- 172.21.15.154:0/490916913 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800b8a0 con 0x7f2b480c7f70 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4cbf3700 1 -- 172.21.15.154:0/490916913 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800ba00 con 0x7f2b480c7f70 2024-03-14T09:02:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4cbf3700 1 -- 172.21.15.154:0/490916913 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800bd90 con 0x7f2b480c7f70 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4fe59700 1 -- 172.21.15.154:0/490916913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c7f70 msgr2=0x7f2b480c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.423+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/490916913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c7f70 0x7f2b480c49f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2b380098f0 tx=0x7f2b38009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 -- 172.21.15.154:0/490916913 shutdown_connections 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/490916913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c7f70 0x7f2b480c49f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/490916913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c4f30 0x7f2b481547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 -- 172.21.15.154:0/490916913 >> 172.21.15.154:0/490916913 conn(0x7f2b48072080 msgr2=0x7f2b48072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 -- 172.21.15.154:0/490916913 shutdown_connections 2024-03-14T09:02:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.424+0000 7f2b4fe59700 1 -- 172.21.15.154:0/490916913 wait complete. 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 Processor -- start 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 -- start start 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b480c4f30 0x7f2b481e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c7f70 0x7f2b481e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481e0020 con 0x7f2b480c8350 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481df410 con 0x7f2b480c4f30 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.425+0000 7f2b4fe59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481df6f0 con 0x7f2b480c7f70 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44748/0 (socket says 172.21.15.154:44748) 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 -- 172.21.15.154:0/849301680 learned_addr learned my addr 172.21.15.154:0/849301680 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 -- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c7f70 msgr2=0x7f2b481e04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4dbf5700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b480c4f30 0x7f2b481e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4d3f4700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c7f70 0x7f2b481e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c7f70 0x7f2b481e04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 -- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b480c4f30 msgr2=0x7f2b481e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b480c4f30 0x7f2b481e9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 -- 172.21.15.154:0/849301680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009550 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4e3f6700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cbd0 tx=0x7f2b4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000abb0 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4000ad10 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b481df9d0 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007270 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.426+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b481485c0 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.427+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b481e01d0 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.428+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2b400073d0 con 0x7f2b480c8350 2024-03-14T09:02:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.428+0000 7f2b3e7fc700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 0x7f2b2c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.428+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2b40088d40 con 0x7f2b480c8350 2024-03-14T09:02:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.428+0000 7f2b4dbf5700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 0x7f2b2c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.429+0000 7f2b4dbf5700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 0x7f2b2c06ccf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009c40 tx=0x7f2b38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.432+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2b481e01d0 con 0x7f2b480c8350 2024-03-14T09:02:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.627+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b480697d0 con 0x7f2b480c8350 2024-03-14T09:02:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.627+0000 7f2b3e7fc700 1 -- 172.21.15.154:0/849301680 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b40017070 con 0x7f2b480c8350 2024-03-14T09:02:35.627 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 msgr2=0x7f2b2c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 0x7f2b2c06ccf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009c40 tx=0x7f2b38008030 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 msgr2=0x7f2b48148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cbd0 tx=0x7f2b4000cf90 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 shutdown_connections 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2b2c06a830 0x7f2b2c06ccf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b480c8350 0x7f2b48148080 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b480c4f30 0x7f2b481e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 --2- 172.21.15.154:0/849301680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2b480c7f70 0x7f2b481e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 >> 172.21.15.154:0/849301680 conn(0x7f2b48072080 msgr2=0x7f2b481543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 shutdown_connections 2024-03-14T09:02:35.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.630+0000 7f2b4fe59700 1 -- 172.21.15.154:0/849301680 wait complete. 2024-03-14T09:02:35.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:02:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.768+0000 7f1ac6f1d700 1 Processor -- start 2024-03-14T09:02:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.768+0000 7f1ac6f1d700 1 -- start start 2024-03-14T09:02:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac6f1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 0x7f1ac006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac6f1d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 0x7f1ac015f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac6f1d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac00ac8e0 con 0x7f1ac006acb0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac6f1d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006b310 con 0x7f1ac0156500 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac6f1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac006de30 con 0x7f1ac014eaf0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac54ba700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 0x7f1ac015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1ac4cb9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 0x7f1ac006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1abffff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1ac006acb0 0x7f1ac0155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:47126/0 (socket says 172.21.15.154:47126) 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.769+0000 7f1abffff700 1 -- 172.21.15.154:0/3922829537 learned_addr learned my addr 172.21.15.154:0/3922829537 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/3922829537 >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 msgr2=0x7f1ac015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac4cb9700 1 --2- 172.21.15.154:0/3922829537 >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 0x7f1ac015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/3922829537 >> v1:172.21.15.191:6789/0 conn(0x7f1ac006acb0 legacy=0x7f1ac0155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/3922829537 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0156950 con 0x7f1ac014eaf0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac54ba700 1 --2- 172.21.15.154:0/3922829537 >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 0x7f1ac015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac4cb9700 1 --2- 172.21.15.154:0/3922829537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 0x7f1ac006a770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009840 tx=0x7f1ab4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3536b6ae6184799 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1abf7fe700 1 -- 172.21.15.154:0/3922829537 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4011830 con 0x7f1ac014eaf0 2024-03-14T09:02:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1abf7fe700 1 -- 172.21.15.154:0/3922829537 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4011990 con 0x7f1ac014eaf0 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1abf7fe700 1 -- 172.21.15.154:0/3922829537 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4011d20 con 0x7f1ac014eaf0 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/3922829537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 msgr2=0x7f1ac006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.770+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/3922829537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 0x7f1ac006a770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009840 tx=0x7f1ab4009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/3922829537 shutdown_connections 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/3922829537 >> v2:172.21.15.191:3300/0 conn(0x7f1ac0156500 0x7f1ac015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/3922829537 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac014eaf0 0x7f1ac006a770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/3922829537 >> 172.21.15.154:0/3922829537 conn(0x7f1ac00c3880 msgr2=0x7f1ac00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/3922829537 shutdown_connections 2024-03-14T09:02:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.771+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/3922829537 wait complete. 2024-03-14T09:02:35.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.772+0000 7f1ac6f1d700 1 Processor -- start 2024-03-14T09:02:35.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.772+0000 7f1ac6f1d700 1 -- start start 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac014eaf0 0x7f1ac00c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 0x7f1ac01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac0151d20 con 0x7f1ac006acb0 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00c28e0 con 0x7f1ac014eaf0 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac6f1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01eae30 con 0x7f1ac0156500 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac4cb9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac4cb9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44760/0 (socket says 172.21.15.154:44760) 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/1952963456 learned_addr learned my addr 172.21.15.154:0/1952963456 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1abffff700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac014eaf0 0x7f1ac00c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.773+0000 7f1ac54ba700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 0x7f1ac01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 msgr2=0x7f1ac01507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 0x7f1ac01507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac014eaf0 msgr2=0x7f1ac00c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac014eaf0 0x7f1ac00c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 -- 172.21.15.154:0/1952963456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8009a70 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac54ba700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 0x7f1ac01507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac4cb9700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40090b0 tx=0x7f1ab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40036a0 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab401f070 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab40094a0 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4003a60 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.774+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01eb0a0 con 0x7f1ac006acb0 2024-03-14T09:02:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.775+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac00697d0 con 0x7f1ac006acb0 2024-03-14T09:02:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.776+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1ab4003bc0 con 0x7f1ac006acb0 2024-03-14T09:02:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.776+0000 7f1abd7fa700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 0x7f1aa406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.776+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1ab4021070 con 0x7f1ac006acb0 2024-03-14T09:02:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.776+0000 7f1abffff700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 0x7f1aa406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.777+0000 7f1abffff700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 0x7f1aa406ccf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ac014c7c0 tx=0x7f1aac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.780+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1ab4055c40 con 0x7f1ac006acb0 2024-03-14T09:02:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.974+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ac006e050 con 0x7f1ac006acb0 2024-03-14T09:02:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.974+0000 7f1abd7fa700 1 -- 172.21.15.154:0/1952963456 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v36) v1 ==== 105+0+11 (secure 0 0 0) 0x7f1ab40150b0 con 0x7f1ac006acb0 2024-03-14T09:02:35.974 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.976+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 msgr2=0x7f1aa406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.976+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 0x7f1aa406ccf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ac014c7c0 tx=0x7f1aac009040 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.976+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 msgr2=0x7f1ac01539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.976+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40090b0 tx=0x7f1ab4004750 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 shutdown_connections 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ac0156500 0x7f1ac01507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1aa406a830 0x7f1aa406ccf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ac006acb0 0x7f1ac01539a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 --2- 172.21.15.154:0/1952963456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ac014eaf0 0x7f1ac00c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 >> 172.21.15.154:0/1952963456 conn(0x7f1ac00c3880 msgr2=0x7f1ac0155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 shutdown_connections 2024-03-14T09:02:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:35.977+0000 7f1ac6f1d700 1 -- 172.21.15.154:0/1952963456 wait complete. 2024-03-14T09:02:35.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:02:36.112 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 Processor -- start 2024-03-14T09:02:36.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 -- start start 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f01780c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f017814fbd0 0x7f01780c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01780acd10 con 0x7f01780c78c0 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01780ac7b0 con 0x7f017814fbd0 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.113+0000 7f017e3a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0178150050 con 0x7f01780702f0 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f01777fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f017814fbd0 0x7f01780c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f0177fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f01780c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f0177fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f01780c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:43644/0 (socket says 172.21.15.154:43644) 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f0177fff700 1 -- 172.21.15.154:0/3277687565 learned_addr learned my addr 172.21.15.154:0/3277687565 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f01777fe700 1 -- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 msgr2=0x7f01780c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f01777fe700 1 --2- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f01780c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f01777fe700 1 -- 172.21.15.154:0/3277687565 >> v1:172.21.15.191:6789/0 conn(0x7f01780c78c0 legacy=0x7f01780c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.114+0000 7f01777fe700 1 -- 172.21.15.154:0/3277687565 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780ac9c0 con 0x7f017814fbd0 2024-03-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f01777fe700 1 --2- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f017814fbd0 0x7f01780c7380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f016c009840 tx=0x7f016c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de56dfb8b9a1f2a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f0176ffd700 1 -- 172.21.15.154:0/3277687565 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c011830 con 0x7f017814fbd0 2024-03-14T09:02:36.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f0176ffd700 1 -- 172.21.15.154:0/3277687565 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c011990 con 0x7f017814fbd0 2024-03-14T09:02:36.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f0176ffd700 1 -- 172.21.15.154:0/3277687565 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c011d20 con 0x7f017814fbd0 2024-03-14T09:02:36.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 -- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f017814fbd0 msgr2=0x7f01780c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f017814fbd0 0x7f01780c7380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f016c009840 tx=0x7f016c009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 -- 172.21.15.154:0/3277687565 shutdown_connections 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f01780c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3277687565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f017814fbd0 0x7f01780c7380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 -- 172.21.15.154:0/3277687565 >> 172.21.15.154:0/3277687565 conn(0x7f01780a9bf0 msgr2=0x7f01780a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 -- 172.21.15.154:0/3277687565 shutdown_connections 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.115+0000 7f017e3a8700 1 -- 172.21.15.154:0/3277687565 wait complete. 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.116+0000 7f017e3a8700 1 Processor -- start 2024-03-14T09:02:36.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 -- start start 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f017814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780c78c0 0x7f017814c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017814c880 con 0x7f017814fbd0 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781e8880 con 0x7f01780c78c0 2024-03-14T09:02:36.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.117+0000 7f017e3a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0178149f10 con 0x7f01780702f0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44770/0 (socket says 172.21.15.154:44770) 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 -- 172.21.15.154:0/3236189409 learned_addr learned my addr 172.21.15.154:0/3236189409 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f01777fe700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780c78c0 0x7f017814c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f0177fff700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f017814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 -- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 msgr2=0x7f017814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f017814be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 -- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780c78c0 msgr2=0x7f017814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780c78c0 0x7f017814c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 -- 172.21.15.154:0/3236189409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0094a0 con 0x7f017814fbd0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017c945700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f017000cb50 tx=0x7f017000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170003c00 con 0x7f017814fbd0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0170003d60 con 0x7f017814fbd0 2024-03-14T09:02:36.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01781e8b20 con 0x7f017814fbd0 2024-03-14T09:02:36.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000aa20 con 0x7f017814fbd0 2024-03-14T09:02:36.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.118+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017814a490 con 0x7f017814fbd0 2024-03-14T09:02:36.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.119+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01780697d0 con 0x7f017814fbd0 2024-03-14T09:02:36.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.120+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f017000ab80 con 0x7f017814fbd0 2024-03-14T09:02:36.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.120+0000 7f0174ff9700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 0x7f016406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.121+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0170089b60 con 0x7f017814fbd0 2024-03-14T09:02:36.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.121+0000 7f0177fff700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 0x7f016406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.121+0000 7f0177fff700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 0x7f016406ccf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0168009b30 tx=0x7f01680092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.124+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0170058720 con 0x7f017814fbd0 2024-03-14T09:02:36.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.320+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f01780c7160 con 0x7f017814fbd0 2024-03-14T09:02:36.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.321+0000 7f0174ff9700 1 -- 172.21.15.154:0/3236189409 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v36) v1 ==== 101+0+11 (secure 0 0 0) 0x7f01700582b0 con 0x7f017814fbd0 2024-03-14T09:02:36.321 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:36.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 msgr2=0x7f016406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 0x7f016406ccf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0168009b30 tx=0x7f01680092c0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 msgr2=0x7f01780a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f017000cb50 tx=0x7f017000ce60 comp rx=0 tx=0).stop 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 shutdown_connections 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f016406a830 0x7f016406ccf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f017814fbd0 0x7f01780a8d90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01780702f0 0x7f017814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 --2- 172.21.15.154:0/3236189409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780c78c0 0x7f017814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 >> 172.21.15.154:0/3236189409 conn(0x7f01780a9bf0 msgr2=0x7f017814ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 shutdown_connections 2024-03-14T09:02:36.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.323+0000 7f017e3a8700 1 -- 172.21.15.154:0/3236189409 wait complete. 2024-03-14T09:02:36.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:02:36.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.461+0000 7fcd2154f700 1 Processor -- start 2024-03-14T09:02:36.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.461+0000 7fcd2154f700 1 -- start start 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd2154f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd1c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd2154f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c13ade0 con 0x7fcd1c0c1780 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd2154f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c06fb30 con 0x7fcd1c0a8540 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd2154f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06ec30 con 0x7fcd1c0c3250 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd1b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcd1c0c1780 0x7fcd1c0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58884/0 (socket says 172.21.15.154:58884) 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd1b7fe700 1 -- 172.21.15.154:0/1810330467 learned_addr learned my addr 172.21.15.154:0/1810330467 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:36.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897771190 0 0) 0x7fcd14001490 con 0x7fcd1c0c1780 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd040035b0 con 0x7fcd1c0c1780 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.462+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd1c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2720547328 0 0) 0x7fcd040035b0 con 0x7fcd1c0c1780 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 msgr2=0x7fcd1c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd1c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 >> v1:172.21.15.191:6789/0 conn(0x7fcd1c0a8540 legacy=0x7fcd1c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c06ef10 con 0x7fcd1c0c1780 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fcd14001b80 con 0x7fcd1c0c1780 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 >> v1:172.21.15.154:6789/0 conn(0x7fcd1c0c1780 legacy=0x7fcd1c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd04003a30 0x7fcd04003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04004850 0x7fcd04004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd040052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06ec30 con 0x7fcd04003a30 2024-03-14T09:02:36.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0400bbd0 con 0x7fcd04004850 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0400bd90 con 0x7fcd1c0c3250 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd04003a30 0x7fcd04003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1b7fe700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd040052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04004850 0x7fcd04004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 -- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 msgr2=0x7fcd040052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c0c3250 0x7fcd040052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 -- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04004850 msgr2=0x7fcd04004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04004850 0x7fcd04004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 -- 172.21.15.154:0/1810330467 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040035b0 con 0x7fcd04003a30 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04004850 0x7fcd04004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd1affd700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd04003a30 0x7fcd04003e50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcd10001800 tx=0x7fcd04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb56ca6e90afdccb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100012f0 con 0x7fcd04003a30 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.463+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd100052f0 con 0x7fcd04003a30 2024-03-14T09:02:36.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd19ffb700 1 -- 172.21.15.154:0/1810330467 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10006860 con 0x7fcd04003a30 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 -- 172.21.15.154:0/1810330467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd04003a30 msgr2=0x7fcd04003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 --2- 172.21.15.154:0/1810330467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd04003a30 0x7fcd04003e50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcd10001800 tx=0x7fcd04005880 comp rx=0 tx=0).stop 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd1b7fe700 1 -- 172.21.15.154:0/1810330467 reap_dead start 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 -- 172.21.15.154:0/1810330467 shutdown_connections 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 -- 172.21.15.154:0/1810330467 >> 172.21.15.154:0/1810330467 conn(0x7fcd1c0c3880 msgr2=0x7fcd1c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 -- 172.21.15.154:0/1810330467 shutdown_connections 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.464+0000 7fcd2154f700 1 -- 172.21.15.154:0/1810330467 wait complete. 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.465+0000 7fcd2154f700 1 Processor -- start 2024-03-14T09:02:36.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.465+0000 7fcd2154f700 1 -- start start 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 0x7fcd1c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 0x7fcd1c0c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c153c50 con 0x7fcd1c136e70 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0c5d00 con 0x7fcd04003a30 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd2154f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1343b0 con 0x7fcd1c136070 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44794/0 (socket says 172.21.15.154:44794) 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 -- 172.21.15.154:0/2713105149 learned_addr learned my addr 172.21.15.154:0/2713105149 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1affd700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 0x7fcd1c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1b7fe700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 0x7fcd1c0c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 -- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 msgr2=0x7fcd1c0c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 0x7fcd1c0c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 -- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 msgr2=0x7fcd1c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 0x7fcd1c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1a7fc700 1 -- 172.21.15.154:0/2713105149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c134690 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1affd700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 0x7fcd1c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.466+0000 7fcd1b7fe700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 0x7fcd1c0c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd1a7fc700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c010880 tx=0x7fcd0c010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c010e80 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c004020 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c134970 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0043b0 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.467+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c134060 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.468+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c0697d0 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.468+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd0c00e8d0 con 0x7fcd1c136e70 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.468+0000 7fcd037fe700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 0x7fcd0806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.469+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fcd0c0892a0 con 0x7fcd1c136e70 2024-03-14T09:02:36.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.469+0000 7fcd1affd700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 0x7fcd0806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.469+0000 7fcd1affd700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 0x7fcd0806cd20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd10014040 tx=0x7fcd10009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.473+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcd0c054360 con 0x7fcd1c136e70 2024-03-14T09:02:36.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.682+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd1c0c5e40 con 0x7fcd1c136e70 2024-03-14T09:02:36.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.683+0000 7fcd037fe700 1 -- 172.21.15.154:0/2713105149 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd0c057980 con 0x7fcd1c136e70 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 msgr2=0x7fcd0806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 0x7fcd0806cd20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcd10014040 tx=0x7fcd10009030 comp rx=0 tx=0).stop 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 msgr2=0x7fcd1c0c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c010880 tx=0x7fcd0c010c40 comp rx=0 tx=0).stop 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 shutdown_connections 2024-03-14T09:02:36.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd1c136070 0x7fcd1c0c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04003a30 0x7fcd1c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd0806a860 0x7fcd0806cd20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 --2- 172.21.15.154:0/2713105149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c136e70 0x7fcd1c0c6190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 >> 172.21.15.154:0/2713105149 conn(0x7fcd1c0c3880 msgr2=0x7fcd1c1380c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 shutdown_connections 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.685+0000 7fcd2154f700 1 -- 172.21.15.154:0/2713105149 wait complete. 2024-03-14T09:02:36.686 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:02:36.697 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:02:36.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:36.697 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:02:36.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:02:36.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:02:36.743 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:02:36.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.922+0000 7f043e01e700 1 Processor -- start 2024-03-14T09:02:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 -- start start 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 0x7f043806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0438070a10 0x7f04380c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380c5f10 con 0x7f043806d600 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380c7d50 con 0x7f0438070a10 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f043e01e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380c8030 con 0x7f043806c6e0 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f0436ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0438070a10 0x7f04380c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f0437fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f043806d600 0x7f04380c68e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58888/0 (socket says 172.21.15.154:58888) 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.923+0000 7f0437fff700 1 -- 172.21.15.154:0/2961777438 learned_addr learned my addr 172.21.15.154:0/2961777438 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04377fe700 1 --2- 172.21.15.154:0/2961777438 >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 0x7f043806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f0436ffd700 1 -- 172.21.15.154:0/2961777438 >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 msgr2=0x7f043806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f0436ffd700 1 --2- 172.21.15.154:0/2961777438 >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 0x7f043806d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f0436ffd700 1 -- 172.21.15.154:0/2961777438 >> v1:172.21.15.154:6789/0 conn(0x7f043806d600 legacy=0x7f04380c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f0436ffd700 1 -- 172.21.15.154:0/2961777438 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380c8310 con 0x7f0438070a10 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04367fc700 1 -- 172.21.15.154:0/2961777438 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267915182 0 0) 0x7f04380c5f10 con 0x7f043806d600 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04377fe700 1 --2- 172.21.15.154:0/2961777438 >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 0x7f043806d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f0436ffd700 1 --2- 172.21.15.154:0/2961777438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438070a10 0x7f04380c63a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0438073f80 tx=0x7f0428009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c565665bf5232c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04367fc700 1 -- 172.21.15.154:0/2961777438 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428011830 con 0x7f0438070a10 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04367fc700 1 -- 172.21.15.154:0/2961777438 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428011990 con 0x7f0438070a10 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.924+0000 7f04367fc700 1 -- 172.21.15.154:0/2961777438 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428011d20 con 0x7f0438070a10 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 -- 172.21.15.154:0/2961777438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438070a10 msgr2=0x7f04380c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 --2- 172.21.15.154:0/2961777438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438070a10 0x7f04380c63a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0438073f80 tx=0x7f0428009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 -- 172.21.15.154:0/2961777438 shutdown_connections 2024-03-14T09:02:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 --2- 172.21.15.154:0/2961777438 >> v2:172.21.15.191:3300/0 conn(0x7f043806c6e0 0x7f043806d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 --2- 172.21.15.154:0/2961777438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0438070a10 0x7f04380c63a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 -- 172.21.15.154:0/2961777438 >> 172.21.15.154:0/2961777438 conn(0x7f04380a9bf0 msgr2=0x7f04380a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 -- 172.21.15.154:0/2961777438 shutdown_connections 2024-03-14T09:02:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.925+0000 7f043e01e700 1 -- 172.21.15.154:0/2961777438 wait complete. 2024-03-14T09:02:36.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.926+0000 7f043e01e700 1 Processor -- start 2024-03-14T09:02:36.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.926+0000 7f043e01e700 1 -- start start 2024-03-14T09:02:36.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f043806d600 0x7f0438099e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0438070a10 0x7f043809a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04381de0c0 con 0x7f043806c6e0 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04381dd4b0 con 0x7f043806d600 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04381dd790 con 0x7f0438070a10 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44802/0 (socket says 172.21.15.154:44802) 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 -- 172.21.15.154:0/3185749562 learned_addr learned my addr 172.21.15.154:0/3185749562 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f0436ffd700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f043806d600 0x7f0438099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 -- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0438070a10 msgr2=0x7f043809a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0438070a10 0x7f043809a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 -- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f043806d600 msgr2=0x7f0438099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f043806d600 0x7f0438099e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:36.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 -- 172.21.15.154:0/3185749562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280094a0 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f04377fe700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f042000e810 tx=0x7f042000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420013610 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420013770 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04381dda70 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.927+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420013b20 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.928+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381dcc00 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.928+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04380697d0 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.929+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0420013c80 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.930+0000 7f042ffff700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 0x7f041806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.930+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0420089230 con 0x7f043806c6e0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.930+0000 7f0436ffd700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 0x7f041806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.930+0000 7f0436ffd700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 0x7f041806cd20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0438070ea0 tx=0x7f0428008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:36.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:36.934+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f04200542d0 con 0x7f043806c6e0 2024-03-14T09:02:37.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.171+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04380c5f80 con 0x7f043806c6e0 2024-03-14T09:02:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.172+0000 7f042ffff700 1 -- 172.21.15.154:0/3185749562 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5951 (secure 0 0 0) 0x7f04200578f0 con 0x7f043806c6e0 2024-03-14T09:02:37.172 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:37.172 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 msgr2=0x7f041806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 0x7f041806cd20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0438070ea0 tx=0x7f0428008030 comp rx=0 tx=0).stop 2024-03-14T09:02:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 msgr2=0x7f04381dc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f042000e810 tx=0x7f042000ebd0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 shutdown_connections 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0438070a10 0x7f043809a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f041806a860 0x7f041806cd20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f043806c6e0 0x7f04381dc6c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 --2- 172.21.15.154:0/3185749562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f043806d600 0x7f0438099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 >> 172.21.15.154:0/3185749562 conn(0x7f04380a9bf0 msgr2=0x7f04380a8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 shutdown_connections 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.174+0000 7f043e01e700 1 -- 172.21.15.154:0/3185749562 wait complete. 2024-03-14T09:02:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 20 2024-03-14T09:02:37.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:37.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.308+0000 7ff183d95700 1 Processor -- start 2024-03-14T09:02:37.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 -- start start 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 0x7ff17c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0c5f10 con 0x7ff17c06d600 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0c7d50 con 0x7ff17c06c6e0 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.309+0000 7ff183d95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0c8030 con 0x7ff17c070a10 2024-03-14T09:02:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181b31700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 0x7ff17c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181b31700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 0x7ff17c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47382/0 (socket says 172.21.15.154:47382) 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:43704/0 (socket says 172.21.15.154:43704) 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181b31700 1 -- 172.21.15.154:0/3669693121 learned_addr learned my addr 172.21.15.154:0/3669693121 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 -- 172.21.15.154:0/3669693121 >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 msgr2=0x7ff17c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 --2- 172.21.15.154:0/3669693121 >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 0x7ff17c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 -- 172.21.15.154:0/3669693121 >> v1:172.21.15.191:6789/0 conn(0x7ff17c06d600 legacy=0x7ff17c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 -- 172.21.15.154:0/3669693121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0c8310 con 0x7ff17c070a10 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.310+0000 7ff181330700 1 --2- 172.21.15.154:0/3669693121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff178009840 tx=0x7ff178009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856756d0f14e5fcc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff180b2f700 1 -- 172.21.15.154:0/3669693121 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178011830 con 0x7ff17c070a10 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff180b2f700 1 -- 172.21.15.154:0/3669693121 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff178011990 con 0x7ff17c070a10 2024-03-14T09:02:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff180b2f700 1 -- 172.21.15.154:0/3669693121 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178011d20 con 0x7ff17c070a10 2024-03-14T09:02:37.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 -- 172.21.15.154:0/3669693121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 msgr2=0x7ff17c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 --2- 172.21.15.154:0/3669693121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff178009840 tx=0x7ff178009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:37.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 -- 172.21.15.154:0/3669693121 shutdown_connections 2024-03-14T09:02:37.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 --2- 172.21.15.154:0/3669693121 >> v2:172.21.15.191:3300/0 conn(0x7ff17c06c6e0 0x7ff17c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 --2- 172.21.15.154:0/3669693121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c070a10 0x7ff17c0c63a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 -- 172.21.15.154:0/3669693121 >> 172.21.15.154:0/3669693121 conn(0x7ff17c0a9bf0 msgr2=0x7ff17c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 -- 172.21.15.154:0/3669693121 shutdown_connections 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.311+0000 7ff183d95700 1 -- 172.21.15.154:0/3669693121 wait complete. 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.312+0000 7ff183d95700 1 Processor -- start 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.312+0000 7ff183d95700 1 -- start start 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff17c06c6e0 0x7ff17c1dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c06d600 0x7ff17c1da180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c09c040 con 0x7ff17c070a10 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1eaa30 con 0x7ff17c06c6e0 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff183d95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1eaf60 con 0x7ff17c06d600 2024-03-14T09:02:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff181b31700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff17c06c6e0 0x7ff17c1dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44804/0 (socket says 172.21.15.154:44804) 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff181330700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c06d600 0x7ff17c1da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 -- 172.21.15.154:0/3733279016 learned_addr learned my addr 172.21.15.154:0/3733279016 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 -- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c06d600 msgr2=0x7ff17c1da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c06d600 0x7ff17c1da180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 -- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff17c06c6e0 msgr2=0x7ff17c1dc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff17c06c6e0 0x7ff17c1dc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.313+0000 7ff182332700 1 -- 172.21.15.154:0/3733279016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780094a0 con 0x7ff17c070a10 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff182332700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff17400cc50 tx=0x7ff17400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174003c00 con 0x7ff17c070a10 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174003d60 con 0x7ff17c070a10 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17400aab0 con 0x7ff17c070a10 2024-03-14T09:02:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c0ad3d0 con 0x7ff17c070a10 2024-03-14T09:02:37.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.314+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0ad060 con 0x7ff17c070a10 2024-03-14T09:02:37.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.315+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c099c80 con 0x7ff17c070a10 2024-03-14T09:02:37.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.315+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff17400ac10 con 0x7ff17c070a10 2024-03-14T09:02:37.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.315+0000 7ff16e7fc700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 0x7ff16006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.316+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff174089060 con 0x7ff17c070a10 2024-03-14T09:02:37.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.316+0000 7ff181b31700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 0x7ff16006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.316+0000 7ff181b31700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 0x7ff16006ccf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff17c1db360 tx=0x7ff1700092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.320+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff17c099c80 con 0x7ff17c070a10 2024-03-14T09:02:37.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.515+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff17c099c80 con 0x7ff17c070a10 2024-03-14T09:02:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.516+0000 7ff16e7fc700 1 -- 172.21.15.154:0/3733279016 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7ff17c099c80 con 0x7ff17c070a10 2024-03-14T09:02:37.516 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:37.516 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 msgr2=0x7ff16006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 0x7ff16006ccf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff17c1db360 tx=0x7ff1700092c0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 msgr2=0x7ff17c1da6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff17400cc50 tx=0x7ff17400cf60 comp rx=0 tx=0).stop 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 shutdown_connections 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff16006a830 0x7ff16006ccf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff17c070a10 0x7ff17c1da6c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff17c06c6e0 0x7ff17c1dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 --2- 172.21.15.154:0/3733279016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff17c06d600 0x7ff17c1da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 >> 172.21.15.154:0/3733279016 conn(0x7ff17c0a9bf0 msgr2=0x7ff17c0c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 shutdown_connections 2024-03-14T09:02:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.518+0000 7ff183d95700 1 -- 172.21.15.154:0/3733279016 wait complete. 2024-03-14T09:02:37.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:01.699649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:02.688126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:02:19.001797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:02:19.712147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:37.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:37.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.656+0000 7f8ec0011700 1 Processor -- start 2024-03-14T09:02:37.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.656+0000 7f8ec0011700 1 -- start start 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.657+0000 7f8ec0011700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.657+0000 7f8ec0011700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb80c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.657+0000 7f8ec0011700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80c7bb0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.657+0000 7f8ec0011700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80c3e00 con 0x7f8eb806c670 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.657+0000 7f8ec0011700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c7e90 con 0x7f8eb806d510 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebddad700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb80c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebddad700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47400/0 (socket says 172.21.15.154:47400) 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebddad700 1 -- 172.21.15.154:0/2088921618 learned_addr learned my addr 172.21.15.154:0/2088921618 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 -- 172.21.15.154:0/2088921618 >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 msgr2=0x7f8eb806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 --2- 172.21.15.154:0/2088921618 >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 -- 172.21.15.154:0/2088921618 >> v1:172.21.15.191:6789/0 conn(0x7f8eb80c31c0 legacy=0x7f8eb80c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 -- 172.21.15.154:0/2088921618 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80c8170 con 0x7f8eb806d510 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebddad700 1 --2- 172.21.15.154:0/2088921618 >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebd5ac700 1 --2- 172.21.15.154:0/2088921618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb80c6140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009840 tx=0x7f8eb4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0975a3fbf0b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebcdab700 1 -- 172.21.15.154:0/2088921618 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4011830 con 0x7f8eb806d510 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebcdab700 1 -- 172.21.15.154:0/2088921618 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4011990 con 0x7f8eb806d510 2024-03-14T09:02:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.658+0000 7f8ebcdab700 1 -- 172.21.15.154:0/2088921618 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4011d20 con 0x7f8eb806d510 2024-03-14T09:02:37.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 -- 172.21.15.154:0/2088921618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 msgr2=0x7f8eb80c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 --2- 172.21.15.154:0/2088921618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb80c6140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009840 tx=0x7f8eb4009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:37.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 -- 172.21.15.154:0/2088921618 shutdown_connections 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 --2- 172.21.15.154:0/2088921618 >> v2:172.21.15.191:3300/0 conn(0x7f8eb806c670 0x7f8eb806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 --2- 172.21.15.154:0/2088921618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb80c6140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 -- 172.21.15.154:0/2088921618 >> 172.21.15.154:0/2088921618 conn(0x7f8eb80a9bf0 msgr2=0x7f8eb80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 -- 172.21.15.154:0/2088921618 shutdown_connections 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.659+0000 7f8ec0011700 1 -- 172.21.15.154:0/2088921618 wait complete. 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.660+0000 7f8ec0011700 1 Processor -- start 2024-03-14T09:02:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.660+0000 7f8ec0011700 1 -- start start 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8eb806c670 0x7f8eb81ee250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb8157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb815d530 con 0x7f8eb80c31c0 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81ed800 con 0x7f8eb806c670 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ec0011700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81eddc0 con 0x7f8eb806d510 2024-03-14T09:02:37.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.661+0000 7f8ebe5ae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44814/0 (socket says 172.21.15.154:44814) 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 -- 172.21.15.154:0/1879005861 learned_addr learned my addr 172.21.15.154:0/1879005861 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebddad700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8eb806c670 0x7f8eb81ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebd5ac700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb8157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 -- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 msgr2=0x7f8eb8157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb8157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 -- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8eb806c670 msgr2=0x7f8eb81ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8eb806c670 0x7f8eb81ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 -- 172.21.15.154:0/1879005861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8009550 con 0x7f8eb80c31c0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ebe5ae700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000cc80 tx=0x7f8eb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0003c00 con 0x7f8eb80c31c0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0003d60 con 0x7f8eb80c31c0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000aaa0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb40094a0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.662+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80a6f30 con 0x7f8eb80c31c0 2024-03-14T09:02:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.663+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb80697d0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.664+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8eb000ac00 con 0x7f8eb80c31c0 2024-03-14T09:02:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.664+0000 7f8eae7fc700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 0x7f8e9c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.664+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8eb0088fd0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.664+0000 7f8ebddad700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 0x7f8e9c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:37.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.665+0000 7f8ebddad700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 0x7f8e9c06ccf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006d30 tx=0x7f8ea80092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:37.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.669+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8eb0054120 con 0x7f8eb80c31c0 2024-03-14T09:02:37.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.862+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8eb80ad3f0 con 0x7f8eb80c31c0 2024-03-14T09:02:37.862 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.862+0000 7f8eae7fc700 1 -- 172.21.15.154:0/1879005861 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8eb0057740 con 0x7f8eb80c31c0 2024-03-14T09:02:37.862 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 msgr2=0x7f8e9c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 0x7f8e9c06ccf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006d30 tx=0x7f8ea80092c0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 msgr2=0x7f8eb815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000cc80 tx=0x7f8eb000cf90 comp rx=0 tx=0).stop 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 shutdown_connections 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e9c06a830 0x7f8e9c06ccf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8eb80c31c0 0x7f8eb815bb90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8eb806c670 0x7f8eb81ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 --2- 172.21.15.154:0/1879005861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb806d510 0x7f8eb8157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 >> 172.21.15.154:0/1879005861 conn(0x7f8eb80a9bf0 msgr2=0x7f8eb806cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 shutdown_connections 2024-03-14T09:02:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.864+0000 7f8ec0011700 1 -- 172.21.15.154:0/1879005861 wait complete. 2024-03-14T09:02:37.876 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:02:37.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:37.999+0000 7efc62bf2700 1 Processor -- start 2024-03-14T09:02:38.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 -- start start 2024-03-14T09:02:38.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc5c14eaf0 0x7efc5c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 0x7efc5c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c0ac8e0 con 0x7efc5c06acb0 2024-03-14T09:02:38.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c06b310 con 0x7efc5c14eaf0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc62bf2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c06de30 con 0x7efc5c156500 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc6098e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc5c14eaf0 0x7efc5c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc6098e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc5c14eaf0 0x7efc5c06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44830/0 (socket says 172.21.15.154:44830) 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.000+0000 7efc6098e700 1 -- 172.21.15.154:0/2368251489 learned_addr learned my addr 172.21.15.154:0/2368251489 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6118f700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 0x7efc5c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6098e700 1 -- 172.21.15.154:0/2368251489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 msgr2=0x7efc5c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6098e700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 0x7efc5c15f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6098e700 1 -- 172.21.15.154:0/2368251489 >> v1:172.21.15.191:6789/0 conn(0x7efc5c06acb0 legacy=0x7efc5c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6098e700 1 -- 172.21.15.154:0/2368251489 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c156930 con 0x7efc5c14eaf0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6118f700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 0x7efc5c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc6098e700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c14eaf0 0x7efc5c06a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc500098f0 tx=0x7efc50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90795237d85a575 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc5b7fe700 1 -- 172.21.15.154:0/2368251489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000b8a0 con 0x7efc5c14eaf0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc5b7fe700 1 -- 172.21.15.154:0/2368251489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5000ba00 con 0x7efc5c14eaf0 2024-03-14T09:02:38.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.001+0000 7efc5b7fe700 1 -- 172.21.15.154:0/2368251489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000bd90 con 0x7efc5c14eaf0 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 -- 172.21.15.154:0/2368251489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c14eaf0 msgr2=0x7efc5c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c14eaf0 0x7efc5c06a770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc500098f0 tx=0x7efc50009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 -- 172.21.15.154:0/2368251489 shutdown_connections 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c156500 0x7efc5c15f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 --2- 172.21.15.154:0/2368251489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c14eaf0 0x7efc5c06a770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 -- 172.21.15.154:0/2368251489 >> 172.21.15.154:0/2368251489 conn(0x7efc5c0c3880 msgr2=0x7efc5c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 -- 172.21.15.154:0/2368251489 shutdown_connections 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.002+0000 7efc62bf2700 1 -- 172.21.15.154:0/2368251489 wait complete. 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.003+0000 7efc62bf2700 1 Processor -- start 2024-03-14T09:02:38.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.003+0000 7efc62bf2700 1 -- start start 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c06acb0 0x7efc5c14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc5c14eaf0 0x7efc5c0c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c149200 con 0x7efc5c156500 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1494a0 con 0x7efc5c14eaf0 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc62bf2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c149780 con 0x7efc5c06acb0 2024-03-14T09:02:38.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44832/0 (socket says 172.21.15.154:44832) 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 -- 172.21.15.154:0/253180211 learned_addr learned my addr 172.21.15.154:0/253180211 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6098e700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c06acb0 0x7efc5c14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc5bfff700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc5c14eaf0 0x7efc5c0c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 -- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c06acb0 msgr2=0x7efc5c14add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c06acb0 0x7efc5c14add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 -- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc5c14eaf0 msgr2=0x7efc5c0c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc5c14eaf0 0x7efc5c0c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 -- 172.21.15.154:0/253180211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009550 con 0x7efc5c156500 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc6118f700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc54007df0 tx=0x7efc5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400dd20 con 0x7efc5c156500 2024-03-14T09:02:38.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.004+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5400f040 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.005+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54003c60 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.005+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c149a60 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.005+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c14b310 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.006+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c0697d0 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.006+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7efc54004030 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.006+0000 7efc597fa700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 0x7efc4006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.006+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7efc540890a0 con 0x7efc5c156500 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.007+0000 7efc6098e700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 0x7efc4006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.007+0000 7efc6098e700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 0x7efc4006ccf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc50009c40 tx=0x7efc50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.011+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7efc54054230 con 0x7efc5c156500 2024-03-14T09:02:38.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.204+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc5c06b470 con 0x7efc5c156500 2024-03-14T09:02:38.204 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.204+0000 7efc597fa700 1 -- 172.21.15.154:0/253180211 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc54057850 con 0x7efc5c156500 2024-03-14T09:02:38.204 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:38.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 msgr2=0x7efc4006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 0x7efc4006ccf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc50009c40 tx=0x7efc50008030 comp rx=0 tx=0).stop 2024-03-14T09:02:38.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 msgr2=0x7efc5c1443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc54007df0 tx=0x7efc5400d3b0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 shutdown_connections 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efc4006a830 0x7efc4006ccf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c156500 0x7efc5c1443b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efc5c06acb0 0x7efc5c14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 --2- 172.21.15.154:0/253180211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efc5c14eaf0 0x7efc5c0c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 >> 172.21.15.154:0/253180211 conn(0x7efc5c0c3880 msgr2=0x7efc5c14c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 shutdown_connections 2024-03-14T09:02:38.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.206+0000 7efc62bf2700 1 -- 172.21.15.154:0/253180211 wait complete. 2024-03-14T09:02:38.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:02:38.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.342+0000 7f5c28874700 1 Processor -- start 2024-03-14T09:02:38.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.342+0000 7f5c28874700 1 -- start start 2024-03-14T09:02:38.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c28874700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5c240c4380 0x7f5c2414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c28874700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c240c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c28874700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c240acd10 con 0x7f5c240702f0 2024-03-14T09:02:38.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c28874700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c24150110 con 0x7f5c240c4380 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c28874700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240ac7b0 con 0x7f5c240c5d60 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c2259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5c240702f0 0x7f5c240c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58898/0 (socket says 172.21.15.154:58898) 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c2259c700 1 -- 172.21.15.154:0/206981173 learned_addr learned my addr 172.21.15.154:0/206981173 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c22d9d700 1 --2- 172.21.15.154:0/206981173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c240c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c21d9b700 1 --2- 172.21.15.154:0/206981173 >> v2:172.21.15.191:3300/0 conn(0x7f5c240c4380 0x7f5c2414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341951107 0 0) 0x7f5c240acd10 con 0x7f5c240702f0 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c0035b0 con 0x7f5c240702f0 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c22d9d700 1 -- 172.21.15.154:0/206981173 >> v2:172.21.15.191:3300/0 conn(0x7f5c240c4380 msgr2=0x7f5c2414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c22d9d700 1 --2- 172.21.15.154:0/206981173 >> v2:172.21.15.191:3300/0 conn(0x7f5c240c4380 0x7f5c2414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c22d9d700 1 -- 172.21.15.154:0/206981173 >> v1:172.21.15.154:6789/0 conn(0x7f5c240702f0 legacy=0x7f5c240c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075318031 0 0) 0x7f5c0c0035b0 con 0x7f5c240702f0 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.343+0000 7f5c22d9d700 1 -- 172.21.15.154:0/206981173 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c240ac9c0 con 0x7f5c240c5d60 2024-03-14T09:02:38.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c22d9d700 1 --2- 172.21.15.154:0/206981173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c240c61b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009d90 tx=0x7f5c1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435f8fd4603a5afc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00e070 con 0x7f5c240c5d60 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c003a20 con 0x7f5c240c5d60 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c2159a700 1 -- 172.21.15.154:0/206981173 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14001670 con 0x7f5c240c5d60 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 -- 172.21.15.154:0/206981173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 msgr2=0x7f5c240c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 --2- 172.21.15.154:0/206981173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c240c61b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009d90 tx=0x7f5c1c00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 -- 172.21.15.154:0/206981173 shutdown_connections 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 --2- 172.21.15.154:0/206981173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c240c61b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 --2- 172.21.15.154:0/206981173 >> v2:172.21.15.191:3300/0 conn(0x7f5c240c4380 0x7f5c2414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 -- 172.21.15.154:0/206981173 >> 172.21.15.154:0/206981173 conn(0x7f5c240a9bf0 msgr2=0x7f5c240a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 -- 172.21.15.154:0/206981173 shutdown_connections 2024-03-14T09:02:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.344+0000 7f5c28874700 1 -- 172.21.15.154:0/206981173 wait complete. 2024-03-14T09:02:38.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 Processor -- start 2024-03-14T09:02:38.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 -- start start 2024-03-14T09:02:38.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 0x7f5c24151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c241e8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c241e8d50 con 0x7f5c240702f0 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2414b3c0 con 0x7f5c240c4380 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c28874700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2414aa20 con 0x7f5c240c5d60 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.346+0000 7f5c2259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44836/0 (socket says 172.21.15.154:44836) 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 -- 172.21.15.154:0/2865809213 learned_addr learned my addr 172.21.15.154:0/2865809213 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c21d9b700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 0x7f5c24151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c22d9d700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c241e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 -- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 msgr2=0x7f5c241e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c241e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 -- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 msgr2=0x7f5c24151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 0x7f5c24151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 -- 172.21.15.154:0/2865809213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c009a70 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c21d9b700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 0x7f5c24151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c2259c700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c750 tx=0x7f5c1400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14004c80 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c14004de0 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2414ad00 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c140072f0 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.347+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2414c6d0 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.348+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c240697d0 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.349+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5c14007450 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.349+0000 7f5c12ffd700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 0x7f5c0406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.349+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f5c14088df0 con 0x7f5c240702f0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.349+0000 7f5c21d9b700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 0x7f5c0406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.350+0000 7f5c21d9b700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 0x7f5c0406ccf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c240c3830 tx=0x7f5c18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.354+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5c14053f40 con 0x7f5c240702f0 2024-03-14T09:02:38.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.546+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c2414f3f0 con 0x7f5c240702f0 2024-03-14T09:02:38.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.546+0000 7f5c12ffd700 1 -- 172.21.15.154:0/2865809213 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c14007700 con 0x7f5c240702f0 2024-03-14T09:02:38.546 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:38.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 msgr2=0x7f5c0406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 0x7f5c0406ccf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c240c3830 tx=0x7f5c18008040 comp rx=0 tx=0).stop 2024-03-14T09:02:38.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 msgr2=0x7f5c24151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c750 tx=0x7f5c1400cb10 comp rx=0 tx=0).stop 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 shutdown_connections 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c240c5d60 0x7f5c241e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c0406a830 0x7f5c0406ccf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c240702f0 0x7f5c24151210 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 --2- 172.21.15.154:0/2865809213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c240c4380 0x7f5c24151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 >> 172.21.15.154:0/2865809213 conn(0x7f5c240a9bf0 msgr2=0x7f5c24148970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 shutdown_connections 2024-03-14T09:02:38.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.548+0000 7f5c28874700 1 -- 172.21.15.154:0/2865809213 wait complete. 2024-03-14T09:02:38.560 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:02:38.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.685+0000 7f4529772700 1 Processor -- start 2024-03-14T09:02:38.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.685+0000 7f4529772700 1 -- start start 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 0x7f45240c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45241550e0 0x7f452406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452406acb0 0x7f4524157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452406dc90 con 0x7f45241550e0 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452406f5e0 con 0x7f45240c1080 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4529772700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4524154c50 con 0x7f452406acb0 2024-03-14T09:02:38.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45241550e0 0x7f452406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45241550e0 0x7f452406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44840/0 (socket says 172.21.15.154:44840) 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 -- 172.21.15.154:0/3409983021 learned_addr learned my addr 172.21.15.154:0/3409983021 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4522ffd700 1 --2- 172.21.15.154:0/3409983021 >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 0x7f45240c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45237fe700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452406acb0 0x7f4524157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 -- 172.21.15.154:0/3409983021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452406acb0 msgr2=0x7f4524157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452406acb0 0x7f4524157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 -- 172.21.15.154:0/3409983021 >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 msgr2=0x7f45240c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 --2- 172.21.15.154:0/3409983021 >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 0x7f45240c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 -- 172.21.15.154:0/3409983021 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240c0860 con 0x7f45241550e0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f4522ffd700 1 --2- 172.21.15.154:0/3409983021 >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 0x7f45240c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.686+0000 7f45227fc700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45241550e0 0x7f452406a770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4524152580 tx=0x7f4514009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe50b13f4e0a91db server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4521ffb700 1 -- 172.21.15.154:0/3409983021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400b870 con 0x7f45241550e0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4521ffb700 1 -- 172.21.15.154:0/3409983021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451400b9d0 con 0x7f45241550e0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4521ffb700 1 -- 172.21.15.154:0/3409983021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400bd60 con 0x7f45241550e0 2024-03-14T09:02:38.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 -- 172.21.15.154:0/3409983021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45241550e0 msgr2=0x7f452406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45241550e0 0x7f452406a770 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4524152580 tx=0x7f4514009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 -- 172.21.15.154:0/3409983021 shutdown_connections 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f452406acb0 0x7f4524157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 --2- 172.21.15.154:0/3409983021 >> v2:172.21.15.191:3300/0 conn(0x7f45240c1080 0x7f45240c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 --2- 172.21.15.154:0/3409983021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45241550e0 0x7f452406a770 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 -- 172.21.15.154:0/3409983021 >> 172.21.15.154:0/3409983021 conn(0x7f45240c39f0 msgr2=0x7f45240c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 -- 172.21.15.154:0/3409983021 shutdown_connections 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.687+0000 7f4529772700 1 -- 172.21.15.154:0/3409983021 wait complete. 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.688+0000 7f4529772700 1 Processor -- start 2024-03-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.688+0000 7f4529772700 1 -- start start 2024-03-14T09:02:38.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 0x7f452414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 0x7f45241ea150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4524156cb0 con 0x7f452406acb0 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4524158680 con 0x7f45241550e0 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4529772700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4524157d90 con 0x7f45240c1080 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44850/0 (socket says 172.21.15.154:44850) 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 -- 172.21.15.154:0/4285264521 learned_addr learned my addr 172.21.15.154:0/4285264521 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f45227fc700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 0x7f452414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f45237fe700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 0x7f45241ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 -- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 msgr2=0x7f452414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 0x7f452414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 -- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 msgr2=0x7f45241ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 0x7f45241ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f4522ffd700 1 -- 172.21.15.154:0/4285264521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4514009550 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f45227fc700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 0x7f452414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.689+0000 7f45237fe700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 0x7f45241ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f4522ffd700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f451800c750 tx=0x7f451800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518011800 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518011960 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4524147bd0 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518011cd0 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.690+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4524158310 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.691+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452404ea80 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.691+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4518021350 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.691+0000 7f450b7fe700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 0x7f451006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.692+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4518088fd0 con 0x7f452406acb0 2024-03-14T09:02:38.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.692+0000 7f45227fc700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 0x7f451006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:38.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.692+0000 7f45227fc700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 0x7f451006ccf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4514014040 tx=0x7f4514008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:38.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.696+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4518054120 con 0x7f452406acb0 2024-03-14T09:02:38.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.890+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45240697d0 con 0x7f452406acb0 2024-03-14T09:02:38.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.890+0000 7f450b7fe700 1 -- 172.21.15.154:0/4285264521 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v36) v1 ==== 105+0+11 (secure 0 0 0) 0x7f4518057740 con 0x7f452406acb0 2024-03-14T09:02:38.890 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 msgr2=0x7f451006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 0x7f451006ccf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4514014040 tx=0x7f4514008030 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 msgr2=0x7f4524147eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f451800c750 tx=0x7f451800cb10 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 shutdown_connections 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45241550e0 0x7f45241ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f451006a830 0x7f451006ccf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452406acb0 0x7f4524147eb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 --2- 172.21.15.154:0/4285264521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f45240c1080 0x7f452414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.892+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 >> 172.21.15.154:0/4285264521 conn(0x7f45240c39f0 msgr2=0x7f45241512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.893+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 shutdown_connections 2024-03-14T09:02:38.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:38.893+0000 7f4529772700 1 -- 172.21.15.154:0/4285264521 wait complete. 2024-03-14T09:02:38.905 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:02:39.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.029+0000 7fb0e2690700 1 Processor -- start 2024-03-14T09:02:39.029 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.029+0000 7fb0e2690700 1 -- start start 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e2690700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e2690700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0dc156960 0x7fb0dc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e2690700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc156050 con 0x7fb0dc1564e0 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e2690700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0a86f0 con 0x7fb0dc156960 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e2690700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c34b0 con 0x7fb0dc0a9650 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0e0c2d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0dc156960 0x7fb0dc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0db7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fb0dc1564e0 0x7fb0dc06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:47190/0 (socket says 172.21.15.154:47190) 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:43786/0 (socket says 172.21.15.154:43786) 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.030+0000 7fb0db7fe700 1 -- 172.21.15.154:0/240968380 learned_addr learned my addr 172.21.15.154:0/240968380 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:39.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0daffd700 1 -- 172.21.15.154:0/240968380 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174637982 0 0) 0x7fb0dc156050 con 0x7fb0dc1564e0 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0daffd700 1 -- 172.21.15.154:0/240968380 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0c40035b0 con 0x7fb0dc1564e0 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 msgr2=0x7fb0dc0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0e0c2d700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/240968380 >> v1:172.21.15.191:6789/0 conn(0x7fb0dc1564e0 legacy=0x7fb0dc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/240968380 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0a8990 con 0x7fb0dc156960 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0dbfff700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0e0c2d700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc156960 0x7fb0dc15cfd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009d90 tx=0x7fb0d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dfd51c3cc135c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0daffd700 1 -- 172.21.15.154:0/240968380 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400e070 con 0x7fb0dc156960 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0daffd700 1 -- 172.21.15.154:0/240968380 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400bb20 con 0x7fb0dc156960 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.031+0000 7fb0daffd700 1 -- 172.21.15.154:0/240968380 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4012430 con 0x7fb0dc156960 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 -- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc156960 msgr2=0x7fb0dc15cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc156960 0x7fb0dc15cfd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009d90 tx=0x7fb0d400b330 comp rx=0 tx=0).stop 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 -- 172.21.15.154:0/240968380 shutdown_connections 2024-03-14T09:02:39.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc156960 0x7fb0dc15cfd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 --2- 172.21.15.154:0/240968380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc0a9650 0x7fb0dc0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 -- 172.21.15.154:0/240968380 >> 172.21.15.154:0/240968380 conn(0x7fb0dc0c3880 msgr2=0x7fb0dc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 -- 172.21.15.154:0/240968380 shutdown_connections 2024-03-14T09:02:39.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.032+0000 7fb0e2690700 1 -- 172.21.15.154:0/240968380 wait complete. 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.033+0000 7fb0e2690700 1 Processor -- start 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.033+0000 7fb0e2690700 1 -- start start 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0a9650 0x7fb0dc13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc1564e0 0x7fb0dc13a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc13b310 con 0x7fb0dc156960 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc13a400 con 0x7fb0dc0a9650 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e2690700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c5de0 con 0x7fb0dc1564e0 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44864/0 (socket says 172.21.15.154:44864) 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/1879227021 learned_addr learned my addr 172.21.15.154:0/1879227021 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0dbfff700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0a9650 0x7fb0dc13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0db7fe700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc1564e0 0x7fb0dc13a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc1564e0 msgr2=0x7fb0dc13a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc1564e0 0x7fb0dc13a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0a9650 msgr2=0x7fb0dc13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0a9650 0x7fb0dc13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 -- 172.21.15.154:0/1879227021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00094a0 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.034+0000 7fb0e0c2d700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b370 tx=0x7fb0d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4012430 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4020070 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4004280 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d4009a70 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc0c63a0 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.035+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc0c2d90 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.036+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb0d40043e0 con 0x7fb0dc156960 2024-03-14T09:02:39.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.036+0000 7fb0d8ff9700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 0x7fb0c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.036+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb0d4022070 con 0x7fb0dc156960 2024-03-14T09:02:39.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.037+0000 7fb0dbfff700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 0x7fb0c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.037+0000 7fb0dbfff700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 0x7fb0c806ccf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc13f630 tx=0x7fb0d0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.041+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb0d4055d20 con 0x7fb0dc156960 2024-03-14T09:02:39.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.233+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fb0dc0697d0 con 0x7fb0dc156960 2024-03-14T09:02:39.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.233+0000 7fb0d8ff9700 1 -- 172.21.15.154:0/1879227021 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v36) v1 ==== 101+0+11 (secure 0 0 0) 0x7fb0d402c020 con 0x7fb0dc156960 2024-03-14T09:02:39.233 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.235+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 msgr2=0x7fb0c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.235+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 0x7fb0c806ccf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc13f630 tx=0x7fb0d0009210 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.235+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 msgr2=0x7fb0dc13add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.235+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b370 tx=0x7fb0d4004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 shutdown_connections 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb0c806a830 0x7fb0c806ccf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0dc156960 0x7fb0dc13add0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0dc0a9650 0x7fb0dc13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 --2- 172.21.15.154:0/1879227021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0dc1564e0 0x7fb0dc13a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 >> 172.21.15.154:0/1879227021 conn(0x7fb0dc0c3880 msgr2=0x7fb0dc0abca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 shutdown_connections 2024-03-14T09:02:39.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.236+0000 7fb0e2690700 1 -- 172.21.15.154:0/1879227021 wait complete. 2024-03-14T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:39.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.368+0000 7fc4d357e700 1 Processor -- start 2024-03-14T09:02:39.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.368+0000 7fc4d357e700 1 -- start start 2024-03-14T09:02:39.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.369+0000 7fc4d357e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.369+0000 7fc4d357e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.369+0000 7fc4d357e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4cc13ade0 con 0x7fc4cc0a8540 2024-03-14T09:02:39.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.369+0000 7fc4d357e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc06fb30 con 0x7fc4cc0c3250 2024-03-14T09:02:39.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.369+0000 7fc4d357e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc06ec30 con 0x7fc4cc0c1780 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d1b1b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:46924/0 (socket says 172.21.15.154:46924) 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 -- 172.21.15.154:0/4275002706 learned_addr learned my addr 172.21.15.154:0/4275002706 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d131a700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc4cc0a8540 0x7fc4cc154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:33054/0 (socket says 172.21.15.154:33054) 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 -- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 msgr2=0x7fc4cc0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc0c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 -- 172.21.15.154:0/4275002706 >> v1:172.21.15.191:6789/0 conn(0x7fc4cc0a8540 legacy=0x7fc4cc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 -- 172.21.15.154:0/4275002706 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc06ef10 con 0x7fc4cc0c3250 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d1b1b700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4c3fff700 1 -- 172.21.15.154:0/4275002706 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251888746 0 0) 0x7fc4bc001110 con 0x7fc4cc0a8540 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4d0b19700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80098f0 tx=0x7fc4c8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb3eef97d4f9273 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4c3fff700 1 -- 172.21.15.154:0/4275002706 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8011830 con 0x7fc4cc0c3250 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4c3fff700 1 -- 172.21.15.154:0/4275002706 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c8011990 con 0x7fc4cc0c3250 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.370+0000 7fc4c3fff700 1 -- 172.21.15.154:0/4275002706 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8011d20 con 0x7fc4cc0c3250 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 -- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0c3250 msgr2=0x7fc4cc13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80098f0 tx=0x7fc4c8009c00 comp rx=0 tx=0).stop 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 -- 172.21.15.154:0/4275002706 shutdown_connections 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 --2- 172.21.15.154:0/4275002706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc13a8a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 -- 172.21.15.154:0/4275002706 >> 172.21.15.154:0/4275002706 conn(0x7fc4cc0c3880 msgr2=0x7fc4cc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 -- 172.21.15.154:0/4275002706 shutdown_connections 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.371+0000 7fc4d357e700 1 -- 172.21.15.154:0/4275002706 wait complete. 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.372+0000 7fc4d357e700 1 Processor -- start 2024-03-14T09:02:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.372+0000 7fc4d357e700 1 -- start start 2024-03-14T09:02:39.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0a8540 0x7fc4cc151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc14fa20 con 0x7fc4cc0c3250 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc151de0 con 0x7fc4cc0a8540 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d357e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc14f3f0 con 0x7fc4cc0c1780 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37444/0 (socket says 172.21.15.154:37444) 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 -- 172.21.15.154:0/2210507181 learned_addr learned my addr 172.21.15.154:0/2210507181 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d131a700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0a8540 0x7fc4cc151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d0b19700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 -- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 msgr2=0x7fc4cc1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 -- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0a8540 msgr2=0x7fc4cc151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0a8540 0x7fc4cc151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.373+0000 7fc4d1b1b700 1 -- 172.21.15.154:0/2210507181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4009a70 con 0x7fc4cc0c3250 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4d1b1b700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400efc0 tx=0x7fc4c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400ca90 con 0x7fc4cc0c3250 2024-03-14T09:02:39.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400f040 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c8009550 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400ce30 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.374+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc1e6f20 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.375+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc14fb60 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.375+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc4c4014480 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.375+0000 7fc4c1ffb700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 0x7fc4b006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.376+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc4c40896b0 con 0x7fc4cc0c3250 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.376+0000 7fc4d131a700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 0x7fc4b006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.376+0000 7fc4d131a700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 0x7fc4b006ccf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000f60 tx=0x7fc4bc007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.380+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc4c4054800 con 0x7fc4cc0c3250 2024-03-14T09:02:39.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.618+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4cc0697d0 con 0x7fc4cc0c3250 2024-03-14T09:02:39.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.619+0000 7fc4c1ffb700 1 -- 172.21.15.154:0/2210507181 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5951 (secure 0 0 0) 0x7fc4c4057e20 con 0x7fc4cc0c3250 2024-03-14T09:02:39.618 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:39.619 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 msgr2=0x7fc4b006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 0x7fc4b006ccf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000f60 tx=0x7fc4bc007e70 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 msgr2=0x7fc4cc14e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400efc0 tx=0x7fc4c400c3a0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 shutdown_connections 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4b006a830 0x7fc4b006ccf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4cc0c3250 0x7fc4cc14e2e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4cc0a8540 0x7fc4cc151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 --2- 172.21.15.154:0/2210507181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4cc0c1780 0x7fc4cc1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 >> 172.21.15.154:0/2210507181 conn(0x7fc4cc0c3880 msgr2=0x7fc4cc0c4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 shutdown_connections 2024-03-14T09:02:39.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.621+0000 7fc4d357e700 1 -- 172.21.15.154:0/2210507181 wait complete. 2024-03-14T09:02:39.623 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 20 2024-03-14T09:02:39.633 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.755+0000 7f34ad6e5700 1 Processor -- start 2024-03-14T09:02:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.755+0000 7f34ad6e5700 1 -- start start 2024-03-14T09:02:39.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34ad6e5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34a8070a10 0x7f34a80c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34ad6e5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80c5f10 con 0x7f34a806c6e0 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34ad6e5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a80c7d50 con 0x7f34a80c31c0 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34ad6e5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a80c8030 con 0x7f34a8070a10 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34a67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34a8070a10 0x7f34a80c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34a6ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f34a806c6e0 0x7f34a806d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55468/0 (socket says 172.21.15.154:55468) 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.756+0000 7f34a6ffd700 1 -- 172.21.15.154:0/2627246837 learned_addr learned my addr 172.21.15.154:0/2627246837 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a67fc700 1 -- 172.21.15.154:0/2627246837 >> v1:172.21.15.191:6789/0 conn(0x7f34a80c31c0 legacy=0x7f34a80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a67fc700 1 -- 172.21.15.154:0/2627246837 >> v1:172.21.15.154:6789/0 conn(0x7f34a806c6e0 legacy=0x7f34a806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a5ffb700 1 -- 172.21.15.154:0/2627246837 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087494573 0 0) 0x7f34a80c5f10 con 0x7f34a806c6e0 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a67fc700 1 -- 172.21.15.154:0/2627246837 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a80c8310 con 0x7f34a8070a10 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a67fc700 1 --2- 172.21.15.154:0/2627246837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a8070a10 0x7f34a80c63a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3498009840 tx=0x7f3498009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db539d4d2fd23e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a5ffb700 1 -- 172.21.15.154:0/2627246837 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498004230 con 0x7f34a8070a10 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a5ffb700 1 -- 172.21.15.154:0/2627246837 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3498004390 con 0x7f34a8070a10 2024-03-14T09:02:39.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34a5ffb700 1 -- 172.21.15.154:0/2627246837 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980036a0 con 0x7f34a8070a10 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2627246837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a8070a10 msgr2=0x7f34a80c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2627246837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a8070a10 0x7f34a80c63a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3498009840 tx=0x7f3498009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.757+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2627246837 shutdown_connections 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.758+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2627246837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a8070a10 0x7f34a80c63a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.758+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2627246837 >> 172.21.15.154:0/2627246837 conn(0x7f34a80a9bf0 msgr2=0x7f34a80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.758+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2627246837 shutdown_connections 2024-03-14T09:02:39.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.758+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2627246837 wait complete. 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.759+0000 7f34ad6e5700 1 Processor -- start 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.759+0000 7f34ad6e5700 1 -- start start 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.759+0000 7f34ad6e5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.759+0000 7f34ad6e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34a8070a10 0x7f34a80abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.759+0000 7f34ad6e5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80c31c0 0x7f34a81dc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34ad6e5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a809a470 con 0x7f34a806c6e0 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34ad6e5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8099bf0 con 0x7f34a80c31c0 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34ad6e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8099eb0 con 0x7f34a8070a10 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37460/0 (socket says 172.21.15.154:37460) 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 -- 172.21.15.154:0/2473356187 learned_addr learned my addr 172.21.15.154:0/2473356187 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:39.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a77fe700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80c31c0 0x7f34a81dc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a67fc700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34a8070a10 0x7f34a80abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 -- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34a8070a10 msgr2=0x7f34a80abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34a8070a10 0x7f34a80abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 -- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80c31c0 msgr2=0x7f34a81dc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80c31c0 0x7f34a81dc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 -- 172.21.15.154:0/2473356187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980094a0 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34a6ffd700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f349c0011e0 tx=0x7f349c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c004c80 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c004e00 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a809a190 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c007310 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.760+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a81dcc00 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.761+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a80ab850 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.762+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f349c007470 con 0x7f34a806c6e0 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.762+0000 7f348f7fe700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 0x7f349406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:39.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.762+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f349c088e60 con 0x7f34a806c6e0 2024-03-14T09:02:39.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.762+0000 7f34a67fc700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 0x7f349406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:39.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.763+0000 7f34a67fc700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 0x7f349406ccf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34a8070ea0 tx=0x7f349801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:39.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.766+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f34a80ab850 con 0x7f34a806c6e0 2024-03-14T09:02:39.961 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.961+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34a80ab850 con 0x7f34a806c6e0 2024-03-14T09:02:39.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.963+0000 7f348f7fe700 1 -- 172.21.15.154:0/2473356187 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7f34a80ab850 con 0x7f34a806c6e0 2024-03-14T09:02:39.963 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:39.963 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 msgr2=0x7f349406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 0x7f349406ccf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34a8070ea0 tx=0x7f349801c040 comp rx=0 tx=0).stop 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 msgr2=0x7f34a80acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f349c0011e0 tx=0x7f349c00cae0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 shutdown_connections 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80c31c0 0x7f34a81dc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f349406a830 0x7f349406ccf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34a806c6e0 0x7f34a80acb70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 --2- 172.21.15.154:0/2473356187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34a8070a10 0x7f34a80abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:39.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 >> 172.21.15.154:0/2473356187 conn(0x7f34a80a9bf0 msgr2=0x7f34a80c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:39.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 shutdown_connections 2024-03-14T09:02:39.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:39.965+0000 7f34ad6e5700 1 -- 172.21.15.154:0/2473356187 wait complete. 2024-03-14T09:02:39.977 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T09:02:39.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:40.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.099+0000 7faa0dbbf700 1 Processor -- start 2024-03-14T09:02:40.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.099+0000 7faa0dbbf700 1 -- start start 2024-03-14T09:02:40.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa0dbbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa0dbbf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa08072710 0x7faa080c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa0dbbf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa08070e10 con 0x7faa08072290 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa0dbbf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa080c1bb0 con 0x7faa08072710 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa0dbbf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080c6170 con 0x7faa0814c5c0 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa08072710 0x7faa080c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa08072710 0x7faa080c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37466/0 (socket says 172.21.15.154:37466) 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 -- 172.21.15.154:0/1419747808 learned_addr learned my addr 172.21.15.154:0/1419747808 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:40.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa077fe700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 -- 172.21.15.154:0/1419747808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 msgr2=0x7faa08152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 -- 172.21.15.154:0/1419747808 >> v1:172.21.15.191:6789/0 conn(0x7faa08072290 legacy=0x7faa080c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 -- 172.21.15.154:0/1419747808 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0814c9a0 con 0x7faa08072710 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa077fe700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.100+0000 7faa07fff700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072710 0x7faa080c26e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee08c6ad4c68b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa067fc700 1 -- 172.21.15.154:0/1419747808 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000e070 con 0x7faa08072710 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa067fc700 1 -- 172.21.15.154:0/1419747808 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0000b9c0 con 0x7faa08072710 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa067fc700 1 -- 172.21.15.154:0/1419747808 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000bd70 con 0x7faa08072710 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1419747808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072710 msgr2=0x7faa080c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072710 0x7faa080c26e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1419747808 shutdown_connections 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1419747808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072710 0x7faa080c26e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1419747808 >> 172.21.15.154:0/1419747808 conn(0x7faa0806a770 msgr2=0x7faa0806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1419747808 shutdown_connections 2024-03-14T09:02:40.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.101+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1419747808 wait complete. 2024-03-14T09:02:40.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.102+0000 7faa0dbbf700 1 Processor -- start 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 -- start start 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08072710 0x7faa081ea970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08147300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081499c0 con 0x7faa08072290 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081ee110 con 0x7faa08072710 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa0dbbf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa081ea170 con 0x7faa0814c5c0 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa077fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa077fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37476/0 (socket says 172.21.15.154:37476) 2024-03-14T09:02:40.108 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.103+0000 7faa077fe700 1 -- 172.21.15.154:0/1358747720 learned_addr learned my addr 172.21.15.154:0/1358747720 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa07fff700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 -- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 msgr2=0x7faa08147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa06ffd700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08072710 0x7faa081ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08147300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 -- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08072710 msgr2=0x7faa081ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08072710 0x7faa081ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 -- 172.21.15.154:0/1358747720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00009b30 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa077fe700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e810 tx=0x7fa9f000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000c020 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f0004250 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa081ea450 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00036a0 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.104+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa080ac350 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.105+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa080c2da0 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.105+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa9f0003800 con 0x7faa08072290 2024-03-14T09:02:40.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.106+0000 7fa9fffff700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 0x7fa9e806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.106+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa9f0088dd0 con 0x7faa08072290 2024-03-14T09:02:40.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.106+0000 7faa06ffd700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 0x7fa9e806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.106+0000 7faa06ffd700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 0x7fa9e806cd20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faa080c6410 tx=0x7fa9f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.110+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa9f0053f20 con 0x7faa08072290 2024-03-14T09:02:40.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.347+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa080697d0 con 0x7faa08072290 2024-03-14T09:02:40.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.348+0000 7fa9fffff700 1 -- 172.21.15.154:0/1358747720 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5951 (secure 0 0 0) 0x7fa9f0057540 con 0x7faa08072290 2024-03-14T09:02:40.348 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:40.348 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 msgr2=0x7fa9e806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 0x7fa9e806cd20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faa080c6410 tx=0x7fa9f8009040 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 msgr2=0x7faa0806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000e810 tx=0x7fa9f000ebd0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 shutdown_connections 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa0814c5c0 0x7faa08147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa9e806a860 0x7fa9e806cd20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa08072290 0x7faa0806da00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 --2- 172.21.15.154:0/1358747720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08072710 0x7faa081ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 >> 172.21.15.154:0/1358747720 conn(0x7faa0806a770 msgr2=0x7faa08150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:40.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 shutdown_connections 2024-03-14T09:02:40.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.350+0000 7faa0dbbf700 1 -- 172.21.15.154:0/1358747720 wait complete. 2024-03-14T09:02:40.351 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 20 2024-03-14T09:02:40.362 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.484+0000 7f8bc9d56700 1 Processor -- start 2024-03-14T09:02:40.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.484+0000 7f8bc9d56700 1 -- start start 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc9d56700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bc4070a10 0x7f8bc40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc9d56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bc40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc9d56700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc40c5f10 con 0x7f8bc406c6e0 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc9d56700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40c7d50 con 0x7f8bc4070a10 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc9d56700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40c8030 con 0x7f8bc40c31c0 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc37fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8bc406c6e0 0x7f8bc406d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55482/0 (socket says 172.21.15.154:55482) 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc37fe700 1 -- 172.21.15.154:0/3956222905 learned_addr learned my addr 172.21.15.154:0/3956222905 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:40.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152771960 0 0) 0x7f8bc40c5f10 con 0x7f8bc406c6e0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb00035b0 con 0x7f8bc406c6e0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/3956222905 >> v2:172.21.15.191:3300/0 conn(0x7f8bc4070a10 0x7f8bc40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc3fff700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bc40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190579001 0 0) 0x7f8bb00035b0 con 0x7f8bc406c6e0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 msgr2=0x7f8bc40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bc40c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 >> v2:172.21.15.191:3300/0 conn(0x7f8bc4070a10 msgr2=0x7f8bc40c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 --2- 172.21.15.154:0/3956222905 >> v2:172.21.15.191:3300/0 conn(0x7f8bc4070a10 0x7f8bc40c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40c8310 con 0x7f8bc406c6e0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.485+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8bb4001670 con 0x7f8bc406c6e0 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 >> v1:172.21.15.154:6789/0 conn(0x7f8bc406c6e0 legacy=0x7f8bc406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bb0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb0004d90 0x7f8bb0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40c7d50 con 0x7f8bb0003f80 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40c8030 con 0x7f8bb0004d90 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb000ba60 con 0x7f8bc40c31c0 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc3fff700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb0004d90 0x7f8bb0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc37fe700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bb0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 -- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 msgr2=0x7f8bb0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bb0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 -- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb0004d90 msgr2=0x7f8bb0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bb0004d90 0x7f8bb0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 -- 172.21.15.154:0/3956222905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00035b0 con 0x7f8bb0003f80 2024-03-14T09:02:40.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc37fe700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc40c31c0 0x7f8bb0003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bb0004850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009b60 tx=0x7f8bb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed776d02c1671a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00e6c0 con 0x7f8bb0003f80 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00e820 con 0x7f8bb0003f80 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.486+0000 7f8bc27fc700 1 -- 172.21.15.154:0/3956222905 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00eb90 con 0x7f8bb0003f80 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 -- 172.21.15.154:0/3956222905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 msgr2=0x7f8bb0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/3956222905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bb0004850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009b60 tx=0x7f8bb0005880 comp rx=0 tx=0).stop 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc3fff700 1 -- 172.21.15.154:0/3956222905 reap_dead start 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 -- 172.21.15.154:0/3956222905 shutdown_connections 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 -- 172.21.15.154:0/3956222905 >> 172.21.15.154:0/3956222905 conn(0x7f8bc40a9bf0 msgr2=0x7f8bc40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 -- 172.21.15.154:0/3956222905 shutdown_connections 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.487+0000 7f8bc9d56700 1 -- 172.21.15.154:0/3956222905 wait complete. 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.488+0000 7f8bc9d56700 1 Processor -- start 2024-03-14T09:02:40.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.488+0000 7f8bc9d56700 1 -- start start 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 0x7f8bc41edce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 0x7f8bc41ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41ed5e0 con 0x7f8bb0003f80 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc409a980 con 0x7f8bc409aff0 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc9d56700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc409dd60 con 0x7f8bc41ee220 2024-03-14T09:02:40.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37490/0 (socket says 172.21.15.154:37490) 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 -- 172.21.15.154:0/1686792671 learned_addr learned my addr 172.21.15.154:0/1686792671 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 0x7f8bc41edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc3fff700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 0x7f8bc41ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 -- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 msgr2=0x7f8bc41ddab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 0x7f8bc41ddab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 -- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 msgr2=0x7f8bc41edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 0x7f8bc41edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 -- 172.21.15.154:0/1686792671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40a66e0 con 0x7f8bb0003f80 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc3fff700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 0x7f8bc41ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 0x7f8bc41edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.489+0000 7f8bc37fe700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400eb50 tx=0x7f8bb400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.490+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400c700 con 0x7f8bb0003f80 2024-03-14T09:02:40.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.490+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb400c860 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.490+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400cbd0 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.490+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc40a69c0 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.490+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc406ee00 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.491+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc41eda00 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.491+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8bb400cd30 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.491+0000 7f8bbbfff700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 0x7f8ba406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.492+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8bb4088f20 con 0x7f8bb0003f80 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.492+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 0x7f8ba406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.492+0000 7f8bc2ffd700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 0x7f8ba406ccf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0090b0 tx=0x7f8bac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.496+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8bc41eda00 con 0x7f8bb0003f80 2024-03-14T09:02:40.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.735+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bc41eda00 con 0x7f8bb0003f80 2024-03-14T09:02:40.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.736+0000 7f8bbbfff700 1 -- 172.21.15.154:0/1686792671 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5951 (secure 0 0 0) 0x7f8bc41eda00 con 0x7f8bb0003f80 2024-03-14T09:02:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:40.736 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:03.727658+0000","modified":"2024-03-14T09:02:06.738355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/3981292790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":3981292790},{"type":"v1","addr":"172.21.15.154:6835","nonce":3981292790}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 msgr2=0x7f8ba406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 0x7f8ba406ccf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0090b0 tx=0x7f8bac008030 comp rx=0 tx=0).stop 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 msgr2=0x7f8bc406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400eb50 tx=0x7f8bb400ef10 comp rx=0 tx=0).stop 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 shutdown_connections 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bc41ee220 0x7f8bc41ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ba406a830 0x7f8ba406ccf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bb0003f80 0x7f8bc406e8c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 --2- 172.21.15.154:0/1686792671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bc409aff0 0x7f8bc41edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 >> 172.21.15.154:0/1686792671 conn(0x7f8bc40a9bf0 msgr2=0x7f8bc406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:40.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 shutdown_connections 2024-03-14T09:02:40.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.738+0000 7f8bc9d56700 1 -- 172.21.15.154:0/1686792671 wait complete. 2024-03-14T09:02:40.739 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 20 2024-03-14T09:02:40.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:40.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.872+0000 7f598ba0a700 1 Processor -- start 2024-03-14T09:02:40.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.872+0000 7f598ba0a700 1 -- start start 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f598ba0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f5984160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f598ba0a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59840ac8e0 con 0x7f598414eaf0 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f598ba0a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598406de30 con 0x7f5984155df0 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f598ba0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598406b310 con 0x7f598415e280 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f59897a6700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f598414eaf0 0x7f598406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55486/0 (socket says 172.21.15.154:55486) 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f59897a6700 1 -- 172.21.15.154:0/1898678668 learned_addr learned my addr 172.21.15.154:0/1898678668 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:40.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f5989fa7700 1 --2- 172.21.15.154:0/1898678668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f5984160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821850640 0 0) 0x7f59840ac8e0 con 0x7f598414eaf0 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59700035b0 con 0x7f598414eaf0 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624006249 0 0) 0x7f598406de30 con 0x7f5984155df0 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f5989fa7700 1 -- 172.21.15.154:0/1898678668 >> v1:172.21.15.191:6789/0 conn(0x7f5984155df0 legacy=0x7f5984156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f5989fa7700 1 -- 172.21.15.154:0/1898678668 >> v1:172.21.15.154:6789/0 conn(0x7f598414eaf0 legacy=0x7f598406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f5989fa7700 1 -- 172.21.15.154:0/1898678668 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984156960 con 0x7f598415e280 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.873+0000 7f5989fa7700 1 --2- 172.21.15.154:0/1898678668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f5984160670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f597c009dc0 tx=0x7f597c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e3c4344b3f04d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00e070 con 0x7f598415e280 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c003a20 con 0x7f598415e280 2024-03-14T09:02:40.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f597bfff700 1 -- 172.21.15.154:0/1898678668 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980001670 con 0x7f598415e280 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 -- 172.21.15.154:0/1898678668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 msgr2=0x7f5984160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 --2- 172.21.15.154:0/1898678668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f5984160670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f597c009dc0 tx=0x7f597c00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 -- 172.21.15.154:0/1898678668 shutdown_connections 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 --2- 172.21.15.154:0/1898678668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f5984160670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 -- 172.21.15.154:0/1898678668 >> 172.21.15.154:0/1898678668 conn(0x7f59840c3880 msgr2=0x7f59840c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:40.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 -- 172.21.15.154:0/1898678668 shutdown_connections 2024-03-14T09:02:40.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.874+0000 7f598ba0a700 1 -- 172.21.15.154:0/1898678668 wait complete. 2024-03-14T09:02:40.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.876+0000 7f598ba0a700 1 Processor -- start 2024-03-14T09:02:40.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.876+0000 7f598ba0a700 1 -- start start 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.876+0000 7f598ba0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f598414eaf0 0x7f598414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f598413c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598414cde0 con 0x7f5984155df0 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598414d0c0 con 0x7f598414eaf0 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598414d3a0 con 0x7f598415e280 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37500/0 (socket says 172.21.15.154:37500) 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 -- 172.21.15.154:0/2411459435 learned_addr learned my addr 172.21.15.154:0/2411459435 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:40.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5989fa7700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f598413c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f59897a6700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f598414eaf0 0x7f598414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 -- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 msgr2=0x7f598413c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f598413c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 -- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f598414eaf0 msgr2=0x7f598414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f598414eaf0 0x7f598414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 -- 172.21.15.154:0/2411459435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c009a70 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5988fa5700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f59740011e0 tx=0x7f597400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740054e0 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5974003680 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598414d680 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.877+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974003a30 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.878+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598414e2b0 con 0x7f5984155df0 2024-03-14T09:02:40.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.878+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5984149cf0 con 0x7f5984155df0 2024-03-14T09:02:40.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.879+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5974003b90 con 0x7f5984155df0 2024-03-14T09:02:40.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.879+0000 7f5979ffb700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 0x7f596806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:40.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.879+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f59740885d0 con 0x7f5984155df0 2024-03-14T09:02:40.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.879+0000 7f59897a6700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 0x7f596806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:40.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.880+0000 7f59897a6700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 0x7f596806ccf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f59800017a0 tx=0x7f5980001630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:40.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:40.883+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5984149cf0 con 0x7f5984155df0 2024-03-14T09:02:41.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.077+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5984149cf0 con 0x7f5984155df0 2024-03-14T09:02:41.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.079+0000 7f5979ffb700 1 -- 172.21.15.154:0/2411459435 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+19809 (secure 0 0 0) 0x7f5984149cf0 con 0x7f5984155df0 2024-03-14T09:02:41.079 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:41.079 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":36,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:21.754786+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:01.699649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:02.688126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 msgr2=0x7f596806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 0x7f596806ccf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f59800017a0 tx=0x7f5980001630 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 msgr2=0x7f598413e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f59740011e0 tx=0x7f597400cae0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 shutdown_connections 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f598415e280 0x7f598413c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f598414eaf0 0x7f598414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f596806a830 0x7f596806ccf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 --2- 172.21.15.154:0/2411459435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5984155df0 0x7f598413e430 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 >> 172.21.15.154:0/2411459435 conn(0x7f59840c3880 msgr2=0x7f598406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 shutdown_connections 2024-03-14T09:02:41.081 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.081+0000 7f598ba0a700 1 -- 172.21.15.154:0/2411459435 wait complete. 2024-03-14T09:02:41.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T09:02:41.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.216+0000 7fad7914a700 1 Processor -- start 2024-03-14T09:02:41.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.216+0000 7fad7914a700 1 -- start start 2024-03-14T09:02:41.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740ab730 0x7fad740aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad74155260 0x7fad7406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad740bf9c0 0x7fad740bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7406b6c0 con 0x7fad740ab730 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7406a770 con 0x7fad74155260 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7914a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7406df70 con 0x7fad740bf9c0 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740ab730 0x7fad740aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad740ab730 0x7fad740aa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37510/0 (socket says 172.21.15.154:37510) 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 -- 172.21.15.154:0/4247037159 learned_addr learned my addr 172.21.15.154:0/4247037159 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7359e700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad740bf9c0 0x7fad740bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad7259c700 1 --2- 172.21.15.154:0/4247037159 >> v2:172.21.15.191:3300/0 conn(0x7fad74155260 0x7fad7406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 -- 172.21.15.154:0/4247037159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad740bf9c0 msgr2=0x7fad740bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad740bf9c0 0x7fad740bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 -- 172.21.15.154:0/4247037159 >> v2:172.21.15.191:3300/0 conn(0x7fad74155260 msgr2=0x7fad7406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 --2- 172.21.15.154:0/4247037159 >> v2:172.21.15.191:3300/0 conn(0x7fad74155260 0x7fad7406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 -- 172.21.15.154:0/4247037159 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7406dc40 con 0x7fad740ab730 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad72d9d700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad740ab730 0x7fad740aa6e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fad74070a80 tx=0x7fad64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48ec8da22115998 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad71d9b700 1 -- 172.21.15.154:0/4247037159 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400b870 con 0x7fad740ab730 2024-03-14T09:02:41.219 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad71d9b700 1 -- 172.21.15.154:0/4247037159 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6400b9d0 con 0x7fad740ab730 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.217+0000 7fad71d9b700 1 -- 172.21.15.154:0/4247037159 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400bd60 con 0x7fad740ab730 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 -- 172.21.15.154:0/4247037159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad740ab730 msgr2=0x7fad740aa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad740ab730 0x7fad740aa6e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fad74070a80 tx=0x7fad64009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 -- 172.21.15.154:0/4247037159 shutdown_connections 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad740bf9c0 0x7fad740bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 --2- 172.21.15.154:0/4247037159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad740ab730 0x7fad740aa6e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 --2- 172.21.15.154:0/4247037159 >> v2:172.21.15.191:3300/0 conn(0x7fad74155260 0x7fad7406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 -- 172.21.15.154:0/4247037159 >> 172.21.15.154:0/4247037159 conn(0x7fad740c3090 msgr2=0x7fad740c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 -- 172.21.15.154:0/4247037159 shutdown_connections 2024-03-14T09:02:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.218+0000 7fad7914a700 1 -- 172.21.15.154:0/4247037159 wait complete. 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.219+0000 7fad7914a700 1 Processor -- start 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.219+0000 7fad7914a700 1 -- start start 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad740bf9c0 0x7fad741eab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad74155260 0x7fad740c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741ee080 con 0x7fad74154810 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741ea070 con 0x7fad740bf9c0 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7914a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad741ea350 con 0x7fad74155260 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37524/0 (socket says 172.21.15.154:37524) 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.220+0000 7fad7359e700 1 -- 172.21.15.154:0/3363804928 learned_addr learned my addr 172.21.15.154:0/3363804928 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7259c700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad74155260 0x7fad740c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad72d9d700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad740bf9c0 0x7fad741eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 -- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad74155260 msgr2=0x7fad740c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad74155260 0x7fad740c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 -- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad740bf9c0 msgr2=0x7fad741eab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad740bf9c0 0x7fad741eab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 -- 172.21.15.154:0/3363804928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64009550 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7359e700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007e60 tx=0x7fad6c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00ede0 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c014420 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0147d0 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad741ea630 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.221+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad741ee320 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.222+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74074120 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.222+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fad6c014930 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.223+0000 7fad5b7fe700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 0x7fad6006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.223+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fad6c0897d0 con 0x7fad74154810 2024-03-14T09:02:41.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.223+0000 7fad72d9d700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 0x7fad6006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.224+0000 7fad72d9d700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 0x7fad6006cd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad64014040 tx=0x7fad64008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.227+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fad6c054890 con 0x7fad74154810 2024-03-14T09:02:41.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.468+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad740697d0 con 0x7fad74154810 2024-03-14T09:02:41.468 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:02:41.468+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:02:41.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.481+0000 7fad5b7fe700 1 -- 172.21.15.154:0/3363804928 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad6c057eb0 con 0x7fad74154810 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 msgr2=0x7fad6006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 0x7fad6006cd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad64014040 tx=0x7fad64008030 comp rx=0 tx=0).stop 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 msgr2=0x7fad741eb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007e60 tx=0x7fad6c00e3c0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 shutdown_connections 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad6006a860 0x7fad6006cd20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad74154810 0x7fad741eb3d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad740bf9c0 0x7fad741eab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 --2- 172.21.15.154:0/3363804928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad74155260 0x7fad740c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 >> 172.21.15.154:0/3363804928 conn(0x7fad740c3090 msgr2=0x7fad740c1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:41.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 shutdown_connections 2024-03-14T09:02:41.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.484+0000 7fad7914a700 1 -- 172.21.15.154:0/3363804928 wait complete. 2024-03-14T09:02:41.485 INFO:tasks.ceph.mds.c.smithi154.stderr: -4014> 2024-03-14T09:02:07.099+0000 7f1b1a907700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:02:41.485 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:02:41.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T09:02:41.515 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:02:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.621+0000 7fc8ae79c700 1 Processor -- start 2024-03-14T09:02:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.621+0000 7fc8ae79c700 1 -- start start 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8ae79c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a80c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8ae79c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc8a80c6680 0x7fc8a815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8ae79c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a80c7bb0 con 0x7fc8a806c670 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8ae79c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80c3e00 con 0x7fc8a80c6680 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8ae79c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c7e90 con 0x7fc8a80c31c0 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8a7fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc8a806c670 0x7fc8a806cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55516/0 (socket says 172.21.15.154:55516) 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8a7fff700 1 -- 172.21.15.154:0/1861905938 learned_addr learned my addr 172.21.15.154:0/1861905938 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8acd39700 1 --2- 172.21.15.154:0/1861905938 >> v2:172.21.15.191:3300/0 conn(0x7fc8a80c6680 0x7fc8a815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.622+0000 7fc8a77fe700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a80c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8a6ffd700 1 -- 172.21.15.154:0/1861905938 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278850800 0 0) 0x7fc8a80c7bb0 con 0x7fc8a806c670 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8a6ffd700 1 -- 172.21.15.154:0/1861905938 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8940035b0 con 0x7fc8a806c670 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8acd39700 1 -- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 msgr2=0x7fc8a80c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8acd39700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a80c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8acd39700 1 -- 172.21.15.154:0/1861905938 >> v1:172.21.15.154:6789/0 conn(0x7fc8a806c670 legacy=0x7fc8a806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8acd39700 1 -- 172.21.15.154:0/1861905938 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80c8170 con 0x7fc8a80c6680 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8acd39700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a815ed60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009d90 tx=0x7fc8a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e271bb51a7be9478 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8a6ffd700 1 -- 172.21.15.154:0/1861905938 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000e070 con 0x7fc8a80c6680 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8a6ffd700 1 -- 172.21.15.154:0/1861905938 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a000bb20 con 0x7fc8a80c6680 2024-03-14T09:02:41.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.623+0000 7fc8a6ffd700 1 -- 172.21.15.154:0/1861905938 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0012430 con 0x7fc8a80c6680 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 -- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 msgr2=0x7fc8a815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a815ed60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009d90 tx=0x7fc8a000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 -- 172.21.15.154:0/1861905938 shutdown_connections 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a815ed60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/1861905938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a80c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 -- 172.21.15.154:0/1861905938 >> 172.21.15.154:0/1861905938 conn(0x7fc8a80a9bf0 msgr2=0x7fc8a80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 -- 172.21.15.154:0/1861905938 shutdown_connections 2024-03-14T09:02:41.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.624+0000 7fc8ae79c700 1 -- 172.21.15.154:0/1861905938 wait complete. 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.625+0000 7fc8ae79c700 1 Processor -- start 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.625+0000 7fc8ae79c700 1 -- start start 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a81e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a81ea120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a806f890 con 0x7fc8a806c670 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81ed150 con 0x7fc8a80c6680 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8ae79c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81ed430 con 0x7fc8a80c31c0 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37548/0 (socket says 172.21.15.154:37548) 2024-03-14T09:02:41.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 -- 172.21.15.154:0/3483343408 learned_addr learned my addr 172.21.15.154:0/3483343408 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a77fe700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a81e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8acd39700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a81ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 -- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 msgr2=0x7fc8a81e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a81e9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 -- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 msgr2=0x7fc8a81ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a81ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 -- 172.21.15.154:0/3483343408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0009a70 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a7fff700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc898001eb0 tx=0x7fc89800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.626+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898005d70 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.627+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8980070d0 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.627+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a81ed710 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.627+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898007460 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.627+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81edcd0 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.628+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a80697d0 con 0x7fc8a806c670 2024-03-14T09:02:41.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.628+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc8980075c0 con 0x7fc8a806c670 2024-03-14T09:02:41.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.628+0000 7fc8a4ff9700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 0x7fc88806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:41.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.629+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fc898088450 con 0x7fc8a806c670 2024-03-14T09:02:41.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.629+0000 7fc8a77fe700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 0x7fc88806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:41.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.629+0000 7fc8a77fe700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 0x7fc88806ccf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8a806c590 tx=0x7fc890009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:41.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.633+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc898053550 con 0x7fc8a806c670 2024-03-14T09:02:41.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:41.873+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc8a81ede90 con 0x7fc8a806c670 2024-03-14T09:02:42.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.486+0000 7fc8a4ff9700 1 -- 172.21.15.154:0/3483343408 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc898056b70 con 0x7fc8a806c670 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 msgr2=0x7fc88806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 0x7fc88806ccf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8a806c590 tx=0x7fc890009210 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 msgr2=0x7fc8a81df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc898001eb0 tx=0x7fc89800c7b0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 shutdown_connections 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8a80c6680 0x7fc8a81ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc88806a830 0x7fc88806ccf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8a806c670 0x7fc8a81df760 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 --2- 172.21.15.154:0/3483343408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8a80c31c0 0x7fc8a81e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 >> 172.21.15.154:0/3483343408 conn(0x7fc8a80a9bf0 msgr2=0x7fc8a80c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 shutdown_connections 2024-03-14T09:02:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.489+0000 7fc8ae79c700 1 -- 172.21.15.154:0/3483343408 wait complete. 2024-03-14T09:02:42.502 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T09:02:42.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 Processor -- start 2024-03-14T09:02:42.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 -- start start 2024-03-14T09:02:42.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc04c1549c0 0x7fc04c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc04c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c070a20 con 0x7fc04c1589c0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c06bde0 con 0x7fc04c1549c0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc051975700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c158da0 con 0x7fc04c155380 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc04affd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc04c1589c0 0x7fc04c0c1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55518/0 (socket says 172.21.15.154:55518) 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.670+0000 7fc04affd700 1 -- 172.21.15.154:0/2948775014 learned_addr learned my addr 172.21.15.154:0/2948775014 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc04c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> v2:172.21.15.191:3300/0 conn(0x7fc04c1549c0 0x7fc04c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052479075 0 0) 0x7fc04c070a20 con 0x7fc04c1589c0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0340035b0 con 0x7fc04c1589c0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110423848 0 0) 0x7fc0340035b0 con 0x7fc04c1589c0 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 msgr2=0x7fc04c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:42.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc04c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 >> v2:172.21.15.191:3300/0 conn(0x7fc04c1549c0 msgr2=0x7fc04c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 --2- 172.21.15.154:0/2948775014 >> v2:172.21.15.191:3300/0 conn(0x7fc04c1549c0 0x7fc04c154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c070d00 con 0x7fc04c1589c0 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fc040001670 con 0x7fc04c1589c0 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc04c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> v2:172.21.15.191:3300/0 conn(0x7fc04c1549c0 0x7fc04c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 >> v1:172.21.15.154:6789/0 conn(0x7fc04c1589c0 legacy=0x7fc04c0c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034003a00 0x7fc034003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034004850 0x7fc034004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc034005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c158da0 con 0x7fc034004850 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03400ba10 con 0x7fc034003a00 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03400bb50 con 0x7fc04c155380 2024-03-14T09:02:42.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034004850 0x7fc034004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04affd700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034003a00 0x7fc034003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.671+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc034005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 -- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 msgr2=0x7fc034005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c155380 0x7fc034005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 -- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034003a00 msgr2=0x7fc034003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034003a00 0x7fc034003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 -- 172.21.15.154:0/2948775014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044009a70 con 0x7fc034004850 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034004850 0x7fc034004cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009d30 tx=0x7fc034005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12deb478eaba6afc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c009e80 con 0x7fc034004850 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c00e9a0 con 0x7fc034004850 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc049ffb700 1 -- 172.21.15.154:0/2948775014 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00ed30 con 0x7fc034004850 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 -- 172.21.15.154:0/2948775014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034004850 msgr2=0x7fc034004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 --2- 172.21.15.154:0/2948775014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc034004850 0x7fc034004cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009d30 tx=0x7fc034005880 comp rx=0 tx=0).stop 2024-03-14T09:02:42.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc04b7fe700 1 -- 172.21.15.154:0/2948775014 reap_dead start 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 -- 172.21.15.154:0/2948775014 shutdown_connections 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 -- 172.21.15.154:0/2948775014 >> 172.21.15.154:0/2948775014 conn(0x7fc04c0a9d00 msgr2=0x7fc04c0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 -- 172.21.15.154:0/2948775014 shutdown_connections 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.672+0000 7fc051975700 1 -- 172.21.15.154:0/2948775014 wait complete. 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 Processor -- start 2024-03-14T09:02:42.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 -- start start 2024-03-14T09:02:42.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 0x7fc04c0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 0x7fc04c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c1487a0 con 0x7fc04c0703e0 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c06ea20 con 0x7fc034004850 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.674+0000 7fc051975700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c148250 con 0x7fc04c156180 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37574/0 (socket says 172.21.15.154:37574) 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 -- 172.21.15.154:0/525910929 learned_addr learned my addr 172.21.15.154:0/525910929 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04affd700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 0x7fc04c0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 0x7fc04c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 -- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 msgr2=0x7fc04c06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 0x7fc04c06fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 -- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 msgr2=0x7fc04c0ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 0x7fc04c0ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 -- 172.21.15.154:0/525910929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c1484c0 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04affd700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 0x7fc04c0ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04a7fc700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 0x7fc04c06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc04b7fe700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc04c06c330 tx=0x7fc04400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04401c070 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400f040 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c06e360 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400e040 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.675+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c14a2f0 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.676+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c06e1c0 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.677+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc044015430 con 0x7fc04c0703e0 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.677+0000 7fc033fff700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 0x7fc03806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.677+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6651+0+0 (secure 0 0 0) 0x7fc044088f00 con 0x7fc04c0703e0 2024-03-14T09:02:42.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.677+0000 7fc04affd700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 0x7fc03806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:42.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.678+0000 7fc04affd700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 0x7fc03806ccf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc04c1523b0 tx=0x7fc040009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:42.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.682+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc04c06e1c0 con 0x7fc04c0703e0 2024-03-14T09:02:42.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:42.876+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc04c06e1c0 con 0x7fc04c0703e0 2024-03-14T09:02:43.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.490+0000 7fc033fff700 1 -- 172.21.15.154:0/525910929 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc04c06e1c0 con 0x7fc04c0703e0 2024-03-14T09:02:43.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 msgr2=0x7fc03806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 0x7fc03806ccf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc04c1523b0 tx=0x7fc040009040 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 msgr2=0x7fc04c1ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc04c06c330 tx=0x7fc04400f490 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 shutdown_connections 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc03806a830 0x7fc03806ccf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc04c0703e0 0x7fc04c1ea950 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc034004850 0x7fc04c0ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 --2- 172.21.15.154:0/525910929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc04c156180 0x7fc04c06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 >> 172.21.15.154:0/525910929 conn(0x7fc04c0a9d00 msgr2=0x7fc04c0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:43.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 shutdown_connections 2024-03-14T09:02:43.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.494+0000 7fc051975700 1 -- 172.21.15.154:0/525910929 wait complete. 2024-03-14T09:02:43.494 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2024-03-14T09:02:43.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T09:02:43.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.637+0000 7fcc5fc54700 1 Processor -- start 2024-03-14T09:02:43.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.637+0000 7fcc5fc54700 1 -- start start 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5fc54700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc581551a0 0x7fcc5815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5fc54700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5806e9f0 con 0x7fcc5814b490 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5fc54700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5806ecd0 con 0x7fcc58153130 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5fc54700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc580a8860 con 0x7fcc581551a0 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc581551a0 0x7fcc5815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5d9f0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcc5814b490 0x7fcc58152a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55522/0 (socket says 172.21.15.154:55522) 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc581551a0 0x7fcc5815ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37586/0 (socket says 172.21.15.154:37586) 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5d9f0700 1 -- 172.21.15.154:0/1034349569 learned_addr learned my addr 172.21.15.154:0/1034349569 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:43.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/1034349569 >> v1:172.21.15.191:6789/0 conn(0x7fcc58153130 legacy=0x7fcc58154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/1034349569 >> v1:172.21.15.154:6789/0 conn(0x7fcc5814b490 legacy=0x7fcc58152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/1034349569 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5806efb0 con 0x7fcc581551a0 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5e1f1700 1 --2- 172.21.15.154:0/1034349569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc5815ce30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009d90 tx=0x7fcc5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5f90993aa0d53a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.638+0000 7fcc5c9ee700 1 -- 172.21.15.154:0/1034349569 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000e070 con 0x7fcc581551a0 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5c9ee700 1 -- 172.21.15.154:0/1034349569 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000bbd0 con 0x7fcc581551a0 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5c9ee700 1 -- 172.21.15.154:0/1034349569 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50012430 con 0x7fcc581551a0 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 -- 172.21.15.154:0/1034349569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 msgr2=0x7fcc5815ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/1034349569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc5815ce30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009d90 tx=0x7fcc5000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 -- 172.21.15.154:0/1034349569 shutdown_connections 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/1034349569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc5815ce30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 -- 172.21.15.154:0/1034349569 >> 172.21.15.154:0/1034349569 conn(0x7fcc580c3980 msgr2=0x7fcc580c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 -- 172.21.15.154:0/1034349569 shutdown_connections 2024-03-14T09:02:43.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.639+0000 7fcc5fc54700 1 -- 172.21.15.154:0/1034349569 wait complete. 2024-03-14T09:02:43.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.640+0000 7fcc5fc54700 1 Processor -- start 2024-03-14T09:02:43.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.640+0000 7fcc5fc54700 1 -- start start 2024-03-14T09:02:43.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc5814b490 0x7fcc580c1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:43.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 0x7fcc581e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5806f670 con 0x7fcc581551a0 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581e8010 con 0x7fcc5814b490 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5fc54700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581ec510 con 0x7fcc58153130 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37596/0 (socket says 172.21.15.154:37596) 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/26755328 learned_addr learned my addr 172.21.15.154:0/26755328 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5d9f0700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc5814b490 0x7fcc580c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5d1ef700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 0x7fcc581e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 msgr2=0x7fcc581e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 0x7fcc581e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc5814b490 msgr2=0x7fcc580c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc5814b490 0x7fcc580c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5e1f1700 1 -- 172.21.15.154:0/26755328 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009a70 con 0x7fcc581551a0 2024-03-14T09:02:43.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.641+0000 7fcc5d1ef700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 0x7fcc581e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc5e1f1700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc50014040 tx=0x7fcc50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50012430 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc50020070 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc581ebff0 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50004280 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.642+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc581ec230 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.643+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc580ad200 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.643+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcc500043e0 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.643+0000 7fcc4e7fc700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 0x7fcc3c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.644+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6213+0+0 (secure 0 0 0) 0x7fcc50022070 con 0x7fcc581551a0 2024-03-14T09:02:43.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.644+0000 7fcc5d9f0700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 0x7fcc3c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:43.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.644+0000 7fcc5d9f0700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 0x7fcc3c06ccf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005910 tx=0x7fcc54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:43.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.648+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcc50056060 con 0x7fcc581551a0 2024-03-14T09:02:43.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:43.847+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcc581ebff0 con 0x7fcc581551a0 2024-03-14T09:02:44.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.495+0000 7fcc4e7fc700 1 -- 172.21.15.154:0/26755328 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcc581ebff0 con 0x7fcc581551a0 2024-03-14T09:02:44.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 msgr2=0x7fcc3c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 0x7fcc3c06ccf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005910 tx=0x7fcc54009040 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 msgr2=0x7fcc58071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc50014040 tx=0x7fcc50004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 shutdown_connections 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcc3c06a830 0x7fcc3c06ccf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc581551a0 0x7fcc58071280 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc5814b490 0x7fcc580c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.498+0000 7fcc5fc54700 1 --2- 172.21.15.154:0/26755328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc58153130 0x7fcc581e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.499+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 >> 172.21.15.154:0/26755328 conn(0x7fcc580c3980 msgr2=0x7fcc58148c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.499+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 shutdown_connections 2024-03-14T09:02:44.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.499+0000 7fcc5fc54700 1 -- 172.21.15.154:0/26755328 wait complete. 2024-03-14T09:02:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2024-03-14T09:02:44.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:44.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.637+0000 7fdbd9a52700 1 Processor -- start 2024-03-14T09:02:44.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.637+0000 7fdbd9a52700 1 -- start start 2024-03-14T09:02:44.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd9a52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbd40c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd9a52700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdbd40c4f30 0x7fdbd415f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd9a52700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd4158240 con 0x7fdbd40c7f70 2024-03-14T09:02:44.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd9a52700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd414ed10 con 0x7fdbd40c4f30 2024-03-14T09:02:44.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd9a52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd40cad00 con 0x7fdbd41586d0 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd2ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdbd40c7f70 0x7fdbd4155230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55528/0 (socket says 172.21.15.154:55528) 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd2ffd700 1 -- 172.21.15.154:0/4158586039 learned_addr learned my addr 172.21.15.154:0/4158586039 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/4158586039 >> v2:172.21.15.191:3300/0 conn(0x7fdbd40c4f30 0x7fdbd415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236903676 0 0) 0x7fdbd4158240 con 0x7fdbd40c7f70 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd27fc700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbd40c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.638+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbbc0035b0 con 0x7fdbd40c7f70 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3160769981 0 0) 0x7fdbbc0035b0 con 0x7fdbd40c7f70 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 msgr2=0x7fdbd40c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbd40c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 >> v2:172.21.15.191:3300/0 conn(0x7fdbd40c4f30 msgr2=0x7fdbd415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 --2- 172.21.15.154:0/4158586039 >> v2:172.21.15.191:3300/0 conn(0x7fdbd40c4f30 0x7fdbd415f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40ca740 con 0x7fdbd40c7f70 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fdbc8001670 con 0x7fdbd40c7f70 2024-03-14T09:02:44.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 >> v1:172.21.15.154:6789/0 conn(0x7fdbd40c7f70 legacy=0x7fdbd4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbbc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbbc003f80 0x7fdbbc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbbc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd414ed10 con 0x7fdbbc004d90 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd40cad00 con 0x7fdbbc003f80 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc00baf0 con 0x7fdbd41586d0 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbbc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd2ffd700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbbc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd27fc700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbbc003f80 0x7fdbbc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 -- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 msgr2=0x7fdbbc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd41586d0 0x7fdbbc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 -- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbbc003f80 msgr2=0x7fdbbc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbbc003f80 0x7fdbbc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 -- 172.21.15.154:0/4158586039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0035b0 con 0x7fdbbc004d90 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.639+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbbc005210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fc0 tx=0x7fdbbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6fd79c87ac1872 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00ec30 con 0x7fdbbc004d90 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc00f040 con 0x7fdbbc004d90 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd1ffb700 1 -- 172.21.15.154:0/4158586039 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc014480 con 0x7fdbbc004d90 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 -- 172.21.15.154:0/4158586039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 msgr2=0x7fdbbc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/4158586039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbbc005210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fc0 tx=0x7fdbbc005880 comp rx=0 tx=0).stop 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd37fe700 1 -- 172.21.15.154:0/4158586039 reap_dead start 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 -- 172.21.15.154:0/4158586039 shutdown_connections 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 -- 172.21.15.154:0/4158586039 >> 172.21.15.154:0/4158586039 conn(0x7fdbd4072080 msgr2=0x7fdbd4072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 -- 172.21.15.154:0/4158586039 shutdown_connections 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.640+0000 7fdbd9a52700 1 -- 172.21.15.154:0/4158586039 wait complete. 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.642+0000 7fdbd9a52700 1 Processor -- start 2024-03-14T09:02:44.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.642+0000 7fdbd9a52700 1 -- start start 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbd4148eb0 0x7fdbd40c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd40a9d60 0x7fdbd41ed500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd40aafa0 con 0x7fdbbc004d90 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41541d0 con 0x7fdbd4148eb0 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd414ddf0 con 0x7fdbd40a9d60 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37604/0 (socket says 172.21.15.154:37604) 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 -- 172.21.15.154:0/1160247151 learned_addr learned my addr 172.21.15.154:0/1160247151 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd27fc700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbd4148eb0 0x7fdbd40c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd37fe700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd40a9d60 0x7fdbd41ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 -- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd40a9d60 msgr2=0x7fdbd41ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd40a9d60 0x7fdbd41ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 -- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbd4148eb0 msgr2=0x7fdbd40c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbd4148eb0 0x7fdbd40c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 -- 172.21.15.154:0/1160247151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40ac590 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd2ffd700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001aa0 tx=0x7fdbc80076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800f8a0 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800fa60 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd40c1b20 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800fe10 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.643+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd41543d0 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.645+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fdbc801d460 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.645+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd40ca3c0 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.645+0000 7fdbbbfff700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 0x7fdbc006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.645+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fdbc8088e90 con 0x7fdbbc004d90 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.646+0000 7fdbd27fc700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 0x7fdbc006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.646+0000 7fdbd27fc700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 0x7fdbc006cd20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4149290 tx=0x7fdbc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.650+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdbc8054240 con 0x7fdbbc004d90 2024-03-14T09:02:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.848+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd40697d0 con 0x7fdbbc004d90 2024-03-14T09:02:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.849+0000 7fdbbbfff700 1 -- 172.21.15.154:0/1160247151 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16228 (secure 0 0 0) 0x7fdbc808a090 con 0x7fdbbc004d90 2024-03-14T09:02:44.849 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:44.849 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":39,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:44.488016+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/3981292790":"2024-03-15T09:02:41.469336+0000","172.21.15.154:6833/3981292790":"2024-03-15T09:02:41.469336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 msgr2=0x7fdbc006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 0x7fdbc006cd20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4149290 tx=0x7fdbc4008040 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 msgr2=0x7fdbd40af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8001aa0 tx=0x7fdbc80076a0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 shutdown_connections 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbd40a9d60 0x7fdbd41ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fdbc006a860 0x7fdbc006cd20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbbc004d90 0x7fdbd40af0d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 --2- 172.21.15.154:0/1160247151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbd4148eb0 0x7fdbd40c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.851+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 >> 172.21.15.154:0/1160247151 conn(0x7fdbd4072080 msgr2=0x7fdbd41564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.852+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 shutdown_connections 2024-03-14T09:02:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.852+0000 7fdbd9a52700 1 -- 172.21.15.154:0/1160247151 wait complete. 2024-03-14T09:02:44.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:02:19.001797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:02:19.712147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:44.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:44.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.988+0000 7f1e80e68700 1 Processor -- start 2024-03-14T09:02:44.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.988+0000 7f1e80e68700 1 -- start start 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e80e68700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1e7c070a10 0x7f1e7c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e80e68700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0c5f10 con 0x7f1e7c0c31c0 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e80e68700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0c7d50 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e80e68700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c8030 con 0x7f1e7c070a10 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e79d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1e7c070a10 0x7f1e7c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7ad9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1e7c0c31c0 0x7f1e7c0c68e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55540/0 (socket says 172.21.15.154:55540) 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7ad9d700 1 -- 172.21.15.154:0/2236872863 learned_addr learned my addr 172.21.15.154:0/2236872863 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e79d9b700 1 -- 172.21.15.154:0/2236872863 >> v1:172.21.15.191:6789/0 conn(0x7f1e7c06c6e0 legacy=0x7f1e7c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e79d9b700 1 -- 172.21.15.154:0/2236872863 >> v1:172.21.15.154:6789/0 conn(0x7f1e7c0c31c0 legacy=0x7f1e7c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e79d9b700 1 -- 172.21.15.154:0/2236872863 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0c8310 con 0x7f1e7c070a10 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7959a700 1 -- 172.21.15.154:0/2236872863 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700448727 0 0) 0x7f1e7c0c5f10 con 0x7f1e7c0c31c0 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e79d9b700 1 --2- 172.21.15.154:0/2236872863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c070a10 0x7f1e7c0c63a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009840 tx=0x7f1e70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beac9ff373c80a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7959a700 1 -- 172.21.15.154:0/2236872863 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70011830 con 0x7f1e7c070a10 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7959a700 1 -- 172.21.15.154:0/2236872863 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e70011990 con 0x7f1e7c070a10 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.989+0000 7f1e7959a700 1 -- 172.21.15.154:0/2236872863 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70011d20 con 0x7f1e7c070a10 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 -- 172.21.15.154:0/2236872863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c070a10 msgr2=0x7f1e7c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 --2- 172.21.15.154:0/2236872863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c070a10 0x7f1e7c0c63a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009840 tx=0x7f1e70009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 -- 172.21.15.154:0/2236872863 shutdown_connections 2024-03-14T09:02:44.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 --2- 172.21.15.154:0/2236872863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c070a10 0x7f1e7c0c63a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 -- 172.21.15.154:0/2236872863 >> 172.21.15.154:0/2236872863 conn(0x7f1e7c0a9bf0 msgr2=0x7f1e7c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:44.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 -- 172.21.15.154:0/2236872863 shutdown_connections 2024-03-14T09:02:44.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.990+0000 7f1e80e68700 1 -- 172.21.15.154:0/2236872863 wait complete. 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.991+0000 7f1e80e68700 1 Processor -- start 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.991+0000 7f1e80e68700 1 -- start start 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 0x7f1e7c1dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e7c0c31c0 0x7f1e7c1ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c1ee420 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c1ed830 con 0x7f1e7c070a10 2024-03-14T09:02:44.994 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e80e68700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0acfa0 con 0x7f1e7c0c31c0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37624/0 (socket says 172.21.15.154:37624) 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 -- 172.21.15.154:0/48021146 learned_addr learned my addr 172.21.15.154:0/48021146 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e79d9b700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 0x7f1e7c1dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 -- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e7c0c31c0 msgr2=0x7f1e7c1ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e7c0c31c0 0x7f1e7c1ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 -- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 msgr2=0x7f1e7c1dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 0x7f1e7c1dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e7a59c700 1 -- 172.21.15.154:0/48021146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700094a0 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.992+0000 7f1e79d9b700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 0x7f1e7c1dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e7a59c700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4e0 tx=0x7f1e6c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0052d0 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005430 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c1eda80 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0038c0 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.993+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c1ee040 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.994+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c06d560 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.994+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1e6c003a20 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.995+0000 7f1e6affd700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 0x7f1e5c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.995+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f1e6c088200 con 0x7f1e7c06c6e0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.995+0000 7f1e79d9b700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 0x7f1e5c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.995+0000 7f1e79d9b700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 0x7f1e5c06ccf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c070ea0 tx=0x7f1e7001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:44.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:44.999+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1e6c053650 con 0x7f1e7c06c6e0 2024-03-14T09:02:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.190+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e7c074110 con 0x7f1e7c06c6e0 2024-03-14T09:02:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.191+0000 7f1e6affd700 1 -- 172.21.15.154:0/48021146 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e6c056c70 con 0x7f1e7c06c6e0 2024-03-14T09:02:45.191 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 msgr2=0x7f1e5c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 0x7f1e5c06ccf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c070ea0 tx=0x7f1e7001c040 comp rx=0 tx=0).stop 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 msgr2=0x7f1e7c0aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4e0 tx=0x7f1e6c00c8a0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 shutdown_connections 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e7c0c31c0 0x7f1e7c1ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e5c06a830 0x7f1e5c06ccf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c06c6e0 0x7f1e7c0aca60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 --2- 172.21.15.154:0/48021146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7c070a10 0x7f1e7c1dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 >> 172.21.15.154:0/48021146 conn(0x7f1e7c0a9bf0 msgr2=0x7f1e7c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:45.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 shutdown_connections 2024-03-14T09:02:45.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.193+0000 7f1e80e68700 1 -- 172.21.15.154:0/48021146 wait complete. 2024-03-14T09:02:45.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:02:45.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.334+0000 7f0913b5e700 1 Processor -- start 2024-03-14T09:02:45.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.334+0000 7f0913b5e700 1 -- start start 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f0913b5e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f090c0a8540 0x7f090c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f0913b5e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c13ade0 con 0x7f090c0c1780 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f0913b5e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c06fb30 con 0x7f090c0c3250 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f0913b5e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c06ec30 con 0x7f090c0a8540 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09120fb700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f090c0c1780 0x7f090c0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55546/0 (socket says 172.21.15.154:55546) 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09120fb700 1 -- 172.21.15.154:0/3178200450 learned_addr learned my addr 172.21.15.154:0/3178200450 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376664970 0 0) 0x7f0904001490 con 0x7f090c0c1780 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f80035b0 con 0x7f090c0c1780 2024-03-14T09:02:45.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09118fa700 1 --2- 172.21.15.154:0/3178200450 >> v2:172.21.15.191:3300/0 conn(0x7f090c0a8540 0x7f090c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 896377978 0 0) 0x7f08f80035b0 con 0x7f090c0c1780 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 >> v2:172.21.15.191:3300/0 conn(0x7f090c0a8540 msgr2=0x7f090c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 --2- 172.21.15.154:0/3178200450 >> v2:172.21.15.191:3300/0 conn(0x7f090c0a8540 0x7f090c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 >> v1:172.21.15.191:6789/0 conn(0x7f090c0c3250 legacy=0x7f090c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c06ef10 con 0x7f090c0c1780 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.335+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0904001b80 con 0x7f090c0c1780 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 >> v1:172.21.15.154:6789/0 conn(0x7f090c0c1780 legacy=0x7f090c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8003a00 0x7f08f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f8004850 0x7f08f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 0x7f08f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c06ec30 con 0x7f08f8005330 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c200 con 0x7f08f8004850 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c3c0 con 0x7f08f8003a00 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 0x7f08f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09110f9700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f8004850 0x7f08f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09118fa700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8003a00 0x7f08f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 -- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8003a00 msgr2=0x7f08f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8003a00 0x7f08f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 -- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f8004850 msgr2=0x7f08f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f8004850 0x7f08f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 -- 172.21.15.154:0/3178200450 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f80035b0 con 0x7f08f8005330 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09118fa700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8003a00 0x7f08f8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 -- 172.21.15.154:0/3178200450 reap_dead start 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09120fb700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 0x7f08f800ba10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f090400a7d0 tx=0x7f08f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f729fa6bed70a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400a460 con 0x7f08f8005330 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904012600 con 0x7f08f8005330 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.336+0000 7f09108f8700 1 -- 172.21.15.154:0/3178200450 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040129b0 con 0x7f08f8005330 2024-03-14T09:02:45.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 -- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 msgr2=0x7f08f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 0x7f08f800ba10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f090400a7d0 tx=0x7f08f8005880 comp rx=0 tx=0).stop 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 -- 172.21.15.154:0/3178200450 shutdown_connections 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 --2- 172.21.15.154:0/3178200450 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08f8005330 0x7f08f800ba10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 -- 172.21.15.154:0/3178200450 >> 172.21.15.154:0/3178200450 conn(0x7f090c0c3880 msgr2=0x7f090c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 -- 172.21.15.154:0/3178200450 shutdown_connections 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.337+0000 7f0913b5e700 1 -- 172.21.15.154:0/3178200450 wait complete. 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.338+0000 7f0913b5e700 1 Processor -- start 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.338+0000 7f0913b5e700 1 -- start start 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8005330 0x7f090c1eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c134790 0x7f090c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1dd100 con 0x7f090c133760 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c134f50 con 0x7f090c134790 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c1ee300 con 0x7f08f8005330 2024-03-14T09:02:45.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37644/0 (socket says 172.21.15.154:37644) 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 -- 172.21.15.154:0/2099824581 learned_addr learned my addr 172.21.15.154:0/2099824581 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09110f9700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c134790 0x7f090c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09118fa700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8005330 0x7f090c1eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 -- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8005330 msgr2=0x7f090c1eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8005330 0x7f090c1eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 -- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c134790 msgr2=0x7f090c0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c134790 0x7f090c0c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 -- 172.21.15.154:0/2099824581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c134c70 con 0x7f090c133760 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09120fb700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0904000f80 tx=0x7f090c06ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400a460 con 0x7f090c133760 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904012d30 con 0x7f090c133760 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c135b40 con 0x7f090c133760 2024-03-14T09:02:45.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.339+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904012ea0 con 0x7f090c133760 2024-03-14T09:02:45.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.340+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c136100 con 0x7f090c133760 2024-03-14T09:02:45.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.340+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c1359a0 con 0x7f090c133760 2024-03-14T09:02:45.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.341+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f090401e630 con 0x7f090c133760 2024-03-14T09:02:45.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.341+0000 7f09027fc700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 0x7f08f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.341+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f090408a9e0 con 0x7f090c133760 2024-03-14T09:02:45.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.342+0000 7f09118fa700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 0x7f08f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.342+0000 7f09118fa700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 0x7f08f006ccf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009b40 tx=0x7f08fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.346+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f090c1359a0 con 0x7f090c133760 2024-03-14T09:02:45.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.542+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f090c13ade0 con 0x7f090c133760 2024-03-14T09:02:45.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.542+0000 7f09027fc700 1 -- 172.21.15.154:0/2099824581 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v39) v1 ==== 105+0+11 (secure 0 0 0) 0x7f090c13ade0 con 0x7f090c133760 2024-03-14T09:02:45.542 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 msgr2=0x7f08f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 0x7f08f006ccf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009b40 tx=0x7f08fc008040 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 msgr2=0x7f090c133c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0904000f80 tx=0x7f090c06ee80 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 shutdown_connections 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f08f006a830 0x7f08f006ccf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f090c133760 0x7f090c133c10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f8005330 0x7f090c1eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 --2- 172.21.15.154:0/2099824581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c134790 0x7f090c0c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 >> 172.21.15.154:0/2099824581 conn(0x7f090c0c3880 msgr2=0x7f090c139990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 shutdown_connections 2024-03-14T09:02:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.545+0000 7f0913b5e700 1 -- 172.21.15.154:0/2099824581 wait complete. 2024-03-14T09:02:45.557 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:02:45.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.682+0000 7f8be2969700 1 Processor -- start 2024-03-14T09:02:45.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.682+0000 7f8be2969700 1 -- start start 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.682+0000 7f8be2969700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8bdc0702f0 0x7f8bdc0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8be2969700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0c5d60 0x7f8bdc0c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8be2969700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc0acd10 con 0x7f8bdc14fbd0 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8be2969700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0c66b0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8be2969700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0ac7b0 con 0x7f8bdc0c5d60 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8bdc0702f0 0x7f8bdc0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8bdc0702f0 0x7f8bdc0c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37650/0 (socket says 172.21.15.154:37650) 2024-03-14T09:02:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdb7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8bdc14fbd0 0x7f8bdc0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55562/0 (socket says 172.21.15.154:55562) 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 -- 172.21.15.154:0/2709464064 learned_addr learned my addr 172.21.15.154:0/2709464064 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8be0f06700 1 --2- 172.21.15.154:0/2709464064 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0c5d60 0x7f8bdc0c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 -- 172.21.15.154:0/2709464064 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0c5d60 msgr2=0x7f8bdc0c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 --2- 172.21.15.154:0/2709464064 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0c5d60 0x7f8bdc0c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 -- 172.21.15.154:0/2709464064 >> v1:172.21.15.154:6789/0 conn(0x7f8bdc14fbd0 legacy=0x7f8bdc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 -- 172.21.15.154:0/2709464064 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc150050 con 0x7f8bdc0702f0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdaffd700 1 -- 172.21.15.154:0/2709464064 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287405348 0 0) 0x7f8bdc0acd10 con 0x7f8bdc14fbd0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdbfff700 1 --2- 172.21.15.154:0/2709464064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc0c4380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0aac90 tx=0x7f8bcc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a9ff72cade78cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdaffd700 1 -- 172.21.15.154:0/2709464064 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00b870 con 0x7f8bdc0702f0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdaffd700 1 -- 172.21.15.154:0/2709464064 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc00b9d0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.683+0000 7f8bdaffd700 1 -- 172.21.15.154:0/2709464064 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00bd60 con 0x7f8bdc0702f0 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 -- 172.21.15.154:0/2709464064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 msgr2=0x7f8bdc0c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 --2- 172.21.15.154:0/2709464064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc0c4380 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0aac90 tx=0x7f8bcc009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 -- 172.21.15.154:0/2709464064 shutdown_connections 2024-03-14T09:02:45.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 --2- 172.21.15.154:0/2709464064 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0c5d60 0x7f8bdc0c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 --2- 172.21.15.154:0/2709464064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc0c4380 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 -- 172.21.15.154:0/2709464064 >> 172.21.15.154:0/2709464064 conn(0x7f8bdc0a9bf0 msgr2=0x7f8bdc0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:45.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 -- 172.21.15.154:0/2709464064 shutdown_connections 2024-03-14T09:02:45.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.684+0000 7f8be2969700 1 -- 172.21.15.154:0/2709464064 wait complete. 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.685+0000 7f8be2969700 1 Processor -- start 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.685+0000 7f8be2969700 1 -- start start 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 0x7f8bdc142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 0x7f8bdc0c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc143380 con 0x7f8bdc0702f0 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc06fdd0 con 0x7f8bdc14fbd0 2024-03-14T09:02:45.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be2969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc1526f0 con 0x7f8bdc0c5d60 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37656/0 (socket says 172.21.15.154:37656) 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 -- 172.21.15.154:0/1022437624 learned_addr learned my addr 172.21.15.154:0/1022437624 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdb7fe700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 0x7f8bdc142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be0f06700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 0x7f8bdc0c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 -- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 msgr2=0x7f8bdc142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 0x7f8bdc142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 -- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 msgr2=0x7f8bdc0c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 0x7f8bdc0c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 -- 172.21.15.154:0/1022437624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009550 con 0x7f8bdc0702f0 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdb7fe700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 0x7f8bdc142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8be0f06700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 0x7f8bdc0c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.686+0000 7f8bdbfff700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009c10 tx=0x7f8bcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.687+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc004310 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.687+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc01f070 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.687+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0036a0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.687+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc1529d0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.687+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc06e2f0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.688+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc0697d0 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.688+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8bcc021030 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.688+0000 7f8bd8ff9700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 0x7f8bc806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.688+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f8bcc08a000 con 0x7f8bdc0702f0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.689+0000 7f8bdb7fe700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 0x7f8bc806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.689+0000 7f8bdb7fe700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 0x7f8bc806ccf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007fd0 tx=0x7f8bd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.693+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8bcc055450 con 0x7f8bdc0702f0 2024-03-14T09:02:45.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.887+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f8bdc0c7160 con 0x7f8bdc0702f0 2024-03-14T09:02:45.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.887+0000 7f8bd8ff9700 1 -- 172.21.15.154:0/1022437624 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v39) v1 ==== 101+0+11 (secure 0 0 0) 0x7f8bcc015020 con 0x7f8bdc0702f0 2024-03-14T09:02:45.887 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:02:45.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 msgr2=0x7f8bc806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 0x7f8bc806ccf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007fd0 tx=0x7f8bd0009040 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 msgr2=0x7f8bdc151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009c10 tx=0x7f8bcc004750 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 shutdown_connections 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc14fbd0 0x7f8bdc0c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8bc806a830 0x7f8bc806ccf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8bdc0702f0 0x7f8bdc151ff0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 --2- 172.21.15.154:0/1022437624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0c5d60 0x7f8bdc142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.889+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 >> 172.21.15.154:0/1022437624 conn(0x7f8bdc0a9bf0 msgr2=0x7f8bdc0c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.890+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 shutdown_connections 2024-03-14T09:02:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:45.890+0000 7f8be2969700 1 -- 172.21.15.154:0/1022437624 wait complete. 2024-03-14T09:02:45.902 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:46.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.023+0000 7f727eb8d700 1 Processor -- start 2024-03-14T09:02:46.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.023+0000 7f727eb8d700 1 -- start start 2024-03-14T09:02:46.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727eb8d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727eb8d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f72780c1780 0x7f72780c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727eb8d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727813ade0 con 0x7f72780a8540 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727eb8d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727806fb30 con 0x7f72780c1780 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727eb8d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727806ec30 con 0x7f72780c3250 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727c929700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f72780a8540 0x7f7278154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55572/0 (socket says 172.21.15.154:55572) 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727c929700 1 -- 172.21.15.154:0/3260206496 learned_addr learned my addr 172.21.15.154:0/3260206496 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727d12a700 1 --2- 172.21.15.154:0/3260206496 >> v2:172.21.15.191:3300/0 conn(0x7f72780c1780 0x7f72780c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f7277fff700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288100638 0 0) 0x7f726c001110 con 0x7f72780a8540 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72680035b0 con 0x7f72780a8540 2024-03-14T09:02:46.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342005830 0 0) 0x7f72680035b0 con 0x7f72780a8540 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727d12a700 1 -- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 msgr2=0x7f727813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727d12a700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727d12a700 1 -- 172.21.15.154:0/3260206496 >> v1:172.21.15.154:6789/0 conn(0x7f72780a8540 legacy=0x7f7278154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f727d12a700 1 -- 172.21.15.154:0/3260206496 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727806ef10 con 0x7f72780c1780 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.024+0000 7f7277fff700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727d12a700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f72780c1c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7270009d90 tx=0x7f727000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5895d551c77a012b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727000e070 con 0x7f72780c1780 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270003a20 con 0x7f72780c1780 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f72777fe700 1 -- 172.21.15.154:0/3260206496 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c001800 con 0x7f72780c1780 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 -- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 msgr2=0x7f72780c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f72780c1c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7270009d90 tx=0x7f727000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 -- 172.21.15.154:0/3260206496 shutdown_connections 2024-03-14T09:02:46.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f72780c1c10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3260206496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 -- 172.21.15.154:0/3260206496 >> 172.21.15.154:0/3260206496 conn(0x7f72780c3880 msgr2=0x7f72780c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:46.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 -- 172.21.15.154:0/3260206496 shutdown_connections 2024-03-14T09:02:46.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.025+0000 7f727eb8d700 1 -- 172.21.15.154:0/3260206496 wait complete. 2024-03-14T09:02:46.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.026+0000 7f727eb8d700 1 Processor -- start 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 -- start start 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f7278154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72781517b0 con 0x7f72780a8540 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72781513d0 con 0x7f72780c1780 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727eb8d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7278150650 con 0x7f72780c3250 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.027+0000 7f727c929700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37670/0 (socket says 172.21.15.154:37670) 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 -- 172.21.15.154:0/3362109732 learned_addr learned my addr 172.21.15.154:0/3362109732 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727d12a700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f7277fff700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f7278154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 -- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 msgr2=0x7f727814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 -- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 msgr2=0x7f7278154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f7278154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 -- 172.21.15.154:0/3362109732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640094a0 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727c929700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c750 tx=0x7f726c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c004c80 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c004de0 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7270009a70 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0072f0 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.028+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72781508f0 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.029+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72780697d0 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.030+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f726c007450 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.030+0000 7f72757fa700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 0x7f725c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.030+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f726c088980 con 0x7f72780a8540 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.030+0000 7f7277fff700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 0x7f725c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.031+0000 7f7277fff700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 0x7f725c06ccf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7278135db0 tx=0x7f7264008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.034+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f726c053dd0 con 0x7f72780a8540 2024-03-14T09:02:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.273+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f727813ade0 con 0x7f72780a8540 2024-03-14T09:02:46.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.273+0000 7f72757fa700 1 -- 172.21.15.154:0/3362109732 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f727813ade0 con 0x7f72780a8540 2024-03-14T09:02:46.273 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:46.273 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2558238943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2558238943},{"type":"v1","addr":"172.21.15.154:6835","nonce":2558238943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 msgr2=0x7f725c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 0x7f725c06ccf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7278135db0 tx=0x7f7264008040 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 msgr2=0x7f7278153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c750 tx=0x7f726c00cb10 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 shutdown_connections 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72780c3250 0x7f727814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f725c06a830 0x7f725c06ccf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72780a8540 0x7f7278153f00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 --2- 172.21.15.154:0/3362109732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72780c1780 0x7f7278154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.275+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 >> 172.21.15.154:0/3362109732 conn(0x7f72780c3880 msgr2=0x7f727806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:46.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.276+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 shutdown_connections 2024-03-14T09:02:46.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.276+0000 7f727eb8d700 1 -- 172.21.15.154:0/3362109732 wait complete. 2024-03-14T09:02:46.277 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 22 2024-03-14T09:02:46.288 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:46.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.408+0000 7f3bdc523700 1 Processor -- start 2024-03-14T09:02:46.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.408+0000 7f3bdc523700 1 -- start start 2024-03-14T09:02:46.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdc523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdc523700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3bd40c1780 0x7f3bd40c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdc523700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd413ade0 con 0x7f3bd40c3250 2024-03-14T09:02:46.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdc523700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd406fb30 con 0x7f3bd40c1780 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdc523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd406ec30 con 0x7f3bd40a8540 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdaac0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3bd40c1780 0x7f3bd40c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdaac0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3bd40c1780 0x7f3bd40c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37686/0 (socket says 172.21.15.154:37686) 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.409+0000 7f3bdaac0700 1 -- 172.21.15.154:0/1739340969 learned_addr learned my addr 172.21.15.154:0/1739340969 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bdaac0700 1 -- 172.21.15.154:0/1739340969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 msgr2=0x7f3bd413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bda2bf700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bdaac0700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bdaac0700 1 -- 172.21.15.154:0/1739340969 >> v1:172.21.15.191:6789/0 conn(0x7f3bd40c3250 legacy=0x7f3bd4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bdaac0700 1 -- 172.21.15.154:0/1739340969 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd406ef10 con 0x7f3bd40c1780 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bdaac0700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c1780 0x7f3bd40c1c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540ba1ba3f208c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bd92bd700 1 -- 172.21.15.154:0/1739340969 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00e070 con 0x7f3bd40c1780 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bd92bd700 1 -- 172.21.15.154:0/1739340969 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc003a20 con 0x7f3bd40c1780 2024-03-14T09:02:46.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.410+0000 7f3bd92bd700 1 -- 172.21.15.154:0/1739340969 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc003dd0 con 0x7f3bd40c1780 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 -- 172.21.15.154:0/1739340969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c1780 msgr2=0x7f3bd40c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c1780 0x7f3bd40c1c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009e50 tx=0x7f3bcc00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 -- 172.21.15.154:0/1739340969 shutdown_connections 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 --2- 172.21.15.154:0/1739340969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c1780 0x7f3bd40c1c10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 -- 172.21.15.154:0/1739340969 >> 172.21.15.154:0/1739340969 conn(0x7f3bd40c3880 msgr2=0x7f3bd40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 -- 172.21.15.154:0/1739340969 shutdown_connections 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.411+0000 7f3bdc523700 1 -- 172.21.15.154:0/1739340969 wait complete. 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 Processor -- start 2024-03-14T09:02:46.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 -- start start 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd41e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bd40c1780 0x7f3bd412e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41ec120 con 0x7f3bd40c3250 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41e8300 con 0x7f3bd40c1780 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.412+0000 7f3bdc523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd41ec310 con 0x7f3bd40a8540 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37700/0 (socket says 172.21.15.154:37700) 2024-03-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 -- 172.21.15.154:0/4093964965 learned_addr learned my addr 172.21.15.154:0/4093964965 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bd9abe700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bd40c1780 0x7f3bd412e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bda2bf700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd41e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 -- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 msgr2=0x7f3bd41e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd41e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 -- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bd40c1780 msgr2=0x7f3bd412e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bd40c1780 0x7f3bd412e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 -- 172.21.15.154:0/4093964965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc009b30 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdaac0700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc015040 tx=0x7f3bcc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0044c0 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc004640 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc012430 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd41e85a0 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.413+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd41ec5f0 con 0x7f3bd40c3250 2024-03-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.415+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd40697d0 con 0x7f3bd40c3250 2024-03-14T09:02:46.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.415+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3bcc012590 con 0x7f3bd40c3250 2024-03-14T09:02:46.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.415+0000 7f3bc6ffd700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 0x7f3bb806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.415+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f3bcc08a120 con 0x7f3bd40c3250 2024-03-14T09:02:46.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.415+0000 7f3bda2bf700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 0x7f3bb806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.416+0000 7f3bda2bf700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 0x7f3bb806ccf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800a950 tx=0x7f3bc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.419 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.420+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3bcc055570 con 0x7f3bd40c3250 2024-03-14T09:02:46.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.618+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bd413ade0 con 0x7f3bd40c3250 2024-03-14T09:02:46.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.619+0000 7f3bc6ffd700 1 -- 172.21.15.154:0/4093964965 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f3bd413ade0 con 0x7f3bd40c3250 2024-03-14T09:02:46.619 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:46.619 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":39,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:44.488016+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/3981292790":"2024-03-15T09:02:41.469336+0000","172.21.15.154:6833/3981292790":"2024-03-15T09:02:41.469336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 msgr2=0x7f3bb806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 0x7f3bb806ccf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800a950 tx=0x7f3bc8008040 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 msgr2=0x7f3bd4154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc015040 tx=0x7f3bcc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 shutdown_connections 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3bb806a830 0x7f3bb806ccf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bd40c3250 0x7f3bd4154ba0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bd40a8540 0x7f3bd41e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 --2- 172.21.15.154:0/4093964965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bd40c1780 0x7f3bd412e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.621+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 >> 172.21.15.154:0/4093964965 conn(0x7f3bd40c3880 msgr2=0x7f3bd40c6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.622+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 shutdown_connections 2024-03-14T09:02:46.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.622+0000 7f3bdc523700 1 -- 172.21.15.154:0/4093964965 wait complete. 2024-03-14T09:02:46.633 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-14T09:02:46.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:46.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.755+0000 7f27f1c21700 1 Processor -- start 2024-03-14T09:02:46.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.755+0000 7f27f1c21700 1 -- start start 2024-03-14T09:02:46.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27f1c21700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27ec155df0 0x7f27ec156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27f1c21700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec15d270 0x7f27ec15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27f1c21700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec0ac8e0 con 0x7f27ec14eaf0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27f1c21700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec06b310 con 0x7f27ec155df0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27f1c21700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec06de30 con 0x7f27ec15d270 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27ec155df0 0x7f27ec156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eb7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f27ec14eaf0 0x7f27ec06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55578/0 (socket says 172.21.15.154:55578) 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eb7fe700 1 -- 172.21.15.154:0/3760112959 learned_addr learned my addr 172.21.15.154:0/3760112959 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27ebfff700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec15d270 0x7f27ec15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 -- 172.21.15.154:0/3760112959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec15d270 msgr2=0x7f27ec15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec15d270 0x7f27ec15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 -- 172.21.15.154:0/3760112959 >> v1:172.21.15.154:6789/0 conn(0x7f27ec14eaf0 legacy=0x7f27ec06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 -- 172.21.15.154:0/3760112959 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0acb30 con 0x7f27ec155df0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27ea7fc700 1 -- 172.21.15.154:0/3760112959 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839941820 0 0) 0x7f27ec0ac8e0 con 0x7f27ec14eaf0 2024-03-14T09:02:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.756+0000 7f27eaffd700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec155df0 0x7f27ec156250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009840 tx=0x7f27d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125a22b1edcf71ee server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27ea7fc700 1 -- 172.21.15.154:0/3760112959 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4011830 con 0x7f27ec155df0 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27ea7fc700 1 -- 172.21.15.154:0/3760112959 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4011990 con 0x7f27ec155df0 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27ea7fc700 1 -- 172.21.15.154:0/3760112959 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4011d20 con 0x7f27ec155df0 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 -- 172.21.15.154:0/3760112959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec155df0 msgr2=0x7f27ec156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec155df0 0x7f27ec156250 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009840 tx=0x7f27d4009b50 comp rx=0 tx=0).stop 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 -- 172.21.15.154:0/3760112959 shutdown_connections 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec15d270 0x7f27ec15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3760112959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec155df0 0x7f27ec156250 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 -- 172.21.15.154:0/3760112959 >> 172.21.15.154:0/3760112959 conn(0x7f27ec0c3880 msgr2=0x7f27ec0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 -- 172.21.15.154:0/3760112959 shutdown_connections 2024-03-14T09:02:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.757+0000 7f27f1c21700 1 -- 172.21.15.154:0/3760112959 wait complete. 2024-03-14T09:02:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.759+0000 7f27f1c21700 1 Processor -- start 2024-03-14T09:02:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.759+0000 7f27f1c21700 1 -- start start 2024-03-14T09:02:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ec14eaf0 0x7f27ec14a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 0x7f27ec13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec153d40 con 0x7f27ec15d270 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec14a4d0 con 0x7f27ec14eaf0 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec156ff0 con 0x7f27ec155df0 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37722/0 (socket says 172.21.15.154:37722) 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 -- 172.21.15.154:0/3405216531 learned_addr learned my addr 172.21.15.154:0/3405216531 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27eaffd700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 0x7f27ec13fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27eb7fe700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ec14eaf0 0x7f27ec14a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 -- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 msgr2=0x7f27ec13fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 0x7f27ec13fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 -- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ec14eaf0 msgr2=0x7f27ec14a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ec14eaf0 0x7f27ec14a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 -- 172.21.15.154:0/3405216531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40094a0 con 0x7f27ec15d270 2024-03-14T09:02:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27eaffd700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 0x7f27ec13fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27ebfff700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f27e4007e60 tx=0x7f27e400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e400ed80 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4014420 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec153780 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40147d0 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.760+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec14a160 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.761+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec149fc0 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.762+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f27e4014930 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.762+0000 7f27e3fff700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 0x7f27cc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.762+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f27e4089500 con 0x7f27ec15d270 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.762+0000 7f27eb7fe700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 0x7f27cc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.763+0000 7f27eb7fe700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 0x7f27cc06ccf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27dc000f60 tx=0x7f27dc007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:46.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:46.767+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f27ec149fc0 con 0x7f27ec15d270 2024-03-14T09:02:47.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.006+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27ec149fc0 con 0x7f27ec15d270 2024-03-14T09:02:47.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.006+0000 7f27e3fff700 1 -- 172.21.15.154:0/3405216531 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f27ec149fc0 con 0x7f27ec15d270 2024-03-14T09:02:47.006 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:47.006 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2558238943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2558238943},{"type":"v1","addr":"172.21.15.154:6835","nonce":2558238943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:47.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 msgr2=0x7f27cc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 0x7f27cc06ccf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27dc000f60 tx=0x7f27dc007be0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 msgr2=0x7f27ec152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f27e4007e60 tx=0x7f27e400e3c0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 shutdown_connections 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27cc06a830 0x7f27cc06ccf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27ec15d270 0x7f27ec152770 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27ec14eaf0 0x7f27ec14a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 --2- 172.21.15.154:0/3405216531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27ec155df0 0x7f27ec13fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 >> 172.21.15.154:0/3405216531 conn(0x7f27ec0c3880 msgr2=0x7f27ec06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.008+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 shutdown_connections 2024-03-14T09:02:47.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.009+0000 7f27f1c21700 1 -- 172.21.15.154:0/3405216531 wait complete. 2024-03-14T09:02:47.010 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 22 2024-03-14T09:02:47.021 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:47.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.142+0000 7f7cf2b81700 1 Processor -- start 2024-03-14T09:02:47.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.142+0000 7f7cf2b81700 1 -- start start 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf2b81700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7cec0a8540 0x7f7cec13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf2b81700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec13ade0 con 0x7f7cec0c1780 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf2b81700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cec06fb30 con 0x7f7cec0c3250 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf2b81700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cec06ec30 con 0x7f7cec0a8540 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf111e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7cec0c1780 0x7f7cec0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55580/0 (socket says 172.21.15.154:55580) 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf111e700 1 -- 172.21.15.154:0/1563313980 learned_addr learned my addr 172.21.15.154:0/1563313980 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:47.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7cf091d700 1 --2- 172.21.15.154:0/1563313980 >> v2:172.21.15.191:3300/0 conn(0x7f7cec0a8540 0x7f7cec13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292328237 0 0) 0x7f7ce4001490 con 0x7f7cec0c1780 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.143+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cdc0035b0 con 0x7f7cec0c1780 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575592161 0 0) 0x7f7cdc0035b0 con 0x7f7cec0c1780 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 >> v2:172.21.15.191:3300/0 conn(0x7f7cec0a8540 msgr2=0x7f7cec13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 --2- 172.21.15.154:0/1563313980 >> v2:172.21.15.191:3300/0 conn(0x7f7cec0a8540 0x7f7cec13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 >> v1:172.21.15.191:6789/0 conn(0x7f7cec0c3250 legacy=0x7f7cec154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec06ef10 con 0x7f7cec0c1780 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f7ce4001b80 con 0x7f7cec0c1780 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 >> v1:172.21.15.154:6789/0 conn(0x7f7cec0c1780 legacy=0x7f7cec0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cdc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cdc004850 0x7f7cdc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cdc005330 0x7f7cdc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec06ec30 con 0x7f7cdc003a00 2024-03-14T09:02:47.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc00c200 con 0x7f7cdc005330 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc00c3c0 con 0x7f7cdc004850 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cdc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf111e700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cdc005330 0x7f7cdc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cebfff700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cdc004850 0x7f7cdc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 -- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cdc004850 msgr2=0x7f7cdc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cdc004850 0x7f7cdc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 -- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cdc005330 msgr2=0x7f7cdc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cdc005330 0x7f7cdc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 -- 172.21.15.154:0/1563313980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc0035b0 con 0x7f7cdc003a00 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf111e700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cdc005330 0x7f7cdc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf111e700 1 -- 172.21.15.154:0/1563313980 reap_dead start 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7cf091d700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cdc003e20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009b30 tx=0x7f7cdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c962860b2115bf3d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000e6c0 con 0x7f7cdc003a00 2024-03-14T09:02:47.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce000e820 con 0x7f7cdc003a00 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.144+0000 7f7ceb7fe700 1 -- 172.21.15.154:0/1563313980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000eb90 con 0x7f7cdc003a00 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 -- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 msgr2=0x7f7cdc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cdc003e20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009b30 tx=0x7f7cdc005880 comp rx=0 tx=0).stop 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 -- 172.21.15.154:0/1563313980 shutdown_connections 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/1563313980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cdc003e20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 -- 172.21.15.154:0/1563313980 >> 172.21.15.154:0/1563313980 conn(0x7f7cec0c3880 msgr2=0x7f7cec0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 -- 172.21.15.154:0/1563313980 shutdown_connections 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.145+0000 7f7cf2b81700 1 -- 172.21.15.154:0/1563313980 wait complete. 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.146+0000 7f7cf2b81700 1 Processor -- start 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.146+0000 7f7cf2b81700 1 -- start start 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cec152230 0x7f7cec0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cec0ac860 0x7f7cec0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec151b60 con 0x7f7cdc003a00 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec151630 con 0x7f7cec0ac860 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf2b81700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec1e8200 con 0x7f7cec152230 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37740/0 (socket says 172.21.15.154:37740) 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 -- 172.21.15.154:0/3853217915 learned_addr learned my addr 172.21.15.154:0/3853217915 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cebfff700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cec152230 0x7f7cec0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf111e700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cec0ac860 0x7f7cec0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 -- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cec152230 msgr2=0x7f7cec0ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cec152230 0x7f7cec0ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 -- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cec0ac860 msgr2=0x7f7cec0ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cec0ac860 0x7f7cec0ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.147+0000 7f7cf091d700 1 -- 172.21.15.154:0/3853217915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec14e700 con 0x7f7cdc003a00 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7cf091d700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000efd0 tx=0x7f7ce00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000e6c0 con 0x7f7cdc003a00 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce00038b0 con 0x7f7cdc003a00 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cec151880 con 0x7f7cdc003a00 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0003c60 con 0x7f7cdc003a00 2024-03-14T09:02:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.148+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec0a9d20 con 0x7f7cdc003a00 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.149+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec1e7cf0 con 0x7f7cdc003a00 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.149+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7ce0013470 con 0x7f7cdc003a00 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.149+0000 7f7ce97fa700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 0x7f7cd006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.150+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f7ce008a410 con 0x7f7cdc003a00 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.150+0000 7f7cebfff700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 0x7f7cd006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.150+0000 7f7cebfff700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 0x7f7cd006ccf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7cec1e50b0 tx=0x7f7cd8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.154+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7cec1e7cf0 con 0x7f7cdc003a00 2024-03-14T09:02:47.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.391+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cec1e7cf0 con 0x7f7cdc003a00 2024-03-14T09:02:47.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.391+0000 7f7ce97fa700 1 -- 172.21.15.154:0/3853217915 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4745 (secure 0 0 0) 0x7f7cec1e7cf0 con 0x7f7cdc003a00 2024-03-14T09:02:47.391 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:47.391 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2558238943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2558238943},{"type":"v1","addr":"172.21.15.154:6835","nonce":2558238943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:20.753295+0000","modified":"2024-03-14T09:02:23.759505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":8,"addr":"172.21.15.154:6834/1134234285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1134234285},{"type":"v1","addr":"172.21.15.154:6834","nonce":1134234285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 msgr2=0x7f7cd006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 0x7f7cd006ccf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7cec1e50b0 tx=0x7f7cd8009040 comp rx=0 tx=0).stop 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 msgr2=0x7f7cec151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000efd0 tx=0x7f7ce00047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 shutdown_connections 2024-03-14T09:02:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cec0ac860 0x7f7cec0ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cd006a830 0x7f7cd006ccf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cdc003a00 0x7f7cec151cf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 --2- 172.21.15.154:0/3853217915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cec152230 0x7f7cec0ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 >> 172.21.15.154:0/3853217915 conn(0x7f7cec0c3880 msgr2=0x7f7cec135f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 shutdown_connections 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.394+0000 7f7cf2b81700 1 -- 172.21.15.154:0/3853217915 wait complete. 2024-03-14T09:02:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 22 2024-03-14T09:02:47.406 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:47.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.528+0000 7f7b285a1700 1 Processor -- start 2024-03-14T09:02:47.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.528+0000 7f7b285a1700 1 -- start start 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.529+0000 7f7b285a1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.529+0000 7f7b285a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 0x7f7b200c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.529+0000 7f7b285a1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2013ade0 con 0x7f7b200a8540 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.529+0000 7f7b285a1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2006fb30 con 0x7f7b200c3250 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.529+0000 7f7b285a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2006ec30 con 0x7f7b200c1780 2024-03-14T09:02:47.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b25b3c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 0x7f7b200c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b2633d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f7b200a8540 0x7f7b20154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:33118/0 (socket says 172.21.15.154:33118) 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b25b3c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47218/0 (socket says 172.21.15.154:47218) 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b2633d700 1 -- 172.21.15.154:0/2458915679 learned_addr learned my addr 172.21.15.154:0/2458915679 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 -- 172.21.15.154:0/2458915679 >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 msgr2=0x7f7b2013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 --2- 172.21.15.154:0/2458915679 >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 -- 172.21.15.154:0/2458915679 >> v1:172.21.15.191:6789/0 conn(0x7f7b200a8540 legacy=0x7f7b20154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 -- 172.21.15.154:0/2458915679 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2006ef10 con 0x7f7b200c1780 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b2533b700 1 -- 172.21.15.154:0/2458915679 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438355282 0 0) 0x7f7b14001110 con 0x7f7b200a8540 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b25b3c700 1 --2- 172.21.15.154:0/2458915679 >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b26b3e700 1 --2- 172.21.15.154:0/2458915679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 0x7f7b200c1c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df6c474afdd6bae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.530+0000 7f7b2533b700 1 -- 172.21.15.154:0/2458915679 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800e070 con 0x7f7b200c1780 2024-03-14T09:02:47.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b2533b700 1 -- 172.21.15.154:0/2458915679 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800bb20 con 0x7f7b200c1780 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b2533b700 1 -- 172.21.15.154:0/2458915679 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18012430 con 0x7f7b200c1780 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 -- 172.21.15.154:0/2458915679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 msgr2=0x7f7b200c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2458915679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 0x7f7b200c1c10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 -- 172.21.15.154:0/2458915679 shutdown_connections 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2458915679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200c1780 0x7f7b200c1c10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2458915679 >> v2:172.21.15.191:3300/0 conn(0x7f7b200c3250 0x7f7b2013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 -- 172.21.15.154:0/2458915679 >> 172.21.15.154:0/2458915679 conn(0x7f7b200c3880 msgr2=0x7f7b200c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 -- 172.21.15.154:0/2458915679 shutdown_connections 2024-03-14T09:02:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.531+0000 7f7b285a1700 1 -- 172.21.15.154:0/2458915679 wait complete. 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.532+0000 7f7b285a1700 1 Processor -- start 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.532+0000 7f7b285a1700 1 -- start start 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200a8540 0x7f7b20134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b200c3250 0x7f7b2014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200ac920 con 0x7f7b200c1780 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2014de50 con 0x7f7b200c3250 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b285a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2014e000 con 0x7f7b200a8540 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b25b3c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b25b3c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37746/0 (socket says 172.21.15.154:37746) 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b25b3c700 1 -- 172.21.15.154:0/2797611538 learned_addr learned my addr 172.21.15.154:0/2797611538 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b26b3e700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b200c3250 0x7f7b2014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.533+0000 7f7b2633d700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200a8540 0x7f7b20134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 -- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200a8540 msgr2=0x7f7b20134050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200a8540 0x7f7b20134050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 -- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b200c3250 msgr2=0x7f7b2014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b200c3250 0x7f7b2014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 -- 172.21.15.154:0/2797611538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0094a0 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b25b3c700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00ec10 tx=0x7f7b1c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c009760 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c00c850 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00cbe0 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b18009a70 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.534+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b201ec490 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.535+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b200697d0 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.535+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7b1c00cd40 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.536+0000 7f7b12ffd700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 0x7f7b0406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.536+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f7b1c089110 con 0x7f7b200c1780 2024-03-14T09:02:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.536+0000 7f7b2633d700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 0x7f7b0406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.537+0000 7f7b2633d700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 0x7f7b0406ccf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b14000f60 tx=0x7f7b14007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.541+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7b1c0544d0 con 0x7f7b200c1780 2024-03-14T09:02:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.737+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b200acd50 con 0x7f7b200c1780 2024-03-14T09:02:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.738+0000 7f7b12ffd700 1 -- 172.21.15.154:0/2797611538 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16228 (secure 0 0 0) 0x7f7b1c057af0 con 0x7f7b200c1780 2024-03-14T09:02:47.738 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:47.738 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":39,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:44.488016+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:02:19.001797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:02:19.712147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/3981292790":"2024-03-15T09:02:41.469336+0000","172.21.15.154:6833/3981292790":"2024-03-15T09:02:41.469336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 msgr2=0x7f7b0406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 0x7f7b0406ccf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b14000f60 tx=0x7f7b14007e70 comp rx=0 tx=0).stop 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 msgr2=0x7f7b2014d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00ec10 tx=0x7f7b1c00ef20 comp rx=0 tx=0).stop 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 shutdown_connections 2024-03-14T09:02:47.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b200c3250 0x7f7b2014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b200a8540 0x7f7b20134050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b0406a830 0x7f7b0406ccf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 --2- 172.21.15.154:0/2797611538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b200c1780 0x7f7b2014d3d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 >> 172.21.15.154:0/2797611538 conn(0x7f7b200c3880 msgr2=0x7f7b201380a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 shutdown_connections 2024-03-14T09:02:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.740+0000 7f7b285a1700 1 -- 172.21.15.154:0/2797611538 wait complete. 2024-03-14T09:02:47.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-14T09:02:47.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.874+0000 7f71d79d5700 1 Processor -- start 2024-03-14T09:02:47.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 -- start start 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71d00c5d60 0x7f71d00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d00acd10 con 0x7f71d014fbd0 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d00ac7b0 con 0x7f71d00c5d60 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d79d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d0150050 con 0x7f71d00702f0 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.875+0000 7f71d5f72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71d00c5d60 0x7f71d00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71d00c5d60 0x7f71d00c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37752/0 (socket says 172.21.15.154:37752) 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 -- 172.21.15.154:0/1508464607 learned_addr learned my addr 172.21.15.154:0/1508464607 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:47.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5771700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 -- 172.21.15.154:0/1508464607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 msgr2=0x7f71d00c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d00c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 -- 172.21.15.154:0/1508464607 >> v1:172.21.15.191:6789/0 conn(0x7f71d014fbd0 legacy=0x7f71d00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 -- 172.21.15.154:0/1508464607 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d00ac9c0 con 0x7f71d00c5d60 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71d5f72700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d00c61d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009e50 tx=0x7f71c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18903c86e8d74c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71c7fff700 1 -- 172.21.15.154:0/1508464607 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800e070 con 0x7f71d00c5d60 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71c7fff700 1 -- 172.21.15.154:0/1508464607 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c800b9c0 con 0x7f71d00c5d60 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.876+0000 7f71c7fff700 1 -- 172.21.15.154:0/1508464607 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c800bd70 con 0x7f71d00c5d60 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 -- 172.21.15.154:0/1508464607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 msgr2=0x7f71d00c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d00c61d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009e50 tx=0x7f71c800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 -- 172.21.15.154:0/1508464607 shutdown_connections 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d00c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 --2- 172.21.15.154:0/1508464607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d00c61d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 -- 172.21.15.154:0/1508464607 >> 172.21.15.154:0/1508464607 conn(0x7f71d00a9bf0 msgr2=0x7f71d00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:47.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 -- 172.21.15.154:0/1508464607 shutdown_connections 2024-03-14T09:02:47.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.877+0000 7f71d79d5700 1 -- 172.21.15.154:0/1508464607 wait complete. 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.878+0000 7f71d79d5700 1 Processor -- start 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.878+0000 7f71d79d5700 1 -- start start 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d01514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71d014fbd0 0x7f71d0149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d01e7d10 con 0x7f71d00c5d60 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d0156410 con 0x7f71d014fbd0 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d01566b0 con 0x7f71d00702f0 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37766/0 (socket says 172.21.15.154:37766) 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 -- 172.21.15.154:0/4180468230 learned_addr learned my addr 172.21.15.154:0/4180468230 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:47.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d5771700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d01514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d5f72700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71d014fbd0 0x7f71d0149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 -- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 msgr2=0x7f71d01514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d01514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 -- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71d014fbd0 msgr2=0x7f71d0149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71d014fbd0 0x7f71d0149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 -- 172.21.15.154:0/4180468230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8009b30 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d4f70700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c450 tx=0x7f71cc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0052d0 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc005430 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d0156990 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.879+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc0038c0 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.880+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d01e7930 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.881+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d01e7e80 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.881+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f71cc003a20 con 0x7f71d00c5d60 2024-03-14T09:02:47.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.881+0000 7f71c5ffb700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 0x7f71b406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:47.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.881+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f71cc088090 con 0x7f71d00c5d60 2024-03-14T09:02:47.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.882+0000 7f71d5771700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 0x7f71b406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:47.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.882+0000 7f71d5771700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 0x7f71b406ccf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006b20 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:47.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:47.886+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f71cc053440 con 0x7f71d00c5d60 2024-03-14T09:02:48.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.124+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f71d00697d0 con 0x7f71d00c5d60 2024-03-14T09:02:48.124 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:02:48.125+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:02:48.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.134+0000 7f71c5ffb700 1 -- 172.21.15.154:0/4180468230 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 163+0+0 (secure 0 0 0) 0x7f71cc00ce60 con 0x7f71d00c5d60 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 msgr2=0x7f71b406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 0x7f71b406ccf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f71c0006b20 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 msgr2=0x7f71d0148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c450 tx=0x7f71cc00c760 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 shutdown_connections 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71d014fbd0 0x7f71d0149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71d00702f0 0x7f71d01514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71b406a830 0x7f71b406ccf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 --2- 172.21.15.154:0/4180468230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71d00c5d60 0x7f71d0148dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 >> 172.21.15.154:0/4180468230 conn(0x7f71d00a9bf0 msgr2=0x7f71d014e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.137+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 shutdown_connections 2024-03-14T09:02:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.138+0000 7f71d79d5700 1 -- 172.21.15.154:0/4180468230 wait complete. 2024-03-14T09:02:48.139 INFO:tasks.ceph.mds.a.smithi154.stderr: -3302> 2024-03-14T09:02:22.099+0000 7fd7d8ae1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:02:48.139 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:02:48.153 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-14T09:02:48.171 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:02:48.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.279+0000 7fa954071700 1 Processor -- start 2024-03-14T09:02:48.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.279+0000 7fa954071700 1 -- start start 2024-03-14T09:02:48.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa954071700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 0x7fa94c0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:48.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa954071700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c0c7bb0 con 0x7fa94c0c6680 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa954071700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c0c7e90 con 0x7fa94c06c670 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa954071700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0c3e00 con 0x7fa94c0c31c0 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95260e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa94c0c6680 0x7fa94c15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55596/0 (socket says 172.21.15.154:55596) 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95260e700 1 -- 172.21.15.154:0/4268166400 learned_addr learned my addr 172.21.15.154:0/4268166400 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95160c700 1 --2- 172.21.15.154:0/4268166400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 0x7fa94c0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420408467 0 0) 0x7fa94c0c7bb0 con 0x7fa94c0c6680 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9380035b0 con 0x7fa94c0c6680 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944957656 0 0) 0x7fa94c0c7e90 con 0x7fa94c06c670 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c0c7bb0 con 0x7fa94c06c670 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809778095 0 0) 0x7fa9380035b0 con 0x7fa94c0c6680 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95160c700 1 -- 172.21.15.154:0/4268166400 >> v1:172.21.15.191:6789/0 conn(0x7fa94c06c670 legacy=0x7fa94c06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95160c700 1 -- 172.21.15.154:0/4268166400 >> v1:172.21.15.154:6789/0 conn(0x7fa94c0c6680 legacy=0x7fa94c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.280+0000 7fa95160c700 1 -- 172.21.15.154:0/4268166400 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0c8170 con 0x7fa94c0c31c0 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa95160c700 1 --2- 172.21.15.154:0/4268166400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 0x7fa94c0c6140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa948009870 tx=0x7fa948009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78cf928aa3ff4159 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440019f0 con 0x7fa94c0c31c0 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa948004030 con 0x7fa94c0c31c0 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa950e0b700 1 -- 172.21.15.154:0/4268166400 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480043c0 con 0x7fa94c0c31c0 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 -- 172.21.15.154:0/4268166400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 msgr2=0x7fa94c0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 --2- 172.21.15.154:0/4268166400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 0x7fa94c0c6140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa948009870 tx=0x7fa948009b80 comp rx=0 tx=0).stop 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 -- 172.21.15.154:0/4268166400 shutdown_connections 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 --2- 172.21.15.154:0/4268166400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c31c0 0x7fa94c0c6140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 -- 172.21.15.154:0/4268166400 >> 172.21.15.154:0/4268166400 conn(0x7fa94c0a9bf0 msgr2=0x7fa94c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 -- 172.21.15.154:0/4268166400 shutdown_connections 2024-03-14T09:02:48.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.281+0000 7fa954071700 1 -- 172.21.15.154:0/4268166400 wait complete. 2024-03-14T09:02:48.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 Processor -- start 2024-03-14T09:02:48.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 -- start start 2024-03-14T09:02:48.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 0x7fa94c0ac760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c6680 0x7fa94c1df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1dcf50 con 0x7fa94c06c670 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1eab10 con 0x7fa94c0c31c0 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa954071700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c1eadf0 con 0x7fa94c0c6680 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa951e0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa951e0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:37796/0 (socket says 172.21.15.154:37796) 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.283+0000 7fa951e0d700 1 -- 172.21.15.154:0/3837019000 learned_addr learned my addr 172.21.15.154:0/3837019000 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa95160c700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 0x7fa94c0ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa95260e700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c6680 0x7fa94c1df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 -- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c6680 msgr2=0x7fa94c1df2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c6680 0x7fa94c1df2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 -- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 msgr2=0x7fa94c0ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 0x7fa94c0ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 -- 172.21.15.154:0/3837019000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480094a0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa95160c700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 0x7fa94c0ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa951e0d700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c500 tx=0x7fa93c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0052d0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c005450 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0038c0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c1dd7a0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.284+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c1d9be0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.285+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c0697d0 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.285+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa93c003a20 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.286+0000 7fa9427fc700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 0x7fa93006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.286+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5885+0+0 (secure 0 0 0) 0x7fa93c088250 con 0x7fa94c06c670 2024-03-14T09:02:48.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.286+0000 7fa95160c700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 0x7fa93006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:48.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.286+0000 7fa95160c700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 0x7fa93006ccf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9480090b0 tx=0x7fa948008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:48.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.290+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa93c053650 con 0x7fa94c06c670 2024-03-14T09:02:48.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:48.530+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa94c06ff50 con 0x7fa94c06c670 2024-03-14T09:02:49.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.136+0000 7fa9427fc700 1 -- 172.21.15.154:0/3837019000 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa93c056c70 con 0x7fa94c06c670 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 msgr2=0x7fa93006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 0x7fa93006ccf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9480090b0 tx=0x7fa948008030 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 msgr2=0x7fa94c0aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c500 tx=0x7fa93c00c8c0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 shutdown_connections 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa94c0c6680 0x7fa94c1df2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa93006a830 0x7fa93006ccf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa94c06c670 0x7fa94c0aaef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 --2- 172.21.15.154:0/3837019000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa94c0c31c0 0x7fa94c0ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 >> 172.21.15.154:0/3837019000 conn(0x7fa94c0a9bf0 msgr2=0x7fa94c06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:49.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 shutdown_connections 2024-03-14T09:02:49.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.139+0000 7fa954071700 1 -- 172.21.15.154:0/3837019000 wait complete. 2024-03-14T09:02:49.152 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-14T09:02:49.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.320+0000 7f83a4b51700 1 Processor -- start 2024-03-14T09:02:49.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.320+0000 7f83a4b51700 1 -- start start 2024-03-14T09:02:49.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f83a4b51700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83a0153130 0x7f83a0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:49.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f83a4b51700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a006e9f0 con 0x7f83a0154fd0 2024-03-14T09:02:49.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f83a4b51700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a006ecd0 con 0x7f83a014b490 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f83a4b51700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a00a8860 con 0x7f83a0153130 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f839dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83a0153130 0x7f83a0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f839dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83a0153130 0x7f83a0154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36384/0 (socket says 172.21.15.154:36384) 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f839dd9b700 1 -- 172.21.15.154:0/2269133967 learned_addr learned my addr 172.21.15.154:0/2269133967 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.321+0000 7f839ed9d700 1 --1- 172.21.15.154:0/2269133967 >> v1:172.21.15.154:6789/0 conn(0x7f83a0154fd0 0x7f83a015be20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50380/0 (socket says 172.21.15.154:50380) 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839dd9b700 1 -- 172.21.15.154:0/2269133967 >> v1:172.21.15.191:6789/0 conn(0x7f83a014b490 legacy=0x7f83a0152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839dd9b700 1 -- 172.21.15.154:0/2269133967 >> v1:172.21.15.154:6789/0 conn(0x7f83a0154fd0 legacy=0x7f83a015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839dd9b700 1 -- 172.21.15.154:0/2269133967 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a006efb0 con 0x7f83a0153130 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839dd9b700 1 --2- 172.21.15.154:0/2269133967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a0154a90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f839000bb90 tx=0x7f839000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13047d974b912974 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839d59a700 1 -- 172.21.15.154:0/2269133967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839000dd10 con 0x7f83a0153130 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839d59a700 1 -- 172.21.15.154:0/2269133967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000de70 con 0x7f83a0153130 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f839d59a700 1 -- 172.21.15.154:0/2269133967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390004230 con 0x7f83a0153130 2024-03-14T09:02:49.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f83a4b51700 1 -- 172.21.15.154:0/2269133967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 msgr2=0x7f83a0154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.322+0000 7f83a4b51700 1 --2- 172.21.15.154:0/2269133967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a0154a90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f839000bb90 tx=0x7f839000bea0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.323+0000 7f83a4b51700 1 -- 172.21.15.154:0/2269133967 shutdown_connections 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.323+0000 7f83a4b51700 1 --2- 172.21.15.154:0/2269133967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a0154a90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.323+0000 7f83a4b51700 1 -- 172.21.15.154:0/2269133967 >> 172.21.15.154:0/2269133967 conn(0x7f83a00c3980 msgr2=0x7f83a00c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.323+0000 7f83a4b51700 1 -- 172.21.15.154:0/2269133967 shutdown_connections 2024-03-14T09:02:49.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.323+0000 7f83a4b51700 1 -- 172.21.15.154:0/2269133967 wait complete. 2024-03-14T09:02:49.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.324+0000 7f83a4b51700 1 Processor -- start 2024-03-14T09:02:49.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.324+0000 7f83a4b51700 1 -- start start 2024-03-14T09:02:49.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 0x7f83a00c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 0x7f83a01e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a006f670 con 0x7f83a0153130 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00a9360 con 0x7f83a0154fd0 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00ab390 con 0x7f83a014b490 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36394/0 (socket says 172.21.15.154:36394) 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 -- 172.21.15.154:0/1466034858 learned_addr learned my addr 172.21.15.154:0/1466034858 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839ed9d700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 0x7f83a01e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839e59c700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 0x7f83a00c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 -- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 msgr2=0x7f83a00c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 0x7f83a00c2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 -- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 msgr2=0x7f83a01e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:49.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 0x7f83a01e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 -- 172.21.15.154:0/1466034858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839000b7f0 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839ed9d700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 0x7f83a01e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839e59c700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 0x7f83a00c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f839dd9b700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f839000bee0 tx=0x7f8390004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390010670 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83900107d0 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a00ac250 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390010b40 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.325+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a0145ab0 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.326+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a00c6390 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.327+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f839002c430 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.327+0000 7f838effd700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 0x7f838006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.327+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5885+0+0 (secure 0 0 0) 0x7f839008b160 con 0x7f83a0153130 2024-03-14T09:02:49.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.327+0000 7f839e59c700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 0x7f838006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:49.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.328+0000 7f839e59c700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 0x7f838006ccf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8394003f40 tx=0x7f8394009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:49.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.332+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f83a00c6390 con 0x7f83a0153130 2024-03-14T09:02:49.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:49.526+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83a00c6390 con 0x7f83a0153130 2024-03-14T09:02:50.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.521+0000 7f838effd700 1 -- 172.21.15.154:0/1466034858 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7f83a00c6390 con 0x7f83a0153130 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 msgr2=0x7f838006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 0x7f838006ccf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8394003f40 tx=0x7f8394009040 comp rx=0 tx=0).stop 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 msgr2=0x7f83a01ed4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f839000bee0 tx=0x7f8390004750 comp rx=0 tx=0).stop 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 shutdown_connections 2024-03-14T09:02:50.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83a0154fd0 0x7f83a01e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83a014b490 0x7f83a00c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f838006a830 0x7f838006ccf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 --2- 172.21.15.154:0/1466034858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83a0153130 0x7f83a01ed4a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 >> 172.21.15.154:0/1466034858 conn(0x7f83a00c3980 msgr2=0x7f83a006db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 shutdown_connections 2024-03-14T09:02:50.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.524+0000 7f83a4b51700 1 -- 172.21.15.154:0/1466034858 wait complete. 2024-03-14T09:02:50.528 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' removed 2024-03-14T09:02:50.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-14T09:02:50.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.675+0000 7fad41c92700 1 Processor -- start 2024-03-14T09:02:50.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.675+0000 7fad41c92700 1 -- start start 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad41c92700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad3c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad41c92700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad3c13bbb0 0x7fad3c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad41c92700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c13b9e0 con 0x7fad3c156820 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad41c92700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c0a9690 con 0x7fad3c13bbb0 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad41c92700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c13b700 con 0x7fad3c13b170 2024-03-14T09:02:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fad3c156820 0x7fad3c06f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50386/0 (socket says 172.21.15.154:50386) 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3b7fe700 1 -- 172.21.15.154:0/3193393048 learned_addr learned my addr 172.21.15.154:0/3193393048 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709127391 0 0) 0x7fad3c13b9e0 con 0x7fad3c156820 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3affd700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad3c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3bfff700 1 --2- 172.21.15.154:0/3193393048 >> v2:172.21.15.191:3300/0 conn(0x7fad3c13bbb0 0x7fad3c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.676+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad280035b0 con 0x7fad3c156820 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418733928 0 0) 0x7fad280035b0 con 0x7fad3c156820 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 msgr2=0x7fad3c1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:50.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad3c1526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 >> v2:172.21.15.191:3300/0 conn(0x7fad3c13bbb0 msgr2=0x7fad3c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 --2- 172.21.15.154:0/3193393048 >> v2:172.21.15.191:3300/0 conn(0x7fad3c13bbb0 0x7fad3c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c155d40 con 0x7fad3c156820 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fad24001670 con 0x7fad3c156820 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 >> v1:172.21.15.154:6789/0 conn(0x7fad3c156820 legacy=0x7fad3c06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad28003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad28003f80 0x7fad28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28004d90 0x7fad28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0a9690 con 0x7fad28003f80 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c13b700 con 0x7fad28004d90 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2800baf0 con 0x7fad3c13b170 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad28003f80 0x7fad28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3bfff700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28004d90 0x7fad28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3b7fe700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad28003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 -- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 msgr2=0x7fad28003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad3c13b170 0x7fad28003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 -- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28004d90 msgr2=0x7fad28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad28004d90 0x7fad28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 -- 172.21.15.154:0/3193393048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280035b0 con 0x7fad28003f80 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3affd700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad28003f80 0x7fad28004850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b60 tx=0x7fad28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cfa9c491ea0c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.677+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00e6c0 con 0x7fad28003f80 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00e820 con 0x7fad28003f80 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad3a7fc700 1 -- 172.21.15.154:0/3193393048 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00eb90 con 0x7fad28003f80 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 -- 172.21.15.154:0/3193393048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad28003f80 msgr2=0x7fad28004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 --2- 172.21.15.154:0/3193393048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad28003f80 0x7fad28004850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009b60 tx=0x7fad28005880 comp rx=0 tx=0).stop 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad3bfff700 1 -- 172.21.15.154:0/3193393048 reap_dead start 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 -- 172.21.15.154:0/3193393048 shutdown_connections 2024-03-14T09:02:50.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 -- 172.21.15.154:0/3193393048 >> 172.21.15.154:0/3193393048 conn(0x7fad3c0c3980 msgr2=0x7fad3c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:50.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 -- 172.21.15.154:0/3193393048 shutdown_connections 2024-03-14T09:02:50.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.678+0000 7fad41c92700 1 -- 172.21.15.154:0/3193393048 wait complete. 2024-03-14T09:02:50.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.679+0000 7fad41c92700 1 Processor -- start 2024-03-14T09:02:50.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.679+0000 7fad41c92700 1 -- start start 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 0x7fad3c0c1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad3c1dcdc0 0x7fad3c1eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c1dc3f0 con 0x7fad3c1de0f0 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c1e8e00 con 0x7fad3c1dcdc0 2024-03-14T09:02:50.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad41c92700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c1e90e0 con 0x7fad28003f80 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36416/0 (socket says 172.21.15.154:36416) 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 -- 172.21.15.154:0/753648000 learned_addr learned my addr 172.21.15.154:0/753648000 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3bfff700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad3c1dcdc0 0x7fad3c1eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3b7fe700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 0x7fad3c0c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 -- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 msgr2=0x7fad3c0c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 0x7fad3c0c1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 -- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad3c1dcdc0 msgr2=0x7fad3c1eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.680+0000 7fad3affd700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad3c1dcdc0 0x7fad3c1eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad3affd700 1 -- 172.21.15.154:0/753648000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c1dbda0 con 0x7fad3c1de0f0 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad3b7fe700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 0x7fad3c0c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad3affd700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fad2800ba40 tx=0x7fad2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:50.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00e6c0 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c003ad0 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c1dc080 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c003e80 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.681+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c1de730 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.682+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c1dc560 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.682+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fad2c013470 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.683+0000 7fad33fff700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 0x7fad1c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.683+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5445+0+0 (secure 0 0 0) 0x7fad2c08a380 con 0x7fad3c1de0f0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.683+0000 7fad3b7fe700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 0x7fad1c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.684+0000 7fad3b7fe700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 0x7fad1c06cd20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0ac070 tx=0x7fad24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:50.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.687+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fad2c0559b0 con 0x7fad3c1de0f0 2024-03-14T09:02:50.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:50.881+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad3c0697d0 con 0x7fad3c1de0f0 2024-03-14T09:02:51.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.524+0000 7fad33fff700 1 -- 172.21.15.154:0/753648000 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v42) v1 ==== 172+0+0 (secure 0 0 0) 0x7fad2c058fd0 con 0x7fad3c1de0f0 2024-03-14T09:02:51.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.527+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 msgr2=0x7fad1c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.527+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 0x7fad1c06cd20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0ac070 tx=0x7fad24009040 comp rx=0 tx=0).stop 2024-03-14T09:02:51.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.527+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 msgr2=0x7fad3c1dc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.527+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fad2800ba40 tx=0x7fad2c0047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 shutdown_connections 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad3c1dcdc0 0x7fad3c1eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad28003f80 0x7fad3c0c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fad1c06a860 0x7fad1c06cd20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 --2- 172.21.15.154:0/753648000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad3c1de0f0 0x7fad3c1dc880 secure :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fad2800ba40 tx=0x7fad2c0047c0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 >> 172.21.15.154:0/753648000 conn(0x7fad3c0c3980 msgr2=0x7fad3c138940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 shutdown_connections 2024-03-14T09:02:51.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.528+0000 7fad41c92700 1 -- 172.21.15.154:0/753648000 wait complete. 2024-03-14T09:02:51.529 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' removed 2024-03-14T09:02:51.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T09:02:51.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 Processor -- start 2024-03-14T09:02:51.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 -- start start 2024-03-14T09:02:51.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd6c14eaf0 0x7fcd6c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 0x7fcd6c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd6c156330 0x7fcd6c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c06b310 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c0ac8e0 con 0x7fcd6c156330 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.666+0000 7fcd72cb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c06de30 con 0x7fcd6c06acb0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd6c14eaf0 0x7fcd6c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd6c14eaf0 0x7fcd6c06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36424/0 (socket says 172.21.15.154:36424) 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 -- 172.21.15.154:0/2733931709 learned_addr learned my addr 172.21.15.154:0/2733931709 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd71251700 1 --2- 172.21.15.154:0/2733931709 >> v2:172.21.15.191:3300/0 conn(0x7fcd6c156330 0x7fcd6c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd6bfff700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 0x7fcd6c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 -- 172.21.15.154:0/2733931709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 msgr2=0x7fcd6c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 0x7fcd6c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 -- 172.21.15.154:0/2733931709 >> v2:172.21.15.191:3300/0 conn(0x7fcd6c156330 msgr2=0x7fcd6c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 --2- 172.21.15.154:0/2733931709 >> v2:172.21.15.191:3300/0 conn(0x7fcd6c156330 0x7fcd6c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 -- 172.21.15.154:0/2733931709 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0acb30 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd6bfff700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 0x7fcd6c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd70a50700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c06a770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c070e50 tx=0x7fcd60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caaf6cec5b7a3e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:51.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd6b7fe700 1 -- 172.21.15.154:0/2733931709 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000b870 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd6b7fe700 1 -- 172.21.15.154:0/2733931709 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000b9d0 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.667+0000 7fcd6b7fe700 1 -- 172.21.15.154:0/2733931709 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000bd60 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 -- 172.21.15.154:0/2733931709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 msgr2=0x7fcd6c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c06a770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c070e50 tx=0x7fcd60009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 -- 172.21.15.154:0/2733931709 shutdown_connections 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/2733931709 >> v2:172.21.15.191:3300/0 conn(0x7fcd6c156330 0x7fcd6c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c06a770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/2733931709 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c06acb0 0x7fcd6c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 -- 172.21.15.154:0/2733931709 >> 172.21.15.154:0/2733931709 conn(0x7fcd6c0c3880 msgr2=0x7fcd6c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 -- 172.21.15.154:0/2733931709 shutdown_connections 2024-03-14T09:02:51.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.668+0000 7fcd72cb4700 1 -- 172.21.15.154:0/2733931709 wait complete. 2024-03-14T09:02:51.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.669+0000 7fcd72cb4700 1 Processor -- start 2024-03-14T09:02:51.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.669+0000 7fcd72cb4700 1 -- start start 2024-03-14T09:02:51.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06acb0 0x7fcd6c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c156330 0x7fcd6c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c153d40 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1534a0 con 0x7fcd6c06acb0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd72cb4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c153780 con 0x7fcd6c156330 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.670+0000 7fcd6bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36440/0 (socket says 172.21.15.154:36440) 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 -- 172.21.15.154:0/3754827204 learned_addr learned my addr 172.21.15.154:0/3754827204 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd70a50700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06acb0 0x7fcd6c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd71251700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c156330 0x7fcd6c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 -- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c156330 msgr2=0x7fcd6c0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c156330 0x7fcd6c0c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 -- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06acb0 msgr2=0x7fcd6c152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06acb0 0x7fcd6c152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 -- 172.21.15.154:0/3754827204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd60009550 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd6bfff700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c7f0 tx=0x7fcd5800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011800 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58011960 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c153a60 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58011d10 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.671+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c148bd0 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.672+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c0c2270 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.672+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd58018360 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.673+0000 7fcd697fa700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 0x7fcd5006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.673+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcd58088980 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.673+0000 7fcd70a50700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 0x7fcd5006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:51.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.674+0000 7fcd70a50700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 0x7fcd5006ccf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c156270 tx=0x7fcd60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:51.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.677+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcd6c0c2270 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.880+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcd6c0c2270 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.883+0000 7fcd697fa700 1 -- 172.21.15.154:0/3754827204 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd6c0c2270 con 0x7fcd6c14eaf0 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.885+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 msgr2=0x7fcd5006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.885+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 0x7fcd5006ccf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c156270 tx=0x7fcd60008030 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 msgr2=0x7fcd6c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c7f0 tx=0x7fcd5800cbb0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 shutdown_connections 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd6c156330 0x7fcd6c0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6c06acb0 0x7fcd6c152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd5006a830 0x7fcd5006ccf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 --2- 172.21.15.154:0/3754827204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6c14eaf0 0x7fcd6c152c50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 >> 172.21.15.154:0/3754827204 conn(0x7fcd6c0c3880 msgr2=0x7fcd6c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 shutdown_connections 2024-03-14T09:02:51.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:51.886+0000 7fcd72cb4700 1 -- 172.21.15.154:0/3754827204 wait complete. 2024-03-14T09:02:51.898 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent' 2024-03-14T09:02:52.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.019+0000 7fe717c43700 1 Processor -- start 2024-03-14T09:02:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.019+0000 7fe717c43700 1 -- start start 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe717c43700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7100c1e90 0x7fe71013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe717c43700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71013ade0 con 0x7fe7100c3250 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe717c43700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71006fb30 con 0x7fe7100a8540 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe717c43700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71006ec30 con 0x7fe7100c1e90 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7161e0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7100c1e90 0x7fe71013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7151de700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe7100c3250 0x7fe7100c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50388/0 (socket says 172.21.15.154:50388) 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7151de700 1 -- 172.21.15.154:0/458451930 learned_addr learned my addr 172.21.15.154:0/458451930 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7161e0700 1 -- 172.21.15.154:0/458451930 >> v1:172.21.15.191:6789/0 conn(0x7fe7100a8540 legacy=0x7fe710154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7161e0700 1 -- 172.21.15.154:0/458451930 >> v1:172.21.15.154:6789/0 conn(0x7fe7100c3250 legacy=0x7fe7100c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7161e0700 1 -- 172.21.15.154:0/458451930 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71006ef10 con 0x7fe7100c1e90 2024-03-14T09:02:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7149dd700 1 -- 172.21.15.154:0/458451930 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564113602 0 0) 0x7fe70c001110 con 0x7fe7100c3250 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7161e0700 1 --2- 172.21.15.154:0/458451930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c1e90 0x7fe71013a8a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe708009d90 tx=0x7fe70800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba402c972d613688 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7149dd700 1 -- 172.21.15.154:0/458451930 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800e070 con 0x7fe7100c1e90 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7149dd700 1 -- 172.21.15.154:0/458451930 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70800bb90 con 0x7fe7100c1e90 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.020+0000 7fe7149dd700 1 -- 172.21.15.154:0/458451930 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708003a20 con 0x7fe7100c1e90 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 -- 172.21.15.154:0/458451930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c1e90 msgr2=0x7fe71013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 --2- 172.21.15.154:0/458451930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c1e90 0x7fe71013a8a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe708009d90 tx=0x7fe70800b330 comp rx=0 tx=0).stop 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 -- 172.21.15.154:0/458451930 shutdown_connections 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 --2- 172.21.15.154:0/458451930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c1e90 0x7fe71013a8a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 -- 172.21.15.154:0/458451930 >> 172.21.15.154:0/458451930 conn(0x7fe7100c3880 msgr2=0x7fe7100c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 -- 172.21.15.154:0/458451930 shutdown_connections 2024-03-14T09:02:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.021+0000 7fe717c43700 1 -- 172.21.15.154:0/458451930 wait complete. 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.022+0000 7fe717c43700 1 Processor -- start 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.022+0000 7fe717c43700 1 -- start start 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 0x7fe7101eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7100c1e90 0x7fe7101ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101eb190 con 0x7fe7100c3250 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101525a0 con 0x7fe7100c1e90 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe717c43700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100acf60 con 0x7fe7100a8540 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36454/0 (socket says 172.21.15.154:36454) 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 -- 172.21.15.154:0/1647923507 learned_addr learned my addr 172.21.15.154:0/1647923507 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7151de700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7100c1e90 0x7fe7101ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7159df700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 0x7fe7101eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 -- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 msgr2=0x7fe7101eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 0x7fe7101eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 -- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7100c1e90 msgr2=0x7fe7101ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7100c1e90 0x7fe7101ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7161e0700 1 -- 172.21.15.154:0/1647923507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708009a70 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.023+0000 7fe7159df700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 0x7fe7101eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe7161e0700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c43c0 tx=0x7fe708004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708012630 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708012790 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708012b40 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe710151d20 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.024+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe710151fe0 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.025+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7100697d0 con 0x7fe7100c3250 2024-03-14T09:02:52.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.025+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe708012ca0 con 0x7fe7100c3250 2024-03-14T09:02:52.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.026+0000 7fe7067fc700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 0x7fe6f406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.026+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe70808ac70 con 0x7fe7100c3250 2024-03-14T09:02:52.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.026+0000 7fe7159df700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 0x7fe6f406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.026+0000 7fe7159df700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 0x7fe6f406ccf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe700009910 tx=0x7fe700008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.030 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.030+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe708056320 con 0x7fe7100c3250 2024-03-14T09:02:52.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.309+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]} v 0) v1 -- 0x7fe71006d420 con 0x7fe7100c3250 2024-03-14T09:02:52.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.528+0000 7fe7067fc700 1 -- 172.21.15.154:0/1647923507 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_all_caps_absent"]}]=0 v108) v1 ==== 159+0+0 (secure 0 0 0) 0x7fe708059940 con 0x7fe7100c3250 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 msgr2=0x7fe6f406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 0x7fe6f406ccf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe700009910 tx=0x7fe700008040 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 msgr2=0x7fe710133070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe7100c43c0 tx=0x7fe708004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 shutdown_connections 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe6f406a830 0x7fe6f406ccf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7100c3250 0x7fe710133070 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7100a8540 0x7fe7101eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 --2- 172.21.15.154:0/1647923507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7100c1e90 0x7fe7101ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 >> 172.21.15.154:0/1647923507 conn(0x7fe7100c3880 msgr2=0x7fe710139670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 shutdown_connections 2024-03-14T09:02:52.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.531+0000 7fe717c43700 1 -- 172.21.15.154:0/1647923507 wait complete. 2024-03-14T09:02:52.543 INFO:tasks.cephfs_test_runner:test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) ... ok 2024-03-14T09:02:52.543 INFO:tasks.cephfs_test_runner:Starting test: test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2024-03-14T09:02:52.543 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent' 2024-03-14T09:02:52.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.665+0000 7f7b16b77700 1 Processor -- start 2024-03-14T09:02:52.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.665+0000 7f7b16b77700 1 -- start start 2024-03-14T09:02:52.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b16b77700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b1014eaf0 0x7f7b1006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b16b77700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7b1006acb0 0x7f7b10155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b16b77700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b100ac8e0 con 0x7f7b10156330 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b16b77700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1006b310 con 0x7f7b1014eaf0 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b16b77700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1006de30 con 0x7f7b1006acb0 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b1014eaf0 0x7f7b1006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b15114700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7b10156330 0x7f7b10157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50404/0 (socket says 172.21.15.154:50404) 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b1014eaf0 0x7f7b1006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36470/0 (socket says 172.21.15.154:36470) 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b15114700 1 -- 172.21.15.154:0/368845172 learned_addr learned my addr 172.21.15.154:0/368845172 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b0ffff700 1 --2- 172.21.15.154:0/368845172 >> v2:172.21.15.191:3300/0 conn(0x7f7b1006acb0 0x7f7b10155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 -- 172.21.15.154:0/368845172 >> v2:172.21.15.191:3300/0 conn(0x7f7b1006acb0 msgr2=0x7f7b10155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 --2- 172.21.15.154:0/368845172 >> v2:172.21.15.191:3300/0 conn(0x7f7b1006acb0 0x7f7b10155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 -- 172.21.15.154:0/368845172 >> v1:172.21.15.154:6789/0 conn(0x7f7b10156330 legacy=0x7f7b10157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 -- 172.21.15.154:0/368845172 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100acb30 con 0x7f7b1014eaf0 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b0f7fe700 1 -- 172.21.15.154:0/368845172 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642679688 0 0) 0x7f7b100ac8e0 con 0x7f7b10156330 2024-03-14T09:02:52.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.666+0000 7f7b14913700 1 --2- 172.21.15.154:0/368845172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b1014eaf0 0x7f7b1006a770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7b1014f1c0 tx=0x7f7b04009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b48f2803af2f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b0f7fe700 1 -- 172.21.15.154:0/368845172 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011830 con 0x7f7b1014eaf0 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b0f7fe700 1 -- 172.21.15.154:0/368845172 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04011990 con 0x7f7b1014eaf0 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b0f7fe700 1 -- 172.21.15.154:0/368845172 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b04011d20 con 0x7f7b1014eaf0 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 -- 172.21.15.154:0/368845172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b1014eaf0 msgr2=0x7f7b1006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 --2- 172.21.15.154:0/368845172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b1014eaf0 0x7f7b1006a770 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7b1014f1c0 tx=0x7f7b04009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 -- 172.21.15.154:0/368845172 shutdown_connections 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 --2- 172.21.15.154:0/368845172 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b1014eaf0 0x7f7b1006a770 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 --2- 172.21.15.154:0/368845172 >> v2:172.21.15.191:3300/0 conn(0x7f7b1006acb0 0x7f7b10155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 -- 172.21.15.154:0/368845172 >> 172.21.15.154:0/368845172 conn(0x7f7b100c3880 msgr2=0x7f7b100c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 -- 172.21.15.154:0/368845172 shutdown_connections 2024-03-14T09:02:52.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.667+0000 7f7b16b77700 1 -- 172.21.15.154:0/368845172 wait complete. 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.668+0000 7f7b16b77700 1 Processor -- start 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.668+0000 7f7b16b77700 1 -- start start 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 0x7f7b10157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1014eaf0 0x7f7b10153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10153170 con 0x7f7b10156330 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10154090 con 0x7f7b1014eaf0 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b16b77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10152840 con 0x7f7b1006acb0 2024-03-14T09:02:52.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b15114700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b15114700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36480/0 (socket says 172.21.15.154:36480) 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b15114700 1 -- 172.21.15.154:0/2590452105 learned_addr learned my addr 172.21.15.154:0/2590452105 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.669+0000 7f7b0ffff700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1014eaf0 0x7f7b10153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b14913700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 0x7f7b10157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 -- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 msgr2=0x7f7b10157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 0x7f7b10157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 -- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1014eaf0 msgr2=0x7f7b10153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1014eaf0 0x7f7b10153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 -- 172.21.15.154:0/2590452105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040094a0 con 0x7f7b10156330 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b14913700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 0x7f7b10157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b15114700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800cca0 tx=0x7f7b0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800ac20 con 0x7f7b10156330 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800ad80 con 0x7f7b10156330 2024-03-14T09:02:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b10152b20 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08007310 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.670+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b10158070 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.671+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b10067970 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.671+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7b08007470 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.672+0000 7f7b0d7fa700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 0x7f7af406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.672+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7b080888c0 con 0x7f7b10156330 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.672+0000 7f7b14913700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 0x7f7af406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.672+0000 7f7b14913700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 0x7f7af406ccf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b040090b0 tx=0x7f7b04008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.676+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7b08054010 con 0x7f7b10156330 2024-03-14T09:02:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:52.950+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]} v 0) v1 -- 0x7f7b10067dc0 con 0x7f7b10156330 2024-03-14T09:02:53.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.530+0000 7f7b0d7fa700 1 -- 172.21.15.154:0/2590452105 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]}]=0 v109) v1 ==== 187+0+0 (secure 0 0 0) 0x7f7b08018020 con 0x7f7b10156330 2024-03-14T09:02:53.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 msgr2=0x7f7af406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 0x7f7af406ccf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b040090b0 tx=0x7f7b04008030 comp rx=0 tx=0).stop 2024-03-14T09:02:53.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 msgr2=0x7f7b101edbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800cca0 tx=0x7f7b0800a060 comp rx=0 tx=0).stop 2024-03-14T09:02:53.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 shutdown_connections 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7af406a830 0x7f7af406ccf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b10156330 0x7f7b101edbd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b1006acb0 0x7f7b10157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 --2- 172.21.15.154:0/2590452105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1014eaf0 0x7f7b10153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 >> 172.21.15.154:0/2590452105 conn(0x7f7b100c3880 msgr2=0x7f7b10149dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 shutdown_connections 2024-03-14T09:02:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.533+0000 7f7b16b77700 1 -- 172.21.15.154:0/2590452105 wait complete. 2024-03-14T09:02:53.547 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T09:02:53.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.673+0000 7f9e1f142700 1 Processor -- start 2024-03-14T09:02:53.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 -- start start 2024-03-14T09:02:53.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9e180702f0 0x7f9e180c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e180c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180acd10 con 0x7f9e1814fbd0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180ac7b0 con 0x7f9e180702f0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e1f142700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18150050 con 0x7f9e180c5d60 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e17fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9e1814fbd0 0x7f9e180c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50420/0 (socket says 172.21.15.154:50420) 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.674+0000 7f9e17fff700 1 -- 172.21.15.154:0/2500646360 learned_addr learned my addr 172.21.15.154:0/2500646360 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1d6df700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e180c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 --2- 172.21.15.154:0/2500646360 >> v2:172.21.15.191:3300/0 conn(0x7f9e180702f0 0x7f9e180c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956017492 0 0) 0x7f9e180acd10 con 0x7f9e1814fbd0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e080035b0 con 0x7f9e1814fbd0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834893401 0 0) 0x7f9e080035b0 con 0x7f9e1814fbd0 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 msgr2=0x7f9e180c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:53.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e180c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 >> v2:172.21.15.191:3300/0 conn(0x7f9e180702f0 msgr2=0x7f9e180c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 --2- 172.21.15.154:0/2500646360 >> v2:172.21.15.191:3300/0 conn(0x7f9e180702f0 0x7f9e180c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180ac9c0 con 0x7f9e1814fbd0 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9e04001670 con 0x7f9e1814fbd0 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 >> v1:172.21.15.154:6789/0 conn(0x7f9e1814fbd0 legacy=0x7f9e180c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e08003a00 0x7f9e08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08004d90 0x7f9e08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180ac7b0 con 0x7f9e08003a00 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18150050 con 0x7f9e08004d90 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0800ba10 con 0x7f9e180c5d60 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e08003a00 0x7f9e08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e17fff700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1d6df700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08004d90 0x7f9e08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.679 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 -- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 msgr2=0x7f9e08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e180c5d60 0x7f9e08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 -- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08004d90 msgr2=0x7f9e08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08004d90 0x7f9e08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.675+0000 7f9e1cede700 1 -- 172.21.15.154:0/2500646360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080035b0 con 0x7f9e08003a00 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1cede700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e08003a00 0x7f9e08003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009b60 tx=0x7f9e08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef2799ba6526ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00e6c0 con 0x7f9e08003a00 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c00e820 con 0x7f9e08003a00 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e177fe700 1 -- 172.21.15.154:0/2500646360 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00eb90 con 0x7f9e08003a00 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 -- 172.21.15.154:0/2500646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e08003a00 msgr2=0x7f9e08003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 --2- 172.21.15.154:0/2500646360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e08003a00 0x7f9e08003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009b60 tx=0x7f9e08005880 comp rx=0 tx=0).stop 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1d6df700 1 -- 172.21.15.154:0/2500646360 reap_dead start 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 -- 172.21.15.154:0/2500646360 shutdown_connections 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 -- 172.21.15.154:0/2500646360 >> 172.21.15.154:0/2500646360 conn(0x7f9e180a9bf0 msgr2=0x7f9e180a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:53.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 -- 172.21.15.154:0/2500646360 shutdown_connections 2024-03-14T09:02:53.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.676+0000 7f9e1f142700 1 -- 172.21.15.154:0/2500646360 wait complete. 2024-03-14T09:02:53.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 Processor -- start 2024-03-14T09:02:53.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 -- start start 2024-03-14T09:02:53.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08003a00 0x7f9e180c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 0x7f9e18155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18148200 con 0x7f9e1806b880 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18148b70 con 0x7f9e08003a00 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.678+0000 7f9e1f142700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18153af0 con 0x7f9e18147cb0 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36498/0 (socket says 172.21.15.154:36498) 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 -- 172.21.15.154:0/3170487274 learned_addr learned my addr 172.21.15.154:0/3170487274 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1cede700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08003a00 0x7f9e180c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e17fff700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 0x7f9e18155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 -- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 msgr2=0x7f9e18155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 0x7f9e18155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 -- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08003a00 msgr2=0x7f9e180c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08003a00 0x7f9e180c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 -- 172.21.15.154:0/3170487274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18153dd0 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e17fff700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 0x7f9e18155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1d6df700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c8bd0 tx=0x7f9e10011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000e040 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000f040 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000fd30 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e18148dc0 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.679+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1814d2d0 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.680+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e18153ff0 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.681+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9e10015430 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.681+0000 7f9e157fa700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 0x7f9dfc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.681+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9e100892c0 con 0x7f9e1806b880 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.681+0000 7f9e1cede700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 0x7f9dfc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:53.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.682+0000 7f9e1cede700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 0x7f9dfc06cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0090b0 tx=0x7f9e0c01f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:53.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.686+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9e10054a10 con 0x7f9e1806b880 2024-03-14T09:02:53.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.882+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9e18148fb0 con 0x7f9e1806b880 2024-03-14T09:02:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.886+0000 7f9e157fa700 1 -- 172.21.15.154:0/3170487274 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e10058030 con 0x7f9e1806b880 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 msgr2=0x7f9dfc06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 0x7f9dfc06cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0090b0 tx=0x7f9e0c01f040 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 msgr2=0x7f9e18155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9e180c8bd0 tx=0x7f9e10011ed0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 shutdown_connections 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9dfc06a860 0x7f9dfc06cd20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e1806b880 0x7f9e18155a60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e08003a00 0x7f9e180c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 --2- 172.21.15.154:0/3170487274 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e18147cb0 0x7f9e18155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 >> 172.21.15.154:0/3170487274 conn(0x7f9e180a9bf0 msgr2=0x7f9e180c2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 shutdown_connections 2024-03-14T09:02:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:53.889+0000 7f9e1f142700 1 -- 172.21.15.154:0/3170487274 wait complete. 2024-03-14T09:02:53.901 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:02:53.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:02:53.956 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:02:54.004 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:02:54.004 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:02:54.004 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:54.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:02:54.021 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:02:54.021 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:02:54.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:02:54.041 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-14T09:02:54.041 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:02:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.183+0000 7fcd56d64700 1 Processor -- start 2024-03-14T09:02:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 -- start start 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd500c4380 0x7fcd5014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 0x7fcd500c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500acd10 con 0x7fcd500702f0 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd50150110 con 0x7fcd500c4380 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd56d64700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd500c66b0 con 0x7fcd500c5d60 2024-03-14T09:02:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd4ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd500c4380 0x7fcd5014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd54b00700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcd500702f0 0x7fcd500c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50434/0 (socket says 172.21.15.154:50434) 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.184+0000 7fcd54b00700 1 -- 172.21.15.154:0/1078187451 learned_addr learned my addr 172.21.15.154:0/1078187451 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd55301700 1 --2- 172.21.15.154:0/1078187451 >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 0x7fcd500c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4ffff700 1 -- 172.21.15.154:0/1078187451 >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 msgr2=0x7fcd500c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/1078187451 >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 0x7fcd500c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4ffff700 1 -- 172.21.15.154:0/1078187451 >> v1:172.21.15.154:6789/0 conn(0x7fcd500702f0 legacy=0x7fcd500c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4ffff700 1 -- 172.21.15.154:0/1078187451 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500ac9c0 con 0x7fcd500c4380 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd55301700 1 --2- 172.21.15.154:0/1078187451 >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 0x7fcd500c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/1078187451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500c4380 0x7fcd5014fbd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcd500aac90 tx=0x7fcd38009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f705ea136e97817 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4f7fe700 1 -- 172.21.15.154:0/1078187451 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011830 con 0x7fcd500c4380 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4f7fe700 1 -- 172.21.15.154:0/1078187451 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38011990 con 0x7fcd500c4380 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.185+0000 7fcd4f7fe700 1 -- 172.21.15.154:0/1078187451 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38011d20 con 0x7fcd500c4380 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 -- 172.21.15.154:0/1078187451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500c4380 msgr2=0x7fcd5014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 --2- 172.21.15.154:0/1078187451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500c4380 0x7fcd5014fbd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcd500aac90 tx=0x7fcd38009b20 comp rx=0 tx=0).stop 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 -- 172.21.15.154:0/1078187451 shutdown_connections 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 --2- 172.21.15.154:0/1078187451 >> v2:172.21.15.191:3300/0 conn(0x7fcd500c5d60 0x7fcd500c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 --2- 172.21.15.154:0/1078187451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500c4380 0x7fcd5014fbd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 -- 172.21.15.154:0/1078187451 >> 172.21.15.154:0/1078187451 conn(0x7fcd500a9bf0 msgr2=0x7fcd500a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 -- 172.21.15.154:0/1078187451 shutdown_connections 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.186+0000 7fcd56d64700 1 -- 172.21.15.154:0/1078187451 wait complete. 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.187+0000 7fcd56d64700 1 Processor -- start 2024-03-14T09:02:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.187+0000 7fcd56d64700 1 -- start start 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 0x7fcd5014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 0x7fcd50154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50153d10 con 0x7fcd500702f0 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501539c0 con 0x7fcd500c4380 2024-03-14T09:02:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50156820 con 0x7fcd500c5d60 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36510/0 (socket says 172.21.15.154:36510) 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 -- 172.21.15.154:0/2121366081 learned_addr learned my addr 172.21.15.154:0/2121366081 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 0x7fcd5014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd55301700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 0x7fcd50154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 -- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 msgr2=0x7fcd50154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 0x7fcd50154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 -- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 msgr2=0x7fcd5014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 0x7fcd5014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 -- 172.21.15.154:0/2121366081 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380094a0 con 0x7fcd500702f0 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd55301700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 0x7fcd50154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 0x7fcd5014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd54b00700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800cc00 tx=0x7fcd4800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800aba0 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4800ad00 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd501ea850 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48012600 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.188+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd501eadd0 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.189+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd50153f80 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.190+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd4801b3b0 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.190+0000 7fcd4d7fa700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 0x7fcd3006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.190+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcd48088700 con 0x7fcd500702f0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.190+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 0x7fcd3006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.191+0000 7fcd4ffff700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 0x7fcd3006ccf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcd5014d030 tx=0x7fcd38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.195+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcd48053e60 con 0x7fcd500702f0 2024-03-14T09:02:54.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.431+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd500697d0 con 0x7fcd500702f0 2024-03-14T09:02:54.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.431+0000 7fcd4d7fa700 1 -- 172.21.15.154:0/2121366081 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fcd48017df0 con 0x7fcd500702f0 2024-03-14T09:02:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:54.431 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2069474099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2069474099},{"type":"v1","addr":"172.21.15.191:6836","nonce":2069474099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4840,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/507323647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":507323647},{"type":"v1","addr":"172.21.15.191:6837","nonce":507323647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2558238943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2558238943},{"type":"v1","addr":"172.21.15.154:6835","nonce":2558238943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3021260949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3021260949},{"type":"v1","addr":"172.21.15.154:6834","nonce":3021260949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2024-03-14T09:02:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 msgr2=0x7fcd3006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 0x7fcd3006ccf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcd5014d030 tx=0x7fcd38008030 comp rx=0 tx=0).stop 2024-03-14T09:02:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 msgr2=0x7fcd501ea310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800cc00 tx=0x7fcd4800a060 comp rx=0 tx=0).stop 2024-03-14T09:02:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 shutdown_connections 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd500c5d60 0x7fcd50154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd3006a830 0x7fcd3006ccf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd500702f0 0x7fcd501ea310 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 --2- 172.21.15.154:0/2121366081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd500c4380 0x7fcd5014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 >> 172.21.15.154:0/2121366081 conn(0x7fcd500a9bf0 msgr2=0x7fcd500c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 shutdown_connections 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.433+0000 7fcd56d64700 1 -- 172.21.15.154:0/2121366081 wait complete. 2024-03-14T09:02:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 24 2024-03-14T09:02:54.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T09:02:54.446 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T09:02:54.446 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T09:02:54.446 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:02:54.446 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T09:02:54.446 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T09:02:54.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T09:02:54.447 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:02:54.447 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T09:02:54.447 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T09:02:54.447 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T09:02:54.447 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:02:54.447 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T09:02:54.447 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T09:02:54.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T09:02:54.448 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:02:54.456 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T09:02:54.456 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T09:02:54.457 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T09:02:54.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T09:02:54.460 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T09:02:54.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T09:02:54.460 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T09:02:54.461 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T09:02:54.499 INFO:tasks.ceph.mds.a:Started 2024-03-14T09:02:54.499 DEBUG:teuthology.parallel:result is None 2024-03-14T09:02:54.500 INFO:tasks.ceph.mds.c:Started 2024-03-14T09:02:54.500 DEBUG:teuthology.parallel:result is None 2024-03-14T09:02:54.502 INFO:tasks.ceph.mds.b:Started 2024-03-14T09:02:54.502 DEBUG:teuthology.parallel:result is None 2024-03-14T09:02:54.503 INFO:tasks.ceph.mds.d:Started 2024-03-14T09:02:54.503 DEBUG:teuthology.parallel:result is None 2024-03-14T09:02:54.503 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T09:02:54.581 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:02:54.581 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:02:54.596 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-14T09:02:54.599 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:02:54.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.636+0000 7f7dc9829700 1 Processor -- start 2024-03-14T09:02:54.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.636+0000 7f7dc9829700 1 -- start start 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.637+0000 7f7dc9829700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 0x7f7dc413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.637+0000 7f7dc9829700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc413ade0 con 0x7f7dc40c3250 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.637+0000 7f7dc9829700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc406fb30 con 0x7f7dc40c1780 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.637+0000 7f7dc9829700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc406ec30 con 0x7f7dc40a8540 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc27fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7dc40c3250 0x7f7dc4154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50442/0 (socket says 172.21.15.154:50442) 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc27fc700 1 -- 172.21.15.154:0/1070531990 learned_addr learned my addr 172.21.15.154:0/1070531990 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc2ffd700 1 --2- 172.21.15.154:0/1070531990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 0x7f7dc413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40168964 0 0) 0x7f7db4001110 con 0x7f7dc40c3250 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dac0035b0 con 0x7f7dc40c3250 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208040840 0 0) 0x7f7dc406fb30 con 0x7f7dc40c1780 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db4001110 con 0x7f7dc40c1780 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc2ffd700 1 -- 172.21.15.154:0/1070531990 >> v1:172.21.15.191:6789/0 conn(0x7f7dc40c1780 legacy=0x7f7dc40c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156487960 0 0) 0x7f7dac0035b0 con 0x7f7dc40c3250 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc2ffd700 1 -- 172.21.15.154:0/1070531990 >> v1:172.21.15.154:6789/0 conn(0x7f7dc40c3250 legacy=0x7f7dc4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc2ffd700 1 -- 172.21.15.154:0/1070531990 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc406ef10 con 0x7f7dc40a8540 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc2ffd700 1 --2- 172.21.15.154:0/1070531990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 0x7f7dc413a8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009870 tx=0x7f7db8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b21f1a8b6795d2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4001800 con 0x7f7dc40a8540 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8004030 con 0x7f7dc40a8540 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.638+0000 7f7dc1ffb700 1 -- 172.21.15.154:0/1070531990 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db80043a0 con 0x7f7dc40a8540 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 -- 172.21.15.154:0/1070531990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 msgr2=0x7f7dc413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1070531990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 0x7f7dc413a8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009870 tx=0x7f7db8009b80 comp rx=0 tx=0).stop 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 -- 172.21.15.154:0/1070531990 shutdown_connections 2024-03-14T09:02:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1070531990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40a8540 0x7f7dc413a8a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 -- 172.21.15.154:0/1070531990 >> 172.21.15.154:0/1070531990 conn(0x7f7dc40c3880 msgr2=0x7f7dc40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:54.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 -- 172.21.15.154:0/1070531990 shutdown_connections 2024-03-14T09:02:54.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.639+0000 7f7dc9829700 1 -- 172.21.15.154:0/1070531990 wait complete. 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.640+0000 7f7dc9829700 1 Processor -- start 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.640+0000 7f7dc9829700 1 -- start start 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 0x7f7dc414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40c1780 0x7f7dc41eb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40ac830 con 0x7f7dc40c3250 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40c12a0 con 0x7f7dc40a8540 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc9829700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40c1540 con 0x7f7dc40c1780 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 0x7f7dc414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36544/0 (socket says 172.21.15.154:36544) 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 -- 172.21.15.154:0/1434490887 learned_addr learned my addr 172.21.15.154:0/1434490887 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc27fc700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40c1780 0x7f7dc41eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 -- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40c1780 msgr2=0x7f7dc41eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40c1780 0x7f7dc41eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 -- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 msgr2=0x7f7dc414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 0x7f7dc414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc37fe700 1 -- 172.21.15.154:0/1434490887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80094a0 con 0x7f7dc40c3250 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.641+0000 7f7dc2ffd700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 0x7f7dc414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dc37fe700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00cbc0 tx=0x7f7dbc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc003c00 con 0x7f7dc40c3250 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc003d60 con 0x7f7dc40c3250 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc41ebe70 con 0x7f7dc40c3250 2024-03-14T09:02:54.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc00aaa0 con 0x7f7dc40c3250 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.642+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41ebb70 con 0x7f7dc40c3250 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.643+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc40ac9e0 con 0x7f7dc40c3250 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.643+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7dbc00ac00 con 0x7f7dc40c3250 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.644+0000 7f7dabfff700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 0x7f7db006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.644+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7dbc0889f0 con 0x7f7dc40c3250 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.644+0000 7f7dc2ffd700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 0x7f7db006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:54.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.645+0000 7f7dc2ffd700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 0x7f7db006ccf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7db80097c0 tx=0x7f7db8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:54.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.648+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7dbc054140 con 0x7f7dc40c3250 2024-03-14T09:02:54.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:54.855+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7dc40697d0 con 0x7f7dc40c3250 2024-03-14T09:02:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.630+0000 7f7dabfff700 1 -- 172.21.15.154:0/1434490887 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7dbc057760 con 0x7f7dc40c3250 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 msgr2=0x7f7db006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 0x7f7db006ccf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7db80097c0 tx=0x7f7db8008030 comp rx=0 tx=0).stop 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 msgr2=0x7f7dc414fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00cbc0 tx=0x7f7dbc00ced0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 shutdown_connections 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7db006a830 0x7f7db006ccf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7dc40c3250 0x7f7dc414fa70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7dc40a8540 0x7f7dc414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 --2- 172.21.15.154:0/1434490887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7dc40c1780 0x7f7dc41eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 >> 172.21.15.154:0/1434490887 conn(0x7f7dc40c3880 msgr2=0x7f7dc4135670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 shutdown_connections 2024-03-14T09:02:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.633+0000 7f7dc9829700 1 -- 172.21.15.154:0/1434490887 wait complete. 2024-03-14T09:02:55.634 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2024-03-14T09:02:55.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T09:02:55.770 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.770+0000 7fea0a5a3700 1 Processor -- start 2024-03-14T09:02:55.770 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 -- start start 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 0x7fea041547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea04154d30 0x7fea04155170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea04157e80 con 0x7fea040c7f70 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea04158ac0 con 0x7fea04154d30 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea0a5a3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea04158120 con 0x7fea040c8350 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea08b40700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea04154d30 0x7fea04155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea08b40700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fea04154d30 0x7fea04155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36560/0 (socket says 172.21.15.154:36560) 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.771+0000 7fea08b40700 1 -- 172.21.15.154:0/4174020343 learned_addr learned my addr 172.21.15.154:0/4174020343 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea037fe700 1 --2- 172.21.15.154:0/4174020343 >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 0x7fea041547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea08b40700 1 -- 172.21.15.154:0/4174020343 >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 msgr2=0x7fea041547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea08b40700 1 --2- 172.21.15.154:0/4174020343 >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 0x7fea041547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea08b40700 1 -- 172.21.15.154:0/4174020343 >> v1:172.21.15.191:6789/0 conn(0x7fea040c7f70 legacy=0x7fea040c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea08b40700 1 -- 172.21.15.154:0/4174020343 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04158400 con 0x7fea04154d30 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea037fe700 1 --2- 172.21.15.154:0/4174020343 >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 0x7fea041547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea08b40700 1 --2- 172.21.15.154:0/4174020343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea04155170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea0408c2d0 tx=0x7fe9fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0bca90581131cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea02ffd700 1 -- 172.21.15.154:0/4174020343 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00e070 con 0x7fea04154d30 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea02ffd700 1 -- 172.21.15.154:0/4174020343 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc003a20 con 0x7fea04154d30 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea02ffd700 1 -- 172.21.15.154:0/4174020343 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc003dd0 con 0x7fea04154d30 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.772+0000 7fea0a5a3700 1 -- 172.21.15.154:0/4174020343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 msgr2=0x7fea04155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/4174020343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea04155170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea0408c2d0 tx=0x7fe9fc00b330 comp rx=0 tx=0).stop 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 -- 172.21.15.154:0/4174020343 shutdown_connections 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/4174020343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea04155170 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/4174020343 >> v2:172.21.15.191:3300/0 conn(0x7fea040c8350 0x7fea041547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 -- 172.21.15.154:0/4174020343 >> 172.21.15.154:0/4174020343 conn(0x7fea04072080 msgr2=0x7fea04072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:55.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 -- 172.21.15.154:0/4174020343 shutdown_connections 2024-03-14T09:02:55.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.773+0000 7fea0a5a3700 1 -- 172.21.15.154:0/4174020343 wait complete. 2024-03-14T09:02:55.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.774+0000 7fea0a5a3700 1 Processor -- start 2024-03-14T09:02:55.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.774+0000 7fea0a5a3700 1 -- start start 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea040c7f70 0x7fea040bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea040c8350 0x7fea040bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040bfa60 con 0x7fea04154d30 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040bf480 con 0x7fea040c8350 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea0a5a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea04152a60 con 0x7fea040c7f70 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36572/0 (socket says 172.21.15.154:36572) 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 -- 172.21.15.154:0/1834642829 learned_addr learned my addr 172.21.15.154:0/1834642829 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea03fff700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea040c7f70 0x7fea040bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea037fe700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea040c8350 0x7fea040bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 -- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea040c7f70 msgr2=0x7fea040bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea040c7f70 0x7fea040bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 -- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea040c8350 msgr2=0x7fea040bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea040c8350 0x7fea040bfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 -- 172.21.15.154:0/1834642829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc009b30 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea08b40700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fea040cb0b0 tx=0x7fe9fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.775+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc004440 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.776+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc0092e0 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.776+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea041eb190 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.776+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0047e0 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.776+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea040be1e0 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.777+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea040bfc10 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.777+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe9fc01c440 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.777+0000 7fea00ff9700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 0x7fe9f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.777+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe9fc089c90 con 0x7fea04154d30 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.778+0000 7fea03fff700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 0x7fe9f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.778+0000 7fea03fff700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 0x7fe9f006ccf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:55.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:55.782+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fea040bfc10 con 0x7fea04154d30 2024-03-14T09:02:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.053+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fea040bfc10 con 0x7fea04154d30 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.054+0000 7fea00ff9700 1 -- 172.21.15.154:0/1834642829 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5021 (secure 0 0 0) 0x7fea040bfc10 con 0x7fea04154d30 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlnCs5CxAA/mf9z7eSw0Fh5yqCxmvhww==", 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.054 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlq18XGBAA9Nr/cXAtRbeaKmJDCLUz6w==", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:02:56.055 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlvIuREhAAO1gOyqDP2tTJcb1IuOdQ0A==", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2024-03-14T09:02:56.056 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlAAuqHxAAy5SQz2lJHPwRad7HCIjjSg==", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCvvPJlDzrPMRAAEAP7kHfAaxZGM+W0rY0DIA==", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.057 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCyvPJlgS2rMBAAoPtjrTbc7bMLMJAIUKD+vg==", 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2024-03-14T09:02:56.058 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC1vPJltO6ULxAAvkKiSoRBoKxFz8zPjGCOnA==", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC4vPJlWIs/LhAAi5LaiHbS1svlQ0UesC9CLg==", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC+vPJl5TleMRAAjIGbvgOjoV3mJYIK1fQoYA==", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDBvPJlhiV5MBAAe7XI8s0kWQRAhvWWqMInrg==", 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.060 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDEvPJlbghELxAASO+DuPJa7zXCH/KkhPi1WA==", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.061 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDHvPJlRba5LhAASpiL/U05lgta2f4xKra5ZQ==", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlhDgFJBAA47spIEFGnChaoxmMQkaA+w==", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2024-03-14T09:02:56.062 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJl58xgJxAAfqdSCJED8MwHCScl1fqdFA==", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw pool=cephfs2_data" 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlMjx9JRAAScnZoVr95MollZzYB2rQIA==", 2024-03-14T09:02:56.063 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.testuser", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAlvfJliS9sNBAA4NShMBqgEq6b944q/iluzg==", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow rw fsname=cephfs", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r fsname=cephfs", 2024-03-14T09:02:56.064 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlSRLFBxAAkIg2TMRbiwcdMggjl+vMWw==", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlQsV4OxAADoB6pH163AX2+5II4EtdFw==", 2024-03-14T09:02:56.065 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-03-14T09:02:56.066 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 msgr2=0x7fe9f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 0x7fe9f006ccf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007fd0 tx=0x7fe9f4007ec0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 msgr2=0x7fea040c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fea040cb0b0 tx=0x7fe9fc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 shutdown_connections 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe9f006a830 0x7fe9f006ccf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea04154d30 0x7fea040c0430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea040c7f70 0x7fea040bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 --2- 172.21.15.154:0/1834642829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea040c8350 0x7fea040bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 >> 172.21.15.154:0/1834642829 conn(0x7fea04072080 msgr2=0x7fea040aa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:56.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 shutdown_connections 2024-03-14T09:02:56.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.056+0000 7fea0a5a3700 1 -- 172.21.15.154:0/1834642829 wait complete. 2024-03-14T09:02:56.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.testuser 2024-03-14T09:02:56.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:02:56.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:02:56.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.198+0000 7f406cbf9700 1 Processor -- start 2024-03-14T09:02:56.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.198+0000 7f406cbf9700 1 -- start start 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406cbf9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40680a8540 0x7f406813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406cbf9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40680c1780 0x7f40680c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406cbf9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406813ade0 con 0x7f40680c3250 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406cbf9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406806fb30 con 0x7f40680c1780 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406cbf9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406806ec30 con 0x7f40680a8540 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40680c1780 0x7f40680c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4065d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f40680c3250 0x7f4068154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50452/0 (socket says 172.21.15.154:50452) 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4065d9b700 1 -- 172.21.15.154:0/1800626068 learned_addr learned my addr 172.21.15.154:0/1800626068 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406659c700 1 --2- 172.21.15.154:0/1800626068 >> v2:172.21.15.191:3300/0 conn(0x7f40680a8540 0x7f406813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 -- 172.21.15.154:0/1800626068 >> v2:172.21.15.191:3300/0 conn(0x7f40680a8540 msgr2=0x7f406813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 --2- 172.21.15.154:0/1800626068 >> v2:172.21.15.191:3300/0 conn(0x7f40680a8540 0x7f406813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 -- 172.21.15.154:0/1800626068 >> v1:172.21.15.154:6789/0 conn(0x7f40680c3250 legacy=0x7f4068154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406559a700 1 -- 172.21.15.154:0/1800626068 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578303284 0 0) 0x7f4058001110 con 0x7f40680c3250 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 -- 172.21.15.154:0/1800626068 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406806ef10 con 0x7f40680c1780 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f4066d9d700 1 --2- 172.21.15.154:0/1800626068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f40680c1c10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4068070da0 tx=0x7f406000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7792aca8a10a11 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406559a700 1 -- 172.21.15.154:0/1800626068 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000e070 con 0x7f40680c1780 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406559a700 1 -- 172.21.15.154:0/1800626068 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000bb20 con 0x7f40680c1780 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.199+0000 7f406559a700 1 -- 172.21.15.154:0/1800626068 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060012430 con 0x7f40680c1780 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 -- 172.21.15.154:0/1800626068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 msgr2=0x7f40680c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1800626068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f40680c1c10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4068070da0 tx=0x7f406000b330 comp rx=0 tx=0).stop 2024-03-14T09:02:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 -- 172.21.15.154:0/1800626068 shutdown_connections 2024-03-14T09:02:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1800626068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f40680c1c10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1800626068 >> v2:172.21.15.191:3300/0 conn(0x7f40680a8540 0x7f406813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 -- 172.21.15.154:0/1800626068 >> 172.21.15.154:0/1800626068 conn(0x7f40680c3880 msgr2=0x7f40680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 -- 172.21.15.154:0/1800626068 shutdown_connections 2024-03-14T09:02:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.200+0000 7f406cbf9700 1 -- 172.21.15.154:0/1800626068 wait complete. 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 Processor -- start 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 -- start start 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 0x7f4068153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40680c3250 0x7f4068152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680a9d20 con 0x7f40680c1780 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068133680 con 0x7f40680a8540 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.202+0000 7f406cbf9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4068133920 con 0x7f40680c3250 2024-03-14T09:02:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36580/0 (socket says 172.21.15.154:36580) 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 -- 172.21.15.154:0/1568272691 learned_addr learned my addr 172.21.15.154:0/1568272691 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f406659c700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 0x7f4068153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4066d9d700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40680c3250 0x7f4068152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 -- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40680c3250 msgr2=0x7f4068152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40680c3250 0x7f4068152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 -- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 msgr2=0x7f4068153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 0x7f4068153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 -- 172.21.15.154:0/1568272691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060009a70 con 0x7f40680c1780 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f406659c700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 0x7f4068153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4065d9b700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f405800c550 tx=0x7f405800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058004c80 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058004e00 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4068133c00 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058007310 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.203+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4068154790 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.204+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4068072140 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.204+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4058007470 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.205+0000 7f4056ffd700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 0x7f404806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.205+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4058088540 con 0x7f40680c1780 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.205+0000 7f406659c700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 0x7f404806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.207 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.206+0000 7f406659c700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 0x7f404806ccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f405c009ac0 tx=0x7f405c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.209+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4058053cb0 con 0x7f40680c1780 2024-03-14T09:02:56.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.485+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.testuser"} v 0) v1 -- 0x7f406813ade0 con 0x7f40680c1780 2024-03-14T09:02:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.488+0000 7f4056ffd700 1 -- 172.21.15.154:0/1568272691 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.testuser"}]=0 v18) v1 ==== 85+0+0 (secure 0 0 0) 0x7f406813ade0 con 0x7f40680c1780 2024-03-14T09:02:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 msgr2=0x7f404806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 0x7f404806ccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f405c009ac0 tx=0x7f405c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 msgr2=0x7f4068151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f405800c550 tx=0x7f405800c910 comp rx=0 tx=0).stop 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 shutdown_connections 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40680c3250 0x7f4068152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40680a8540 0x7f4068153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f404806a830 0x7f404806ccf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 --2- 172.21.15.154:0/1568272691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40680c1780 0x7f4068151bc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 >> 172.21.15.154:0/1568272691 conn(0x7f40680c3880 msgr2=0x7f406806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 shutdown_connections 2024-03-14T09:02:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.490+0000 7f406cbf9700 1 -- 172.21.15.154:0/1568272691 wait complete. 2024-03-14T09:02:56.502 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:02:56.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.664+0000 7f7c3a0a7700 1 Processor -- start 2024-03-14T09:02:56.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.664+0000 7f7c3a0a7700 1 -- start start 2024-03-14T09:02:56.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c3a0a7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7c340702f0 0x7f7c340c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c3a0a7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c340c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c3a0a7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c340acd10 con 0x7f7c3414fbd0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c3a0a7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c340ac7b0 con 0x7f7c340702f0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c3a0a7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34150050 con 0x7f7c340c5d60 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c32ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7c3414fbd0 0x7f7c340c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50454/0 (socket says 172.21.15.154:50454) 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c32ffd700 1 -- 172.21.15.154:0/1671907505 learned_addr learned my addr 172.21.15.154:0/1671907505 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c33fff700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c340c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c337fe700 1 --2- 172.21.15.154:0/1671907505 >> v2:172.21.15.191:3300/0 conn(0x7f7c340702f0 0x7f7c340c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.665+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922074592 0 0) 0x7f7c340acd10 con 0x7f7c3414fbd0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c1c0035b0 con 0x7f7c3414fbd0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 871325976 0 0) 0x7f7c1c0035b0 con 0x7f7c3414fbd0 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 msgr2=0x7f7c340c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:56.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c340c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 >> v2:172.21.15.191:3300/0 conn(0x7f7c340702f0 msgr2=0x7f7c340c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 --2- 172.21.15.154:0/1671907505 >> v2:172.21.15.191:3300/0 conn(0x7f7c340702f0 0x7f7c340c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c340ac9c0 con 0x7f7c3414fbd0 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f7c24001670 con 0x7f7c3414fbd0 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 >> v1:172.21.15.154:6789/0 conn(0x7f7c3414fbd0 legacy=0x7f7c340c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c1c003a00 0x7f7c1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c1c004850 0x7f7c1c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340ac7b0 con 0x7f7c1c004850 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34150050 con 0x7f7c1c003a00 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c00ba10 con 0x7f7c340c5d60 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c1c004850 0x7f7c1c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c337fe700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c1c003a00 0x7f7c1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c33fff700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 -- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 msgr2=0x7f7c1c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c340c5d60 0x7f7c1c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 -- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c1c003a00 msgr2=0x7f7c1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c1c003a00 0x7f7c1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 -- 172.21.15.154:0/1671907505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0035b0 con 0x7f7c1c004850 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.666+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c1c004850 0x7f7c1c004cd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c2f0 tx=0x7f7c1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3315eac8fbb6ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400ce30 con 0x7f7c1c004850 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c24005960 con 0x7f7c1c004850 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c327fc700 1 -- 172.21.15.154:0/1671907505 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24005cf0 con 0x7f7c1c004850 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/1671907505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c1c004850 msgr2=0x7f7c1c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/1671907505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c1c004850 0x7f7c1c004cd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c2f0 tx=0x7f7c1c005880 comp rx=0 tx=0).stop 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c33fff700 1 -- 172.21.15.154:0/1671907505 reap_dead start 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/1671907505 shutdown_connections 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/1671907505 >> 172.21.15.154:0/1671907505 conn(0x7f7c340a9bf0 msgr2=0x7f7c340a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/1671907505 shutdown_connections 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.667+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/1671907505 wait complete. 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.668+0000 7f7c3a0a7700 1 Processor -- start 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.668+0000 7f7c3a0a7700 1 -- start start 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004850 0x7f7c341e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 0x7f7c34155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34154120 con 0x7f7c34150460 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34154ec0 con 0x7f7c341559f0 2024-03-14T09:02:56.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c3a0a7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c341e8ae0 con 0x7f7c1c004850 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c32ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36592/0 (socket says 172.21.15.154:36592) 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c32ffd700 1 -- 172.21.15.154:0/246435060 learned_addr learned my addr 172.21.15.154:0/246435060 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c337fe700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004850 0x7f7c341e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.669+0000 7f7c33fff700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 0x7f7c34155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 -- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004850 msgr2=0x7f7c341e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004850 0x7f7c341e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 -- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 msgr2=0x7f7c34155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 0x7f7c34155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 -- 172.21.15.154:0/246435060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c341e8d80 con 0x7f7c34150460 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c33fff700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 0x7f7c34155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c32ffd700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005d80 tx=0x7f7c24000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400ce60 con 0x7f7c34150460 2024-03-14T09:02:56.675 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c24004a80 con 0x7f7c34150460 2024-03-14T09:02:56.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c34153a60 con 0x7f7c34150460 2024-03-14T09:02:56.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24004e30 con 0x7f7c34150460 2024-03-14T09:02:56.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.670+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c34156a30 con 0x7f7c34150460 2024-03-14T09:02:56.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.671+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c34154260 con 0x7f7c34150460 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.671+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7c240070d0 con 0x7f7c34150460 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.672+0000 7f7c1bfff700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 0x7f7c2006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.672+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c24089e40 con 0x7f7c34150460 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.672+0000 7f7c337fe700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 0x7f7c2006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.672+0000 7f7c337fe700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 0x7f7c2006ccf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009be0 tx=0x7f7c28009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:56.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.676+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7c24055640 con 0x7f7c34150460 2024-03-14T09:02:56.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.872+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c34074110 con 0x7f7c34150460 2024-03-14T09:02:56.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.873+0000 7f7c1bfff700 1 -- 172.21.15.154:0/246435060 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+12437 (secure 0 0 0) 0x7f7c24007dc0 con 0x7f7c34150460 2024-03-14T09:02:56.872 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:56.873 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:55.628037+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 msgr2=0x7f7c2006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 0x7f7c2006ccf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009be0 tx=0x7f7c28009210 comp rx=0 tx=0).stop 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 msgr2=0x7f7c341e8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c24005d80 tx=0x7f7c24000fe0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 shutdown_connections 2024-03-14T09:02:56.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c341559f0 0x7f7c34155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c1c004850 0x7f7c341e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7c2006a830 0x7f7c2006ccf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 --2- 172.21.15.154:0/246435060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c34150460 0x7f7c341e8400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 >> 172.21.15.154:0/246435060 conn(0x7f7c340a9bf0 msgr2=0x7f7c3414dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 shutdown_connections 2024-03-14T09:02:56.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:56.875+0000 7f7c3a0a7700 1 -- 172.21.15.154:0/246435060 wait complete. 2024-03-14T09:02:56.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:02:56.887 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:02:57.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 Processor -- start 2024-03-14T09:02:57.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 -- start start 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c14c5c0 0x7f338c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f338c072290 0x7f338c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f338c0c1fd0 0x7f338c0c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0c1bb0 con 0x7f338c0c1fd0 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0c6170 con 0x7f338c072290 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3392477700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c070e10 con 0x7f338c14c5c0 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.012+0000 7f3390a14700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f338c0c1fd0 0x7f338c0c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f338c0c1fd0 0x7f338c0c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36606/0 (socket says 172.21.15.154:36606) 2024-03-14T09:02:57.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 -- 172.21.15.154:0/733866855 learned_addr learned my addr 172.21.15.154:0/733866855 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f338bfff700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c14c5c0 0x7f338c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f338b7fe700 1 --2- 172.21.15.154:0/733866855 >> v2:172.21.15.191:3300/0 conn(0x7f338c072290 0x7f338c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 -- 172.21.15.154:0/733866855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c14c5c0 msgr2=0x7f338c152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c14c5c0 0x7f338c152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 -- 172.21.15.154:0/733866855 >> v2:172.21.15.191:3300/0 conn(0x7f338c072290 msgr2=0x7f338c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 --2- 172.21.15.154:0/733866855 >> v2:172.21.15.191:3300/0 conn(0x7f338c072290 0x7f338c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 -- 172.21.15.154:0/733866855 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c14c9a0 con 0x7f338c0c1fd0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f3390a14700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c0c1fd0 0x7f338c0c2460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f338c06f440 tx=0x7f338400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b262b13a113aacef server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f338affd700 1 -- 172.21.15.154:0/733866855 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338400e070 con 0x7f338c0c1fd0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f338affd700 1 -- 172.21.15.154:0/733866855 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338400b9c0 con 0x7f338c0c1fd0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.013+0000 7f338affd700 1 -- 172.21.15.154:0/733866855 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338400bd70 con 0x7f338c0c1fd0 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 -- 172.21.15.154:0/733866855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c0c1fd0 msgr2=0x7f338c0c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c0c1fd0 0x7f338c0c2460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f338c06f440 tx=0x7f338400b330 comp rx=0 tx=0).stop 2024-03-14T09:02:57.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 -- 172.21.15.154:0/733866855 shutdown_connections 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c0c1fd0 0x7f338c0c2460 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 --2- 172.21.15.154:0/733866855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c14c5c0 0x7f338c152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 --2- 172.21.15.154:0/733866855 >> v2:172.21.15.191:3300/0 conn(0x7f338c072290 0x7f338c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 -- 172.21.15.154:0/733866855 >> 172.21.15.154:0/733866855 conn(0x7f338c06a770 msgr2=0x7f338c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 -- 172.21.15.154:0/733866855 shutdown_connections 2024-03-14T09:02:57.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.014+0000 7f3392477700 1 -- 172.21.15.154:0/733866855 wait complete. 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.015+0000 7f3392477700 1 Processor -- start 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.015+0000 7f3392477700 1 -- start start 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c0c1fd0 0x7f338c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c14c5c0 0x7f338c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c14e090 con 0x7f338c072290 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1eb210 con 0x7f338c14c5c0 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1eb4f0 con 0x7f338c0c1fd0 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36620/0 (socket says 172.21.15.154:36620) 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 -- 172.21.15.154:0/2477583604 learned_addr learned my addr 172.21.15.154:0/2477583604 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338b7fe700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c0c1fd0 0x7f338c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3390a14700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c14c5c0 0x7f338c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 -- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c0c1fd0 msgr2=0x7f338c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c0c1fd0 0x7f338c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 -- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c14c5c0 msgr2=0x7f338c14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c14c5c0 0x7f338c14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 -- 172.21.15.154:0/2477583604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009b30 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f338bfff700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f337400d800 tx=0x7f337400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33740125f0 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3374012750 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c1eb7d0 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.016+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374012b00 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.017+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c14ce90 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.017+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c1eb070 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.018+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3374012c60 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.018+0000 7f3388ff9700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 0x7f336c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.018+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3374088990 con 0x7f338c072290 2024-03-14T09:02:57.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.019+0000 7f338b7fe700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 0x7f336c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.019+0000 7f338b7fe700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 0x7f336c06ccf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f338c14e490 tx=0x7f337c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.023+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f338c1eb070 con 0x7f338c072290 2024-03-14T09:02:57.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.213+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f338c1eb070 con 0x7f338c072290 2024-03-14T09:02:57.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.213+0000 7f3388ff9700 1 -- 172.21.15.154:0/2477583604 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f338c1eb070 con 0x7f338c072290 2024-03-14T09:02:57.213 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:02:57.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.215+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 msgr2=0x7f336c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.215+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 0x7f336c06ccf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f338c14e490 tx=0x7f337c008040 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 msgr2=0x7f338c1505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f337400d800 tx=0x7f337400db10 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 shutdown_connections 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f338c14c5c0 0x7f338c14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f336c06a830 0x7f336c06ccf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f338c072290 0x7f338c1505e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 --2- 172.21.15.154:0/2477583604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f338c0c1fd0 0x7f338c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 >> 172.21.15.154:0/2477583604 conn(0x7f338c06a770 msgr2=0x7f338c155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 shutdown_connections 2024-03-14T09:02:57.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.216+0000 7f3392477700 1 -- 172.21.15.154:0/2477583604 wait complete. 2024-03-14T09:02:57.228 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:02:57.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.349+0000 7f0465c83700 1 Processor -- start 2024-03-14T09:02:57.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.349+0000 7f0465c83700 1 -- start start 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.349+0000 7f0465c83700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04601553d0 0x7f0460158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.349+0000 7f0465c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f04600bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f0465c83700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0460154f40 con 0x7f04600c1d50 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f0465c83700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04600c16a0 con 0x7f04601553d0 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f0465c83700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0460158d80 con 0x7f04600bf9c0 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045effd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f04600c1d50 0x7f04600c0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50464/0 (socket says 172.21.15.154:50464) 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045effd700 1 -- 172.21.15.154:0/1507445044 learned_addr learned my addr 172.21.15.154:0/1507445044 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436808831 0 0) 0x7f0460154f40 con 0x7f04600c1d50 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045f7fe700 1 --2- 172.21.15.154:0/1507445044 >> v2:172.21.15.191:3300/0 conn(0x7f04601553d0 0x7f0460158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04480035b0 con 0x7f04600c1d50 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045ffff700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f04600bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3407062284 0 0) 0x7f04480035b0 con 0x7f04600c1d50 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 msgr2=0x7f04600bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f04600bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 >> v2:172.21.15.191:3300/0 conn(0x7f04601553d0 msgr2=0x7f0460158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 --2- 172.21.15.154:0/1507445044 >> v2:172.21.15.191:3300/0 conn(0x7f04601553d0 0x7f0460158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460154930 con 0x7f04600c1d50 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0450001670 con 0x7f04600c1d50 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 >> v1:172.21.15.154:6789/0 conn(0x7f04600c1d50 legacy=0x7f04600c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448003a00 0x7f0448003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448004850 0x7f0448004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f0448005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600c16a0 con 0x7f0448004850 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460158d80 con 0x7f0448003a00 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044800ba10 con 0x7f04600bf9c0 2024-03-14T09:02:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045effd700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448004850 0x7f0448004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.350+0000 7f045f7fe700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448003a00 0x7f0448003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045ffff700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f0448005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 -- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 msgr2=0x7f0448005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f0448005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 -- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448003a00 msgr2=0x7f0448003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448003a00 0x7f0448003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 -- 172.21.15.154:0/1507445044 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04480035b0 con 0x7f0448004850 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045ffff700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f04600bf9c0 0x7f0448005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045effd700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448004850 0x7f0448004cd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f045000c2f0 tx=0x7f0448005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f33ee42781904b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450007c40 con 0x7f0448004850 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045000ce30 con 0x7f0448004850 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045e7fc700 1 -- 172.21.15.154:0/1507445044 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450005ba0 con 0x7f0448004850 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f0465c83700 1 -- 172.21.15.154:0/1507445044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448004850 msgr2=0x7f0448004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f0465c83700 1 --2- 172.21.15.154:0/1507445044 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448004850 0x7f0448004cd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f045000c2f0 tx=0x7f0448005880 comp rx=0 tx=0).stop 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f045ffff700 1 -- 172.21.15.154:0/1507445044 reap_dead start 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f0465c83700 1 -- 172.21.15.154:0/1507445044 shutdown_connections 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.351+0000 7f0465c83700 1 -- 172.21.15.154:0/1507445044 >> 172.21.15.154:0/1507445044 conn(0x7f046006a770 msgr2=0x7f046006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.352+0000 7f0465c83700 1 -- 172.21.15.154:0/1507445044 shutdown_connections 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.352+0000 7f0465c83700 1 -- 172.21.15.154:0/1507445044 wait complete. 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.353+0000 7f0465c83700 1 Processor -- start 2024-03-14T09:02:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.353+0000 7f0465c83700 1 -- start start 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448004850 0x7f0460156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f046014f2d0 0x7f04601ed510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046014d270 con 0x7f046014dfd0 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600abe10 con 0x7f0448004850 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046014f780 con 0x7f046014f2d0 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36638/0 (socket says 172.21.15.154:36638) 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 -- 172.21.15.154:0/2974048886 learned_addr learned my addr 172.21.15.154:0/2974048886 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 -- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f046014f2d0 msgr2=0x7f04601ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f046014f2d0 0x7f04601ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 -- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448004850 msgr2=0x7f0460156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448004850 0x7f0460156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 -- 172.21.15.154:0/2974048886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04600ac300 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f045effd700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0450008fd0 tx=0x7f04600c07d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500070d0 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450007250 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046014cf90 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450007600 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.354+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046014e870 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.355+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046014e6b0 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.356+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f045001e430 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.356+0000 7f0447fff700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 0x7f044c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.356+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0450089d60 con 0x7f046014dfd0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.356+0000 7f045f7fe700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 0x7f044c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.357+0000 7f045f7fe700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 0x7f044c06ccf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0454006ba0 tx=0x7f0454006b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.361+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0450055620 con 0x7f046014dfd0 2024-03-14T09:02:57.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.558+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f046014ea00 con 0x7f046014dfd0 2024-03-14T09:02:57.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.558+0000 7f0447fff700 1 -- 172.21.15.154:0/2974048886 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0450058c40 con 0x7f046014dfd0 2024-03-14T09:02:57.558 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:02:57.558 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:02:57.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:02:57.559 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:02:57.559 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:02:57.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:02:57.559 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:02:57.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 msgr2=0x7f044c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 0x7f044c06ccf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0454006ba0 tx=0x7f0454006b30 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 msgr2=0x7f046014ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0450008fd0 tx=0x7f04600c07d0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 shutdown_connections 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f046014f2d0 0x7f04601ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0448004850 0x7f0460156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f044c06a830 0x7f044c06ccf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 --2- 172.21.15.154:0/2974048886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f046014dfd0 0x7f046014ed90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 >> 172.21.15.154:0/2974048886 conn(0x7f046006a770 msgr2=0x7f04600725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 shutdown_connections 2024-03-14T09:02:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.560+0000 7f0465c83700 1 -- 172.21.15.154:0/2974048886 wait complete. 2024-03-14T09:02:57.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T09:02:57.572 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T09:02:57.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.697+0000 7f9390570700 1 Processor -- start 2024-03-14T09:02:57.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 -- start start 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f938806d050 0x7f9388158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 0x7f93880c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9388158c90 con 0x7f93881554e0 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938806b110 con 0x7f938806d050 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.698+0000 7f9390570700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938806ab50 con 0x7f9388155060 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f938806d050 0x7f9388158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f938806d050 0x7f9388158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36642/0 (socket says 172.21.15.154:36642) 2024-03-14T09:02:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 -- 172.21.15.154:0/4116049259 learned_addr learned my addr 172.21.15.154:0/4116049259 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938db0b700 1 --2- 172.21.15.154:0/4116049259 >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 0x7f93880c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 -- 172.21.15.154:0/4116049259 >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 msgr2=0x7f93880c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 --2- 172.21.15.154:0/4116049259 >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 0x7f93880c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 -- 172.21.15.154:0/4116049259 >> v1:172.21.15.191:6789/0 conn(0x7f93881554e0 legacy=0x7f93880c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 -- 172.21.15.154:0/4116049259 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938806ae30 con 0x7f938806d050 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938db0b700 1 --2- 172.21.15.154:0/4116049259 >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 0x7f93880c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938e30c700 1 --2- 172.21.15.154:0/4116049259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938806d050 0x7f9388158750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93880723f0 tx=0x7f9378009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7e4124a3d7e23f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938d30a700 1 -- 172.21.15.154:0/4116049259 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378004230 con 0x7f938806d050 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938d30a700 1 -- 172.21.15.154:0/4116049259 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378004390 con 0x7f938806d050 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.699+0000 7f938d30a700 1 -- 172.21.15.154:0/4116049259 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93780036a0 con 0x7f938806d050 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 -- 172.21.15.154:0/4116049259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938806d050 msgr2=0x7f9388158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 --2- 172.21.15.154:0/4116049259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938806d050 0x7f9388158750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93880723f0 tx=0x7f9378009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 -- 172.21.15.154:0/4116049259 shutdown_connections 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 --2- 172.21.15.154:0/4116049259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f938806d050 0x7f9388158750 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 --2- 172.21.15.154:0/4116049259 >> v2:172.21.15.191:3300/0 conn(0x7f9388155060 0x7f93880c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 -- 172.21.15.154:0/4116049259 >> 172.21.15.154:0/4116049259 conn(0x7f93880c39f0 msgr2=0x7f93880c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:57.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 -- 172.21.15.154:0/4116049259 shutdown_connections 2024-03-14T09:02:57.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.700+0000 7f9390570700 1 -- 172.21.15.154:0/4116049259 wait complete. 2024-03-14T09:02:57.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.701+0000 7f9390570700 1 Processor -- start 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.701+0000 7f9390570700 1 -- start start 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f938806d050 0x7f93881ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9388155060 0x7f9388156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388147e80 con 0x7f93881554e0 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388155c90 con 0x7f9388155060 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f9390570700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9388148080 con 0x7f938806d050 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36654/0 (socket says 172.21.15.154:36654) 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 -- 172.21.15.154:0/2540994864 learned_addr learned my addr 172.21.15.154:0/2540994864 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938db0b700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9388155060 0x7f9388156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938e30c700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f938806d050 0x7f93881ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 -- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f938806d050 msgr2=0x7f93881ed9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f938806d050 0x7f93881ed9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 -- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9388155060 msgr2=0x7f9388156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9388155060 0x7f9388156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 -- 172.21.15.154:0/2540994864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378009550 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.702+0000 7f938eb0d700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f938000b9e0 tx=0x7f938000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380003c00 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9380003da0 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9388155960 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000d920 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881ee5f0 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.703+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93880697d0 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.704+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f938000da80 con 0x7f93881554e0 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.704+0000 7f937effd700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 0x7f936c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.704+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9380011070 con 0x7f93881554e0 2024-03-14T09:02:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.705+0000 7f938e30c700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 0x7f936c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.705+0000 7f938e30c700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 0x7f936c06ccf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9378009130 tx=0x7f9378008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.709+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9380054010 con 0x7f93881554e0 2024-03-14T09:02:57.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:57.901+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f938814e1a0 con 0x7f93881554e0 2024-03-14T09:02:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.556+0000 7f937effd700 1 -- 172.21.15.154:0/2540994864 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9380057630 con 0x7f93881554e0 2024-03-14T09:02:58.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.561+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 msgr2=0x7f936c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:58.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.561+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 0x7f936c06ccf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9378009130 tx=0x7f9378008030 comp rx=0 tx=0).stop 2024-03-14T09:02:58.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.561+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 msgr2=0x7f9388156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:58.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.561+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f938000b9e0 tx=0x7f938000bcf0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.562 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 shutdown_connections 2024-03-14T09:02:58.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f936c06a830 0x7f936c06ccf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93881554e0 0x7f9388156660 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f938806d050 0x7f93881ed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 --2- 172.21.15.154:0/2540994864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9388155060 0x7f9388156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 >> 172.21.15.154:0/2540994864 conn(0x7f93880c39f0 msgr2=0x7f9388071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:58.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.562+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 shutdown_connections 2024-03-14T09:02:58.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.563+0000 7f9390570700 1 -- 172.21.15.154:0/2540994864 wait complete. 2024-03-14T09:02:58.564 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2024-03-14T09:02:58.577 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:02:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.710+0000 7ffa74c20700 1 Processor -- start 2024-03-14T09:02:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.710+0000 7ffa74c20700 1 -- start start 2024-03-14T09:02:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa74c20700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffa7014eaf0 0x7ffa7006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa74c20700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffa7006acb0 0x7ffa70155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa74c20700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700ac8e0 con 0x7ffa70156330 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa74c20700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006b310 con 0x7ffa7014eaf0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa74c20700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006de30 con 0x7ffa7006acb0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffa7014eaf0 0x7ffa7006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffa7014eaf0 0x7ffa7006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36668/0 (socket says 172.21.15.154:36668) 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 -- 172.21.15.154:0/2968250610 learned_addr learned my addr 172.21.15.154:0/2968250610 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6dd9b700 1 --2- 172.21.15.154:0/2968250610 >> v2:172.21.15.191:3300/0 conn(0x7ffa7006acb0 0x7ffa70155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 -- 172.21.15.154:0/2968250610 >> v2:172.21.15.191:3300/0 conn(0x7ffa7006acb0 msgr2=0x7ffa70155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 --2- 172.21.15.154:0/2968250610 >> v2:172.21.15.191:3300/0 conn(0x7ffa7006acb0 0x7ffa70155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 -- 172.21.15.154:0/2968250610 >> v1:172.21.15.191:6789/0 conn(0x7ffa70156330 legacy=0x7ffa70157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 -- 172.21.15.154:0/2968250610 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700acb30 con 0x7ffa7014eaf0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6e59c700 1 --2- 172.21.15.154:0/2968250610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa7006a770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa7014f1c0 tx=0x7ffa60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6456449426093df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6d59a700 1 -- 172.21.15.154:0/2968250610 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004230 con 0x7ffa7014eaf0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6d59a700 1 -- 172.21.15.154:0/2968250610 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60004390 con 0x7ffa7014eaf0 2024-03-14T09:02:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.711+0000 7ffa6d59a700 1 -- 172.21.15.154:0/2968250610 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600036a0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 -- 172.21.15.154:0/2968250610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 msgr2=0x7ffa7006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 --2- 172.21.15.154:0/2968250610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa7006a770 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa7014f1c0 tx=0x7ffa60009bd0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 -- 172.21.15.154:0/2968250610 shutdown_connections 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 --2- 172.21.15.154:0/2968250610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa7006a770 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 --2- 172.21.15.154:0/2968250610 >> v2:172.21.15.191:3300/0 conn(0x7ffa7006acb0 0x7ffa70155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 -- 172.21.15.154:0/2968250610 >> 172.21.15.154:0/2968250610 conn(0x7ffa700c3880 msgr2=0x7ffa700c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 -- 172.21.15.154:0/2968250610 shutdown_connections 2024-03-14T09:02:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.712+0000 7ffa74c20700 1 -- 172.21.15.154:0/2968250610 wait complete. 2024-03-14T09:02:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.713+0000 7ffa74c20700 1 Processor -- start 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.713+0000 7ffa74c20700 1 -- start start 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa7006acb0 0x7ffa701e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa70156330 0x7ffa701516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7014fb00 con 0x7ffa7014eaf0 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700c2340 con 0x7ffa70156330 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa74c20700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700c1470 con 0x7ffa7006acb0 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa6dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa6dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36684/0 (socket says 172.21.15.154:36684) 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa6dd9b700 1 -- 172.21.15.154:0/982237653 learned_addr learned my addr 172.21.15.154:0/982237653 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa6e59c700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa7006acb0 0x7ffa701e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.714+0000 7ffa6ed9d700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa70156330 0x7ffa701516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 -- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa7006acb0 msgr2=0x7ffa701e13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa7006acb0 0x7ffa701e13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 -- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa70156330 msgr2=0x7ffa701516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa70156330 0x7ffa701516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 -- 172.21.15.154:0/982237653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009550 con 0x7ffa7014eaf0 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa6dd9b700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c7f0 tx=0x7ffa6400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011800 con 0x7ffa7014eaf0 2024-03-14T09:02:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64011960 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa70151be0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011cd0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.715+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700c19a0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.716+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa700697d0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.716+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffa64004020 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.717+0000 7ffa5effd700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 0x7ffa5006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.717+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ffa640885b0 con 0x7ffa7014eaf0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.717+0000 7ffa6e59c700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 0x7ffa5006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.717+0000 7ffa6e59c700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 0x7ffa5006ccf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009130 tx=0x7ffa60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:58.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.722+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffa64053c40 con 0x7ffa7014eaf0 2024-03-14T09:02:58.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:58.915+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ffa7006b470 con 0x7ffa7014eaf0 2024-03-14T09:02:59.563 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.563+0000 7ffa5effd700 1 -- 172.21.15.154:0/982237653 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7ffa64057260 con 0x7ffa7014eaf0 2024-03-14T09:02:59.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 msgr2=0x7ffa5006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 0x7ffa5006ccf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009130 tx=0x7ffa60008030 comp rx=0 tx=0).stop 2024-03-14T09:02:59.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 msgr2=0x7ffa70151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c7f0 tx=0x7ffa6400cb00 comp rx=0 tx=0).stop 2024-03-14T09:02:59.569 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 shutdown_connections 2024-03-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa70156330 0x7ffa701516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa7006acb0 0x7ffa701e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa5006a830 0x7ffa5006ccf0 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009130 tx=0x7ffa60008030 comp rx=0 tx=0).stop 2024-03-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 --2- 172.21.15.154:0/982237653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa7014eaf0 0x7ffa70151160 secure :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c7f0 tx=0x7ffa6400cb00 comp rx=0 tx=0).stop 2024-03-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.569+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 >> 172.21.15.154:0/982237653 conn(0x7ffa700c3880 msgr2=0x7ffa701572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:59.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.572+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 shutdown_connections 2024-03-14T09:02:59.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.573+0000 7ffa74c20700 1 -- 172.21.15.154:0/982237653 wait complete. 2024-03-14T09:02:59.579 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2024-03-14T09:02:59.595 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T09:02:59.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.724+0000 7f98702fc700 1 Processor -- start 2024-03-14T09:02:59.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.724+0000 7f98702fc700 1 -- start start 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f98702fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f986813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f98702fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98680c1780 0x7f98680c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f98702fc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986813ade0 con 0x7f98680c3250 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f98702fc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986806fb30 con 0x7f98680c1780 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f98702fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986806ec30 con 0x7f98680a8540 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98680c1780 0x7f98680c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98680c1780 0x7f98680c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39732/0 (socket says 172.21.15.154:39732) 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 -- 172.21.15.154:0/2784674634 learned_addr learned my addr 172.21.15.154:0/2784674634 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e098700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f986813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 -- 172.21.15.154:0/2784674634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 msgr2=0x7f986813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f986813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 -- 172.21.15.154:0/2784674634 >> v1:172.21.15.191:6789/0 conn(0x7f98680c3250 legacy=0x7f9868154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 -- 172.21.15.154:0/2784674634 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986806ef10 con 0x7f98680c1780 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986e899700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98680c1c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f986000d150 tx=0x7f986000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b76ce765c056ae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.725+0000 7f986d096700 1 -- 172.21.15.154:0/2784674634 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860010070 con 0x7f98680c1780 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f986d096700 1 -- 172.21.15.154:0/2784674634 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986000dd10 con 0x7f98680c1780 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f986d096700 1 -- 172.21.15.154:0/2784674634 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000b310 con 0x7f98680c1780 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 -- 172.21.15.154:0/2784674634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 msgr2=0x7f98680c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98680c1c10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f986000d150 tx=0x7f986000d460 comp rx=0 tx=0).stop 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 -- 172.21.15.154:0/2784674634 shutdown_connections 2024-03-14T09:02:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f986813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 --2- 172.21.15.154:0/2784674634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98680c1c10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 -- 172.21.15.154:0/2784674634 >> 172.21.15.154:0/2784674634 conn(0x7f98680c3880 msgr2=0x7f98680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 -- 172.21.15.154:0/2784674634 shutdown_connections 2024-03-14T09:02:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.726+0000 7f98702fc700 1 -- 172.21.15.154:0/2784674634 wait complete. 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.727+0000 7f98702fc700 1 Processor -- start 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.727+0000 7f98702fc700 1 -- start start 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f98680ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98680c3250 0x7f9868133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868134370 con 0x7f98680c1780 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680abd70 con 0x7f98680c3250 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f98702fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680ab440 con 0x7f98680a8540 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39736/0 (socket says 172.21.15.154:39736) 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 -- 172.21.15.154:0/3646054423 learned_addr learned my addr 172.21.15.154:0/3646054423 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986e899700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98680c3250 0x7f9868133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986e098700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f98680ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 -- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 msgr2=0x7f98680ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f98680ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 -- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98680c3250 msgr2=0x7f9868133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98680c3250 0x7f9868133e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.728+0000 7f986d897700 1 -- 172.21.15.154:0/3646054423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986000bd40 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f986d897700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f985800c430 tx=0x7f985800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580050d0 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858005250 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98680ab720 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858006860 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.729+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98680ac9d0 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.730+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868072c60 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.730+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9858003680 con 0x7f98680c1780 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.731+0000 7f985effd700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 0x7f984c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:02:59.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.731+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9858087c70 con 0x7f98680c1780 2024-03-14T09:02:59.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.731+0000 7f986e098700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 0x7f984c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:02:59.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.731+0000 7f986e098700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 0x7f984c06ccf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9864009b40 tx=0x7f9864009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:02:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.735+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f98580530f0 con 0x7f98680c1780 2024-03-14T09:02:59.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.970+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f986813ade0 con 0x7f98680c1780 2024-03-14T09:02:59.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.978+0000 7f985effd700 1 -- 172.21.15.154:0/3646054423 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7f986813ade0 con 0x7f98680c1780 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 msgr2=0x7f984c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 0x7f984c06ccf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9864009b40 tx=0x7f9864009210 comp rx=0 tx=0).stop 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 msgr2=0x7f98681338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f985800c430 tx=0x7f985800c740 comp rx=0 tx=0).stop 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 shutdown_connections 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98680c3250 0x7f9868133e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98680a8540 0x7f98680ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f984c06a830 0x7f984c06ccf0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9864009b40 tx=0x7f9864009210 comp rx=0 tx=0).stop 2024-03-14T09:02:59.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 --2- 172.21.15.154:0/3646054423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98680c1780 0x7f98681338f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:02:59.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.983+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 >> 172.21.15.154:0/3646054423 conn(0x7f98680c3880 msgr2=0x7f98680c63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:02:59.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.984+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 shutdown_connections 2024-03-14T09:02:59.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:02:59.984+0000 7f98702fc700 1 -- 172.21.15.154:0/3646054423 wait complete. 2024-03-14T09:02:59.987 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 8 and data pool 9 2024-03-14T09:03:00.005 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.127+0000 7f536ef1d700 1 Processor -- start 2024-03-14T09:03:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.127+0000 7f536ef1d700 1 -- start start 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536ef1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 0x7f5368156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536ef1d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53680ac8e0 con 0x7f536815d270 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536ef1d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536806de30 con 0x7f536814eaf0 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536ef1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536806b310 con 0x7f5368155df0 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536d4ba700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f536815d270 0x7f536815f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55070/0 (socket says 172.21.15.154:55070) 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f536d4ba700 1 -- 172.21.15.154:0/1283980533 learned_addr learned my addr 172.21.15.154:0/1283980533 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.128+0000 7f5367fff700 1 --2- 172.21.15.154:0/1283980533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 0x7f5368156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131015080 0 0) 0x7f53680ac8e0 con 0x7f536815d270 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53580035b0 con 0x7f536815d270 2024-03-14T09:03:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097752123 0 0) 0x7f536806de30 con 0x7f536814eaf0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53680ac8e0 con 0x7f536814eaf0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f5367fff700 1 -- 172.21.15.154:0/1283980533 >> v1:172.21.15.191:6789/0 conn(0x7f536814eaf0 legacy=0x7f536806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f5367fff700 1 -- 172.21.15.154:0/1283980533 >> v1:172.21.15.154:6789/0 conn(0x7f536815d270 legacy=0x7f536815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f5367fff700 1 -- 172.21.15.154:0/1283980533 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53680acb30 con 0x7f5368155df0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f5367fff700 1 --2- 172.21.15.154:0/1283980533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 0x7f5368156250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5354009870 tx=0x7f5354009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60d1d3135be4146 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354011830 con 0x7f5368155df0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354011990 con 0x7f5368155df0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.129+0000 7f53677fe700 1 -- 172.21.15.154:0/1283980533 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354011d20 con 0x7f5368155df0 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 -- 172.21.15.154:0/1283980533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 msgr2=0x7f5368156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 --2- 172.21.15.154:0/1283980533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 0x7f5368156250 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5354009870 tx=0x7f5354009b80 comp rx=0 tx=0).stop 2024-03-14T09:03:00.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 -- 172.21.15.154:0/1283980533 shutdown_connections 2024-03-14T09:03:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 --2- 172.21.15.154:0/1283980533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5368155df0 0x7f5368156250 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 -- 172.21.15.154:0/1283980533 >> 172.21.15.154:0/1283980533 conn(0x7f53680c3880 msgr2=0x7f53680c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 -- 172.21.15.154:0/1283980533 shutdown_connections 2024-03-14T09:03:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.130+0000 7f536ef1d700 1 -- 172.21.15.154:0/1283980533 wait complete. 2024-03-14T09:03:00.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.131+0000 7f536ef1d700 1 Processor -- start 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.131+0000 7f536ef1d700 1 -- start start 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368155df0 0x7f536814de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f536815d270 0x7f53680c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536814d340 con 0x7f536814eaf0 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536814e3b0 con 0x7f5368155df0 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ef1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536814c970 con 0x7f536815d270 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39742/0 (socket says 172.21.15.154:39742) 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 -- 172.21.15.154:0/3283862415 learned_addr learned my addr 172.21.15.154:0/3283862415 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f5367fff700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368155df0 0x7f536814de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536d4ba700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f536815d270 0x7f53680c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 -- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f536815d270 msgr2=0x7f53680c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f536815d270 0x7f53680c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 -- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368155df0 msgr2=0x7f536814de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368155df0 0x7f536814de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 -- 172.21.15.154:0/3283862415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540094a0 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.132+0000 7f536ccb9700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c500 tx=0x7f535c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c0052d0 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c005450 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536814aa00 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c0038c0 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536814c690 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.133+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53681497d0 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.134+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f535c003a20 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.134+0000 7f53657fa700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 0x7f534c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.134+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f535c088180 con 0x7f536814eaf0 2024-03-14T09:03:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.135+0000 7f5367fff700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 0x7f534c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.135+0000 7f5367fff700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 0x7f534c06ccf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53540097c0 tx=0x7f535401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.139+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f535c053510 con 0x7f536814eaf0 2024-03-14T09:03:00.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.338+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53680697d0 con 0x7f536814eaf0 2024-03-14T09:03:00.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.339+0000 7f53657fa700 1 -- 172.21.15.154:0/3283862415 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f535c056b30 con 0x7f536814eaf0 2024-03-14T09:03:00.339 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:00.339 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:59.971531+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:00.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 msgr2=0x7f534c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 0x7f534c06ccf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53540097c0 tx=0x7f535401c040 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 msgr2=0x7f5368149af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c500 tx=0x7f535c00c810 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 shutdown_connections 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f536815d270 0x7f53680c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f534c06a830 0x7f534c06ccf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536814eaf0 0x7f5368149af0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 --2- 172.21.15.154:0/3283862415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368155df0 0x7f536814de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 >> 172.21.15.154:0/3283862415 conn(0x7f53680c3880 msgr2=0x7f5368157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:00.342 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 shutdown_connections 2024-03-14T09:03:00.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.341+0000 7f536ef1d700 1 -- 172.21.15.154:0/3283862415 wait complete. 2024-03-14T09:03:00.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:00.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.474+0000 7f491a71e700 1 Processor -- start 2024-03-14T09:03:00.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.474+0000 7f491a71e700 1 -- start start 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f491a71e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f4914070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f491a71e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4914155da0 0x7f4914156230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f491a71e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49140c3ed0 con 0x7f4914153100 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f491a71e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49140c48e0 con 0x7f4914155da0 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f491a71e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49140ae0c0 con 0x7f49140703b0 2024-03-14T09:03:00.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4914155da0 0x7f4914156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4914155da0 0x7f4914156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39748/0 (socket says 172.21.15.154:39748) 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4913fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4914153100 0x7f491414fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55084/0 (socket says 172.21.15.154:55084) 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 -- 172.21.15.154:0/3542780674 learned_addr learned my addr 172.21.15.154:0/3542780674 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f49137fe700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f4914070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 -- 172.21.15.154:0/3542780674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 msgr2=0x7f4914070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f4914070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 -- 172.21.15.154:0/3542780674 >> v1:172.21.15.154:6789/0 conn(0x7f4914153100 legacy=0x7f491414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4918cbb700 1 -- 172.21.15.154:0/3542780674 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49141534e0 con 0x7f4914155da0 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f4912ffd700 1 -- 172.21.15.154:0/3542780674 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710883652 0 0) 0x7f49140c3ed0 con 0x7f4914153100 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.475+0000 7f49137fe700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f4914070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f4918cbb700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914155da0 0x7f4914156230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f509c5c0d514c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f4912ffd700 1 -- 172.21.15.154:0/3542780674 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00e070 con 0x7f4914155da0 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f4912ffd700 1 -- 172.21.15.154:0/3542780674 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c00b9c0 con 0x7f4914155da0 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f4912ffd700 1 -- 172.21.15.154:0/3542780674 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00bd70 con 0x7f4914155da0 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 -- 172.21.15.154:0/3542780674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914155da0 msgr2=0x7f4914156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914155da0 0x7f4914156230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f490c009e50 tx=0x7f490c00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 -- 172.21.15.154:0/3542780674 shutdown_connections 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914155da0 0x7f4914156230 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 --2- 172.21.15.154:0/3542780674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f4914070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 -- 172.21.15.154:0/3542780674 >> 172.21.15.154:0/3542780674 conn(0x7f49140a9d00 msgr2=0x7f49140aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 -- 172.21.15.154:0/3542780674 shutdown_connections 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.476+0000 7f491a71e700 1 -- 172.21.15.154:0/3542780674 wait complete. 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.477+0000 7f491a71e700 1 Processor -- start 2024-03-14T09:03:00.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 -- start start 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f49141ea160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 0x7f49141ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141ea6a0 con 0x7f4914153100 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491409b220 con 0x7f4914155da0 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f491a71e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4914099a70 con 0x7f49140703b0 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f49137fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f49137fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39752/0 (socket says 172.21.15.154:39752) 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f49137fe700 1 -- 172.21.15.154:0/3338095457 learned_addr learned my addr 172.21.15.154:0/3338095457 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f4918cbb700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 0x7f49141ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.478+0000 7f4913fff700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f49141ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 -- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 msgr2=0x7f49141ea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f49141ea160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 -- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 msgr2=0x7f49141ecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 0x7f49141ecf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 -- 172.21.15.154:0/3338095457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c009b30 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f4918cbb700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 0x7f49141ecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f49137fe700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f490400c750 tx=0x7f490400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011800 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4904011960 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49140c19e0 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904011cd0 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.479+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4914099d70 con 0x7f4914153100 2024-03-14T09:03:00.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.480+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49141ecc70 con 0x7f4914153100 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.480+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4904021350 con 0x7f4914153100 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.481+0000 7f4910ff9700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 0x7f490006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.481+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4904088c50 con 0x7f4914153100 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.481+0000 7f4913fff700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 0x7f490006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.481+0000 7f4913fff700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 0x7f490006ccf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4908007fd0 tx=0x7f4908007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.485+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f49141ecc70 con 0x7f4914153100 2024-03-14T09:03:00.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.684+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49141ecc70 con 0x7f4914153100 2024-03-14T09:03:00.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.685+0000 7f4910ff9700 1 -- 172.21.15.154:0/3338095457 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f49141ecc70 con 0x7f4914153100 2024-03-14T09:03:00.685 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:00.685 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:02:59.971531+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 msgr2=0x7f490006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 0x7f490006ccf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4908007fd0 tx=0x7f4908007de0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 msgr2=0x7f49140acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f490400c750 tx=0x7f490400ca60 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 shutdown_connections 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4914155da0 0x7f49141ecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49140703b0 0x7f49141ea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f490006a830 0x7f490006ccf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 --2- 172.21.15.154:0/3338095457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4914153100 0x7f49140acfb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 >> 172.21.15.154:0/3338095457 conn(0x7f49140a9d00 msgr2=0x7f4914150160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:00.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 shutdown_connections 2024-03-14T09:03:00.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.687+0000 7f491a71e700 1 -- 172.21.15.154:0/3338095457 wait complete. 2024-03-14T09:03:00.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T09:03:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.824+0000 7f4dfd29e700 1 Processor -- start 2024-03-14T09:03:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.824+0000 7f4dfd29e700 1 -- start start 2024-03-14T09:03:00.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4df814e980 0x7f4df81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df80c4a80 0x7f4df80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4df8158370 0x7f4df81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df80c6b10 con 0x7f4df814e980 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df814ed60 con 0x7f4df8158370 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4dfd29e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8155820 con 0x7f4df80c4a80 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4df814e980 0x7f4df81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4df814e980 0x7f4df81552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39756/0 (socket says 172.21.15.154:39756) 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 -- 172.21.15.154:0/1710705439 learned_addr learned my addr 172.21.15.154:0/1710705439 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:00.834 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df77fe700 1 --2- 172.21.15.154:0/1710705439 >> v2:172.21.15.191:3300/0 conn(0x7f4df8158370 0x7f4df81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df67fc700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df80c4a80 0x7f4df80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 -- 172.21.15.154:0/1710705439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df80c4a80 msgr2=0x7f4df80c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df80c4a80 0x7f4df80c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 -- 172.21.15.154:0/1710705439 >> v2:172.21.15.191:3300/0 conn(0x7f4df8158370 msgr2=0x7f4df81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1710705439 >> v2:172.21.15.191:3300/0 conn(0x7f4df8158370 0x7f4df81587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.825+0000 7f4df6ffd700 1 -- 172.21.15.154:0/1710705439 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80c6cf0 con 0x7f4df814e980 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df81552e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4df808c330 tx=0x7f4dec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18d628e8c512230 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4df5ffb700 1 -- 172.21.15.154:0/1710705439 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00b870 con 0x7f4df814e980 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4df5ffb700 1 -- 172.21.15.154:0/1710705439 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00b9d0 con 0x7f4df814e980 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4df5ffb700 1 -- 172.21.15.154:0/1710705439 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00bd60 con 0x7f4df814e980 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1710705439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 msgr2=0x7f4df81552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df81552e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4df808c330 tx=0x7f4dec009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1710705439 shutdown_connections 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1710705439 >> v2:172.21.15.191:3300/0 conn(0x7f4df8158370 0x7f4df81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df81552e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1710705439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df80c4a80 0x7f4df80c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1710705439 >> 172.21.15.154:0/1710705439 conn(0x7f4df80abcf0 msgr2=0x7f4df80ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1710705439 shutdown_connections 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.826+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1710705439 wait complete. 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 Processor -- start 2024-03-14T09:03:00.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 -- start start 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 0x7f4df81ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df8158370 0x7f4df814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df81eae20 con 0x7f4df814e980 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80c3e50 con 0x7f4df80c4a80 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4dfd29e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80c4130 con 0x7f4df8158370 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4df67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4df67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39760/0 (socket says 172.21.15.154:39760) 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.828+0000 7f4df67fc700 1 -- 172.21.15.154:0/1895710148 learned_addr learned my addr 172.21.15.154:0/1895710148 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 0x7f4df81ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 -- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df8158370 msgr2=0x7f4df814b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df8158370 0x7f4df814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 -- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 msgr2=0x7f4df81ea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:00.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 0x7f4df81ea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 -- 172.21.15.154:0/1895710148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009550 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 0x7f4df81ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4df67fc700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4de800e810 tx=0x7f4de800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8013610 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8013770 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8013b00 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df8152770 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.829+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df8155350 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.830+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df80697d0 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.830+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4de8013c60 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.831+0000 7f4ddf7fe700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 0x7f4de406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.831+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4de8088940 con 0x7f4df814e980 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.831+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 0x7f4de406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:00.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.831+0000 7f4df6ffd700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 0x7f4de406ccf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009130 tx=0x7f4dec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:00.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:00.835+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4de8053df0 con 0x7f4df814e980 2024-03-14T09:03:01.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.075+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4df814c440 con 0x7f4df814e980 2024-03-14T09:03:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.983+0000 7f4ddf7fe700 1 -- 172.21.15.154:0/1895710148 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4de80210b0 con 0x7f4df814e980 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 msgr2=0x7f4de406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 0x7f4de406ccf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009130 tx=0x7f4dec008030 comp rx=0 tx=0).stop 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 msgr2=0x7f4df8154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4de800e810 tx=0x7f4de800ebd0 comp rx=0 tx=0).stop 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 shutdown_connections 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4df8158370 0x7f4df814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4df80c4a80 0x7f4df81ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4de406a830 0x7f4de406ccf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:01.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 --2- 172.21.15.154:0/1895710148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df814e980 0x7f4df8154e10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:01.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 >> 172.21.15.154:0/1895710148 conn(0x7f4df80abcf0 msgr2=0x7f4df81567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:01.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 shutdown_connections 2024-03-14T09:03:01.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:01.986+0000 7f4dfd29e700 1 -- 172.21.15.154:0/1895710148 wait complete. 2024-03-14T09:03:02.001 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:03:02.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T09:03:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.131+0000 7fc7d35d5700 1 Processor -- start 2024-03-14T09:03:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.131+0000 7fc7d35d5700 1 -- start start 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d35d5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7cc0c1e90 0x7fc7cc13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d35d5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc13ade0 con 0x7fc7cc0c3250 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d35d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc06fb30 con 0x7fc7cc0a8540 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d35d5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc06ec30 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d1b72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7cc0c1e90 0x7fc7cc13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d0b70700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc7cc0c3250 0x7fc7cc0c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55100/0 (socket says 172.21.15.154:55100) 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d0b70700 1 -- 172.21.15.154:0/2280523661 learned_addr learned my addr 172.21.15.154:0/2280523661 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d1b72700 1 -- 172.21.15.154:0/2280523661 >> v1:172.21.15.191:6789/0 conn(0x7fc7cc0a8540 legacy=0x7fc7cc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d1b72700 1 -- 172.21.15.154:0/2280523661 >> v1:172.21.15.154:6789/0 conn(0x7fc7cc0c3250 legacy=0x7fc7cc0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d1b72700 1 -- 172.21.15.154:0/2280523661 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc06ef10 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7c3fff700 1 -- 172.21.15.154:0/2280523661 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229814936 0 0) 0x7fc7bc001110 con 0x7fc7cc0c3250 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7d1b72700 1 --2- 172.21.15.154:0/2280523661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc13a8a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24cdc535efa3550 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7c3fff700 1 -- 172.21.15.154:0/2280523661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400e070 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7c3fff700 1 -- 172.21.15.154:0/2280523661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400bb90 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.132+0000 7fc7c3fff700 1 -- 172.21.15.154:0/2280523661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4003a20 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 -- 172.21.15.154:0/2280523661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 msgr2=0x7fc7cc13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/2280523661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc13a8a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 -- 172.21.15.154:0/2280523661 shutdown_connections 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/2280523661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc13a8a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 -- 172.21.15.154:0/2280523661 >> 172.21.15.154:0/2280523661 conn(0x7fc7cc0c3880 msgr2=0x7fc7cc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 -- 172.21.15.154:0/2280523661 shutdown_connections 2024-03-14T09:03:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.133+0000 7fc7d35d5700 1 -- 172.21.15.154:0/2280523661 wait complete. 2024-03-14T09:03:02.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.134+0000 7fc7d35d5700 1 Processor -- start 2024-03-14T09:03:02.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.134+0000 7fc7d35d5700 1 -- start start 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0a8540 0x7fc7cc14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0c3250 0x7fc7cc1df260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc1eab40 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc1eb070 con 0x7fc7cc0c3250 2024-03-14T09:03:02.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d35d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0ac8d0 con 0x7fc7cc0a8540 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39768/0 (socket says 172.21.15.154:39768) 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 -- 172.21.15.154:0/569759619 learned_addr learned my addr 172.21.15.154:0/569759619 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d1371700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0a8540 0x7fc7cc14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d1b72700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0c3250 0x7fc7cc1df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 -- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0a8540 msgr2=0x7fc7cc14e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0a8540 0x7fc7cc14e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 -- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0c3250 msgr2=0x7fc7cc1df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0c3250 0x7fc7cc1df260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 -- 172.21.15.154:0/569759619 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009a70 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.135+0000 7fc7d0b70700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c5b0 tx=0x7fc7bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.136+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc004c80 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.136+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc004e00 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.136+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc007310 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.136+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc0a9d20 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.136+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc1ead90 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.137+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc0697d0 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.137+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc7bc007470 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.137+0000 7fc7c1ffb700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 0x7fc7b006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.137+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7bc0886b0 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.138+0000 7fc7d1371700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 0x7fc7b006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.138+0000 7fc7d1371700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 0x7fc7b006ccf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ace40 tx=0x7fc7c80090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:02.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.142+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc7bc053b60 con 0x7fc7cc0c1e90 2024-03-14T09:03:02.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:02.384+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc7cc13ade0 con 0x7fc7cc0c1e90 2024-03-14T09:03:03.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.320+0000 7fc7c1ffb700 1 -- 172.21.15.154:0/569759619 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc7cc13ade0 con 0x7fc7cc0c1e90 2024-03-14T09:03:03.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.323+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 msgr2=0x7fc7b006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.323+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 0x7fc7b006ccf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ace40 tx=0x7fc7c80090e0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.323+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 msgr2=0x7fc7cc154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.323+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c5b0 tx=0x7fc7bc00c8c0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 shutdown_connections 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7cc0c3250 0x7fc7cc1df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc7cc0a8540 0x7fc7cc14e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc7b006a830 0x7fc7b006ccf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 --2- 172.21.15.154:0/569759619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7cc0c1e90 0x7fc7cc154710 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 >> 172.21.15.154:0/569759619 conn(0x7fc7cc0c3880 msgr2=0x7fc7cc06de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 shutdown_connections 2024-03-14T09:03:03.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.324+0000 7fc7d35d5700 1 -- 172.21.15.154:0/569759619 wait complete. 2024-03-14T09:03:03.338 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T09:03:03.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:03.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.465+0000 7ffab292b700 1 Processor -- start 2024-03-14T09:03:03.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.465+0000 7ffab292b700 1 -- start start 2024-03-14T09:03:03.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffaac14eaf0 0x7ffaac06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 0x7ffaac155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffaac156330 0x7ffaac157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac06b310 con 0x7ffaac156330 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0ac8e0 con 0x7ffaac14eaf0 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab292b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac06de30 con 0x7ffaac06acb0 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab0ec8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffaac156330 0x7ffaac157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab0ec8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffaac156330 0x7ffaac157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39782/0 (socket says 172.21.15.154:39782) 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffab0ec8700 1 -- 172.21.15.154:0/364811455 learned_addr learned my addr 172.21.15.154:0/364811455 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffaabfff700 1 --2- 172.21.15.154:0/364811455 >> v2:172.21.15.191:3300/0 conn(0x7ffaac14eaf0 0x7ffaac06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.466+0000 7ffaab7fe700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 0x7ffaac155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 -- 172.21.15.154:0/364811455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 msgr2=0x7ffaac155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 0x7ffaac155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 -- 172.21.15.154:0/364811455 >> v2:172.21.15.191:3300/0 conn(0x7ffaac14eaf0 msgr2=0x7ffaac06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 --2- 172.21.15.154:0/364811455 >> v2:172.21.15.191:3300/0 conn(0x7ffaac14eaf0 0x7ffaac06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 -- 172.21.15.154:0/364811455 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0acb30 con 0x7ffaac156330 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffaab7fe700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 0x7ffaac155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:03.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab0ec8700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac156330 0x7ffaac157ef0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffaac070da0 tx=0x7ffaa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9fc5bb885f804e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffaaaffd700 1 -- 172.21.15.154:0/364811455 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400e070 con 0x7ffaac156330 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffaaaffd700 1 -- 172.21.15.154:0/364811455 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa400b9c0 con 0x7ffaac156330 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffaaaffd700 1 -- 172.21.15.154:0/364811455 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400bd70 con 0x7ffaac156330 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 -- 172.21.15.154:0/364811455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac156330 msgr2=0x7ffaac157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac156330 0x7ffaac157ef0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ffaac070da0 tx=0x7ffaa400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 -- 172.21.15.154:0/364811455 shutdown_connections 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac156330 0x7ffaac157ef0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 --2- 172.21.15.154:0/364811455 >> v2:172.21.15.191:3300/0 conn(0x7ffaac14eaf0 0x7ffaac06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 --2- 172.21.15.154:0/364811455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac06acb0 0x7ffaac155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.467+0000 7ffab292b700 1 -- 172.21.15.154:0/364811455 >> 172.21.15.154:0/364811455 conn(0x7ffaac0c3880 msgr2=0x7ffaac0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.468+0000 7ffab292b700 1 -- 172.21.15.154:0/364811455 shutdown_connections 2024-03-14T09:03:03.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.468+0000 7ffab292b700 1 -- 172.21.15.154:0/364811455 wait complete. 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.469+0000 7ffab292b700 1 Processor -- start 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.469+0000 7ffab292b700 1 -- start start 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaac14eaf0 0x7ffaac14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac156330 0x7ffaac14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac14af00 con 0x7ffaac06acb0 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac149210 con 0x7ffaac14eaf0 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab292b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac146c50 con 0x7ffaac156330 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39786/0 (socket says 172.21.15.154:39786) 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 -- 172.21.15.154:0/3043498348 learned_addr learned my addr 172.21.15.154:0/3043498348 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaab7fe700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaac14eaf0 0x7ffaac14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffab0ec8700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac156330 0x7ffaac14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 -- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac156330 msgr2=0x7ffaac14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac156330 0x7ffaac14b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 -- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaac14eaf0 msgr2=0x7ffaac14b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaac14eaf0 0x7ffaac14b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.470+0000 7ffaabfff700 1 -- 172.21.15.154:0/3043498348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4009b30 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffaabfff700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00e810 tx=0x7ffa9c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00c070 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c013610 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac1499e0 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0139c0 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.471+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac149700 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.472+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac14b170 con 0x7ffaac06acb0 2024-03-14T09:03:03.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.472+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffa9c013b20 con 0x7ffaac06acb0 2024-03-14T09:03:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.472+0000 7ffaa8ff9700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 0x7ffa9806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.473+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa9c088b50 con 0x7ffaac06acb0 2024-03-14T09:03:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.473+0000 7ffaab7fe700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 0x7ffa9806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.473+0000 7ffaab7fe700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 0x7ffa9806ccf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffaac148630 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.477+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffa9c053ef0 con 0x7ffaac06acb0 2024-03-14T09:03:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.713+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaac0697d0 con 0x7ffaac06acb0 2024-03-14T09:03:03.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.713+0000 7ffaa8ff9700 1 -- 172.21.15.154:0/3043498348 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4746 (secure 0 0 0) 0x7ffa9c057510 con 0x7ffaac06acb0 2024-03-14T09:03:03.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:03.713 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T09:03:03.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.715+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 msgr2=0x7ffa9806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.715+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 0x7ffa9806ccf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffaac148630 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.715+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 msgr2=0x7ffaac153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.715+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00e810 tx=0x7ffa9c00eb20 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 shutdown_connections 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffaac156330 0x7ffaac14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffa9806a830 0x7ffa9806ccf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffaac06acb0 0x7ffaac153950 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 --2- 172.21.15.154:0/3043498348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffaac14eaf0 0x7ffaac14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 >> 172.21.15.154:0/3043498348 conn(0x7ffaac0c3880 msgr2=0x7ffaac14c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 shutdown_connections 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.716+0000 7ffab292b700 1 -- 172.21.15.154:0/3043498348 wait complete. 2024-03-14T09:03:03.716 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 30 2024-03-14T09:03:03.728 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:03.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 Processor -- start 2024-03-14T09:03:03.849 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 -- start start 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d6814e980 0x7f8d681552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6814ed60 con 0x7f8d68158a80 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d68155820 con 0x7f8d680c4a80 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.849+0000 7f8d6d973700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d680c6b10 con 0x7f8d6814e980 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d66ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d6814e980 0x7f8d681552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d677fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8d68158a80 0x7f8d6815f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55106/0 (socket says 172.21.15.154:55106) 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d677fe700 1 -- 172.21.15.154:0/3948153529 learned_addr learned my addr 172.21.15.154:0/3948153529 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d66ffd700 1 -- 172.21.15.154:0/3948153529 >> v1:172.21.15.191:6789/0 conn(0x7f8d680c4a80 legacy=0x7f8d68158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d66ffd700 1 -- 172.21.15.154:0/3948153529 >> v1:172.21.15.154:6789/0 conn(0x7f8d68158a80 legacy=0x7f8d6815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d66ffd700 1 -- 172.21.15.154:0/3948153529 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680c6cf0 con 0x7f8d6814e980 2024-03-14T09:03:03.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d66ffd700 1 --2- 172.21.15.154:0/3948153529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d6814e980 0x7f8d681552e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009840 tx=0x7f8d58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad7c8b44e14cf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d65ffb700 1 -- 172.21.15.154:0/3948153529 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58004230 con 0x7f8d6814e980 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d65ffb700 1 -- 172.21.15.154:0/3948153529 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58004390 con 0x7f8d6814e980 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.850+0000 7f8d65ffb700 1 -- 172.21.15.154:0/3948153529 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d580036a0 con 0x7f8d6814e980 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 -- 172.21.15.154:0/3948153529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d6814e980 msgr2=0x7f8d681552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 --2- 172.21.15.154:0/3948153529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d6814e980 0x7f8d681552e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009840 tx=0x7f8d58009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 -- 172.21.15.154:0/3948153529 shutdown_connections 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 --2- 172.21.15.154:0/3948153529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d6814e980 0x7f8d681552e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 -- 172.21.15.154:0/3948153529 >> 172.21.15.154:0/3948153529 conn(0x7f8d680abcf0 msgr2=0x7f8d680ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 -- 172.21.15.154:0/3948153529 shutdown_connections 2024-03-14T09:03:03.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.851+0000 7f8d6d973700 1 -- 172.21.15.154:0/3948153529 wait complete. 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.852+0000 7f8d6d973700 1 Processor -- start 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 -- start start 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d6814e980 0x7f8d68153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d68158a80 0x7f8d680acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68153520 con 0x7f8d680c4a80 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d68152210 con 0x7f8d6814e980 2024-03-14T09:03:03.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d6d973700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c0cc0 con 0x7f8d68158a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d66ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d66ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39794/0 (socket says 172.21.15.154:39794) 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.853+0000 7f8d66ffd700 1 -- 172.21.15.154:0/1425922975 learned_addr learned my addr 172.21.15.154:0/1425922975 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 -- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d68158a80 msgr2=0x7f8d680acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d68158a80 0x7f8d680acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 -- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d6814e980 msgr2=0x7f8d68153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d6814e980 0x7f8d68153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 -- 172.21.15.154:0/1425922975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580094a0 con 0x7f8d680c4a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d66ffd700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8d580090b0 tx=0x7f8d58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58011830 con 0x7f8d680c4a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5801e070 con 0x7f8d680c4a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d681550d0 con 0x7f8d680c4a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58011bf0 con 0x7f8d680c4a80 2024-03-14T09:03:03.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.854+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d680c0950 con 0x7f8d680c4a80 2024-03-14T09:03:03.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.855+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d680697d0 con 0x7f8d680c4a80 2024-03-14T09:03:03.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.856+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8d58011d50 con 0x7f8d680c4a80 2024-03-14T09:03:03.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.856+0000 7f8d4ffff700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 0x7f8d5406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:03.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.856+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d58089fb0 con 0x7f8d680c4a80 2024-03-14T09:03:03.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.856+0000 7f8d667fc700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 0x7f8d5406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:03.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.857+0000 7f8d667fc700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 0x7f8d5406ccf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d680c9de0 tx=0x7f8d5c006c50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:03.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:03.860+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8d580bd4f0 con 0x7f8d680c4a80 2024-03-14T09:03:04.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.055+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d680ad720 con 0x7f8d680c4a80 2024-03-14T09:03:04.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.056+0000 7f8d4ffff700 1 -- 172.21.15.154:0/1425922975 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16145 (secure 0 0 0) 0x7f8d58020090 con 0x7f8d680c4a80 2024-03-14T09:03:04.056 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:04.056 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":51,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:00.973845+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:04.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 msgr2=0x7f8d5406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 0x7f8d5406ccf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d680c9de0 tx=0x7f8d5c006c50 comp rx=0 tx=0).stop 2024-03-14T09:03:04.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 msgr2=0x7f8d680ad940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8d580090b0 tx=0x7f8d58004750 comp rx=0 tx=0).stop 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 shutdown_connections 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d68158a80 0x7f8d680acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8d5406a830 0x7f8d5406ccf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680c4a80 0x7f8d680ad940 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 --2- 172.21.15.154:0/1425922975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d6814e980 0x7f8d68153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 >> 172.21.15.154:0/1425922975 conn(0x7f8d680abcf0 msgr2=0x7f8d68073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 shutdown_connections 2024-03-14T09:03:04.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.058+0000 7f8d6d973700 1 -- 172.21.15.154:0/1425922975 wait complete. 2024-03-14T09:03:04.071 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:03:04.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:04.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.192+0000 7f13e013e700 1 Processor -- start 2024-03-14T09:03:04.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 -- start start 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80702f0 0x7f13d80c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13d814fbd0 0x7f13d80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80acd10 con 0x7f13d80c78c0 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80ac7b0 con 0x7f13d814fbd0 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.193+0000 7f13e013e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8150050 con 0x7f13d80702f0 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13d814fbd0 0x7f13d80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13d814fbd0 0x7f13d80c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39802/0 (socket says 172.21.15.154:39802) 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 -- 172.21.15.154:0/380172458 learned_addr learned my addr 172.21.15.154:0/380172458 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13ddeda700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80702f0 0x7f13d80c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 -- 172.21.15.154:0/380172458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80702f0 msgr2=0x7f13d80c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80702f0 0x7f13d80c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 -- 172.21.15.154:0/380172458 >> v1:172.21.15.191:6789/0 conn(0x7f13d80c78c0 legacy=0x7f13d80c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 -- 172.21.15.154:0/380172458 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80ac9c0 con 0x7f13d814fbd0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dd6d9700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c7380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0098f0 tx=0x7f13cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39d9e8f73b6ff98 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dced8700 1 -- 172.21.15.154:0/380172458 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00b8a0 con 0x7f13d814fbd0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dced8700 1 -- 172.21.15.154:0/380172458 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc00ba00 con 0x7f13d814fbd0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.194+0000 7f13dced8700 1 -- 172.21.15.154:0/380172458 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00bd90 con 0x7f13d814fbd0 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 -- 172.21.15.154:0/380172458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 msgr2=0x7f13d80c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c7380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0098f0 tx=0x7f13cc009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 -- 172.21.15.154:0/380172458 shutdown_connections 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80702f0 0x7f13d80c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 --2- 172.21.15.154:0/380172458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c7380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 -- 172.21.15.154:0/380172458 >> 172.21.15.154:0/380172458 conn(0x7f13d80a9bf0 msgr2=0x7f13d80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 -- 172.21.15.154:0/380172458 shutdown_connections 2024-03-14T09:03:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.195+0000 7f13e013e700 1 -- 172.21.15.154:0/380172458 wait complete. 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.196+0000 7f13e013e700 1 Processor -- start 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.196+0000 7f13e013e700 1 -- start start 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13d80702f0 0x7f13d814a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80c78c0 0x7f13d81ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81eec30 con 0x7f13d814fbd0 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d806d2c0 con 0x7f13d80702f0 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d806cd00 con 0x7f13d80c78c0 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39808/0 (socket says 172.21.15.154:39808) 2024-03-14T09:03:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 -- 172.21.15.154:0/2277867083 learned_addr learned my addr 172.21.15.154:0/2277867083 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13dd6d9700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80c78c0 0x7f13d81ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13ddeda700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13d80702f0 0x7f13d814a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 -- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80c78c0 msgr2=0x7f13d81ee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80c78c0 0x7f13d81ee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 -- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13d80702f0 msgr2=0x7f13d814a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13d80702f0 0x7f13d814a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 -- 172.21.15.154:0/2277867083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc009550 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13de6db700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13d000cb50 tx=0x7f13d000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0003c00 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d0003d60 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d806cfe0 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000aac0 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.197+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81eab60 con 0x7f13d814fbd0 2024-03-14T09:03:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.198+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d81ead50 con 0x7f13d814fbd0 2024-03-14T09:03:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.199+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f13d000ac20 con 0x7f13d814fbd0 2024-03-14T09:03:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.199+0000 7f13ca7fc700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 0x7f13bc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.199+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13d0088d60 con 0x7f13d814fbd0 2024-03-14T09:03:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.199+0000 7f13ddeda700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 0x7f13bc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.200+0000 7f13ddeda700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 0x7f13bc06ccf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f13d400b9b0 tx=0x7f13d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.203+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f13d0054180 con 0x7f13d814fbd0 2024-03-14T09:03:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.394+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13d8070ad0 con 0x7f13bc06a830 2024-03-14T09:03:04.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.403+0000 7f13ca7fc700 1 -- 172.21.15.154:0/2277867083 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f13d400be80 con 0x7f13bc06a830 2024-03-14T09:03:04.407 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:04.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 msgr2=0x7f13bc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 0x7f13bc06ccf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f13d400b9b0 tx=0x7f13d4008040 comp rx=0 tx=0).stop 2024-03-14T09:03:04.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 msgr2=0x7f13d80c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f13d000cb50 tx=0x7f13d000ce60 comp rx=0 tx=0).stop 2024-03-14T09:03:04.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 shutdown_connections 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f13bc06a830 0x7f13bc06ccf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d814fbd0 0x7f13d80c14e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13d80702f0 0x7f13d814a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 --2- 172.21.15.154:0/2277867083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13d80c78c0 0x7f13d81ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 >> 172.21.15.154:0/2277867083 conn(0x7f13d80a9bf0 msgr2=0x7f13d806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 shutdown_connections 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.406+0000 7f13e013e700 1 -- 172.21.15.154:0/2277867083 wait complete. 2024-03-14T09:03:04.410 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:04.418 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-03-14T09:03:03.541978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222232,"kb_used_data":5992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752488,"statfs":{"total":773094113280,"available":772866547712,"internally_reserved":0,"allocated":6135808,"data_stored":3244612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985046"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152708+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:01.152708+0000","last_peered":"2024-03-14T09:03:01.152708+0000","last_clean":"2024-03-14T09:03:01.152708+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:01.152708+0000","last_undegraded":"2024-03-14T09:03:01.152708+0000","last_fullsized":"2024-03-14T09:03:01.152708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153053+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:01.153053+0000","last_peered":"2024-03-14T09:03:01.153053+0000","last_clean":"2024-03-14T09:03:01.153053+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:01.153053+0000","last_undegraded":"2024-03-14T09:03:01.153053+0000","last_fullsized":"2024-03-14T09:03:01.153053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153111+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:01.153111+0000","last_peered":"2024-03-14T09:03:01.153111+0000","last_clean":"2024-03-14T09:03:01.153111+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:01.153111+0000","last_undegraded":"2024-03-14T09:03:01.153111+0000","last_fullsized":"2024-03-14T09:03:01.153111+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986319+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:00.986319+0000","last_peered":"2024-03-14T09:03:00.986319+0000","last_clean":"2024-03-14T09:03:00.986319+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:00.986319+0000","last_undegraded":"2024-03-14T09:03:00.986319+0000","last_fullsized":"2024-03-14T09:03:00.986319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152588+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:01.152588+0000","last_peered":"2024-03-14T09:03:01.152588+0000","last_clean":"2024-03-14T09:03:01.152588+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:01.152588+0000","last_undegraded":"2024-03-14T09:03:01.152588+0000","last_fullsized":"2024-03-14T09:03:01.152588+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152125+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:01.152125+0000","last_peered":"2024-03-14T09:03:01.152125+0000","last_clean":"2024-03-14T09:03:01.152125+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:01.152125+0000","last_undegraded":"2024-03-14T09:03:01.152125+0000","last_fullsized":"2024-03-14T09:03:01.152125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986038+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:00.986038+0000","last_peered":"2024-03-14T09:03:00.986038+0000","last_clean":"2024-03-14T09:03:00.986038+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:00.986038+0000","last_undegraded":"2024-03-14T09:03:00.986038+0000","last_fullsized":"2024-03-14T09:03:00.986038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152829+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:01.152829+0000","last_peered":"2024-03-14T09:03:01.152829+0000","last_clean":"2024-03-14T09:03:01.152829+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:01.152829+0000","last_undegraded":"2024-03-14T09:03:01.152829+0000","last_fullsized":"2024-03-14T09:03:01.152829+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986250+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:00.986250+0000","last_peered":"2024-03-14T09:03:00.986250+0000","last_clean":"2024-03-14T09:03:00.986250+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:00.986250+0000","last_undegraded":"2024-03-14T09:03:00.986250+0000","last_fullsized":"2024-03-14T09:03:00.986250+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982948+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:00.982948+0000","last_peered":"2024-03-14T09:03:00.982948+0000","last_clean":"2024-03-14T09:03:00.982948+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:00.982948+0000","last_undegraded":"2024-03-14T09:03:00.982948+0000","last_fullsized":"2024-03-14T09:03:00.982948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982938+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:00.982938+0000","last_peered":"2024-03-14T09:03:00.982938+0000","last_clean":"2024-03-14T09:03:00.982938+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:00.982938+0000","last_undegraded":"2024-03-14T09:03:00.982938+0000","last_fullsized":"2024-03-14T09:03:00.982938+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986419+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:00.986419+0000","last_peered":"2024-03-14T09:03:00.986419+0000","last_clean":"2024-03-14T09:03:00.986419+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:00.986419+0000","last_undegraded":"2024-03-14T09:03:00.986419+0000","last_fullsized":"2024-03-14T09:03:00.986419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153012+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:01.153012+0000","last_peered":"2024-03-14T09:03:01.153012+0000","last_clean":"2024-03-14T09:03:01.153012+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:01.153012+0000","last_undegraded":"2024-03-14T09:03:01.153012+0000","last_fullsized":"2024-03-14T09:03:01.153012+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986322+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:00.986322+0000","last_peered":"2024-03-14T09:03:00.986322+0000","last_clean":"2024-03-14T09:03:00.986322+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:00.986322+0000","last_undegraded":"2024-03-14T09:03:00.986322+0000","last_fullsized":"2024-03-14T09:03:00.986322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986134+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:00.986134+0000","last_peered":"2024-03-14T09:03:00.986134+0000","last_clean":"2024-03-14T09:03:00.986134+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:00.986134+0000","last_undegraded":"2024-03-14T09:03:00.986134+0000","last_fullsized":"2024-03-14T09:03:00.986134+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152062+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:01.152062+0000","last_peered":"2024-03-14T09:03:01.152062+0000","last_clean":"2024-03-14T09:03:01.152062+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:01.152062+0000","last_undegraded":"2024-03-14T09:03:01.152062+0000","last_fullsized":"2024-03-14T09:03:01.152062+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153102+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:01.153102+0000","last_peered":"2024-03-14T09:03:01.153102+0000","last_clean":"2024-03-14T09:03:01.153102+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:01.153102+0000","last_undegraded":"2024-03-14T09:03:01.153102+0000","last_fullsized":"2024-03-14T09:03:01.153102+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152864+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:01.152864+0000","last_peered":"2024-03-14T09:03:01.152864+0000","last_clean":"2024-03-14T09:03:01.152864+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:01.152864+0000","last_undegraded":"2024-03-14T09:03:01.152864+0000","last_fullsized":"2024-03-14T09:03:01.152864+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152546+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:01.152546+0000","last_peered":"2024-03-14T09:03:01.152546+0000","last_clean":"2024-03-14T09:03:01.152546+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:01.152546+0000","last_undegraded":"2024-03-14T09:03:01.152546+0000","last_fullsized":"2024-03-14T09:03:01.152546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152927+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:01.152927+0000","last_peered":"2024-03-14T09:03:01.152927+0000","last_clean":"2024-03-14T09:03:01.152927+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:01.152927+0000","last_undegraded":"2024-03-14T09:03:01.152927+0000","last_fullsized":"2024-03-14T09:03:01.152927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153645+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:01.153645+0000","last_peered":"2024-03-14T09:03:01.153645+0000","last_clean":"2024-03-14T09:03:01.153645+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:01.153645+0000","last_undegraded":"2024-03-14T09:03:01.153645+0000","last_fullsized":"2024-03-14T09:03:01.153645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152798+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:01.152798+0000","last_peered":"2024-03-14T09:03:01.152798+0000","last_clean":"2024-03-14T09:03:01.152798+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:01.152798+0000","last_undegraded":"2024-03-14T09:03:01.152798+0000","last_fullsized":"2024-03-14T09:03:01.152798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152630+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:01.152630+0000","last_peered":"2024-03-14T09:03:01.152630+0000","last_clean":"2024-03-14T09:03:01.152630+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:01.152630+0000","last_undegraded":"2024-03-14T09:03:01.152630+0000","last_fullsized":"2024-03-14T09:03:01.152630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153421+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:01.153421+0000","last_peered":"2024-03-14T09:03:01.153421+0000","last_clean":"2024-03-14T09:03:01.153421+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:01.153421+0000","last_undegraded":"2024-03-14T09:03:01.153421+0000","last_fullsized":"2024-03-14T09:03:01.153421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152756+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:01.152756+0000","last_peered":"2024-03-14T09:03:01.152756+0000","last_clean":"2024-03-14T09:03:01.152756+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:01.152756+0000","last_undegraded":"2024-03-14T09:03:01.152756+0000","last_fullsized":"2024-03-14T09:03:01.152756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986602+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:00.986602+0000","last_peered":"2024-03-14T09:03:00.986602+0000","last_clean":"2024-03-14T09:03:00.986602+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:00.986602+0000","last_undegraded":"2024-03-14T09:03:00.986602+0000","last_fullsized":"2024-03-14T09:03:00.986602+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153336+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:01.153336+0000","last_peered":"2024-03-14T09:03:01.153336+0000","last_clean":"2024-03-14T09:03:01.153336+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:01.153336+0000","last_undegraded":"2024-03-14T09:03:01.153336+0000","last_fullsized":"2024-03-14T09:03:01.153336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152558+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:01.152558+0000","last_peered":"2024-03-14T09:03:01.152558+0000","last_clean":"2024-03-14T09:03:01.152558+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:01.152558+0000","last_undegraded":"2024-03-14T09:03:01.152558+0000","last_fullsized":"2024-03-14T09:03:01.152558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152714+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:01.152714+0000","last_peered":"2024-03-14T09:03:01.152714+0000","last_clean":"2024-03-14T09:03:01.152714+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:01.152714+0000","last_undegraded":"2024-03-14T09:03:01.152714+0000","last_fullsized":"2024-03-14T09:03:01.152714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152893+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:01.152893+0000","last_peered":"2024-03-14T09:03:01.152893+0000","last_clean":"2024-03-14T09:03:01.152893+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:01.152893+0000","last_undegraded":"2024-03-14T09:03:01.152893+0000","last_fullsized":"2024-03-14T09:03:01.152893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152403+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:01.152403+0000","last_peered":"2024-03-14T09:03:01.152403+0000","last_clean":"2024-03-14T09:03:01.152403+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:01.152403+0000","last_undegraded":"2024-03-14T09:03:01.152403+0000","last_fullsized":"2024-03-14T09:03:01.152403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152129+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:01.152129+0000","last_peered":"2024-03-14T09:03:01.152129+0000","last_clean":"2024-03-14T09:03:01.152129+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:01.152129+0000","last_undegraded":"2024-03-14T09:03:01.152129+0000","last_fullsized":"2024-03-14T09:03:01.152129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153415+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:01.153415+0000","last_peered":"2024-03-14T09:03:01.153415+0000","last_clean":"2024-03-14T09:03:01.153415+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:01.153415+0000","last_undegraded":"2024-03-14T09:03:01.153415+0000","last_fullsized":"2024-03-14T09:03:01.153415+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153519+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:01.153519+0000","last_peered":"2024-03-14T09:03:01.153519+0000","last_clean":"2024-03-14T09:03:01.153519+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:01.153519+0000","last_undegraded":"2024-03-14T09:03:01.153519+0000","last_fullsized":"2024-03-14T09:03:01.153519+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.983059+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:00.983059+0000","last_peered":"2024-03-14T09:03:00.983059+0000","last_clean":"2024-03-14T09:03:00.983059+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:00.983059+0000","last_undegraded":"2024-03-14T09:03:00.983059+0000","last_fullsized":"2024-03-14T09:03:00.983059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153115+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:01.153115+0000","last_peered":"2024-03-14T09:03:01.153115+0000","last_clean":"2024-03-14T09:03:01.153115+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:01.153115+0000","last_undegraded":"2024-03-14T09:03:01.153115+0000","last_fullsized":"2024-03-14T09:03:01.153115+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153272+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:01.153272+0000","last_peered":"2024-03-14T09:03:01.153272+0000","last_clean":"2024-03-14T09:03:01.153272+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:01.153272+0000","last_undegraded":"2024-03-14T09:03:01.153272+0000","last_fullsized":"2024-03-14T09:03:01.153272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153615+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:01.153615+0000","last_peered":"2024-03-14T09:03:01.153615+0000","last_clean":"2024-03-14T09:03:01.153615+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:01.153615+0000","last_undegraded":"2024-03-14T09:03:01.153615+0000","last_fullsized":"2024-03-14T09:03:01.153615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153440+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:01.153440+0000","last_peered":"2024-03-14T09:03:01.153440+0000","last_clean":"2024-03-14T09:03:01.153440+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:01.153440+0000","last_undegraded":"2024-03-14T09:03:01.153440+0000","last_fullsized":"2024-03-14T09:03:01.153440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986536+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:00.986536+0000","last_peered":"2024-03-14T09:03:00.986536+0000","last_clean":"2024-03-14T09:03:00.986536+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:00.986536+0000","last_undegraded":"2024-03-14T09:03:00.986536+0000","last_fullsized":"2024-03-14T09:03:00.986536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153055+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:01.153055+0000","last_peered":"2024-03-14T09:03:01.153055+0000","last_clean":"2024-03-14T09:03:01.153055+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:01.153055+0000","last_undegraded":"2024-03-14T09:03:01.153055+0000","last_fullsized":"2024-03-14T09:03:01.153055+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985798+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:00.985798+0000","last_peered":"2024-03-14T09:03:00.985798+0000","last_clean":"2024-03-14T09:03:00.985798+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:00.985798+0000","last_undegraded":"2024-03-14T09:03:00.985798+0000","last_fullsized":"2024-03-14T09:03:00.985798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152769+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:01.152769+0000","last_peered":"2024-03-14T09:03:01.152769+0000","last_clean":"2024-03-14T09:03:01.152769+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:01.152769+0000","last_undegraded":"2024-03-14T09:03:01.152769+0000","last_fullsized":"2024-03-14T09:03:01.152769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152229+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:01.152229+0000","last_peered":"2024-03-14T09:03:01.152229+0000","last_clean":"2024-03-14T09:03:01.152229+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:01.152229+0000","last_undegraded":"2024-03-14T09:03:01.152229+0000","last_fullsized":"2024-03-14T09:03:01.152229+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153480+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:01.153480+0000","last_peered":"2024-03-14T09:03:01.153480+0000","last_clean":"2024-03-14T09:03:01.153480+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:01.153480+0000","last_undegraded":"2024-03-14T09:03:01.153480+0000","last_fullsized":"2024-03-14T09:03:01.153480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985830+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:00.985830+0000","last_peered":"2024-03-14T09:03:00.985830+0000","last_clean":"2024-03-14T09:03:00.985830+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:00.985830+0000","last_undegraded":"2024-03-14T09:03:00.985830+0000","last_fullsized":"2024-03-14T09:03:00.985830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153093+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:01.153093+0000","last_peered":"2024-03-14T09:03:01.153093+0000","last_clean":"2024-03-14T09:03:01.153093+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:01.153093+0000","last_undegraded":"2024-03-14T09:03:01.153093+0000","last_fullsized":"2024-03-14T09:03:01.153093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985915+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:00.985915+0000","last_peered":"2024-03-14T09:03:00.985915+0000","last_clean":"2024-03-14T09:03:00.985915+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:00.985915+0000","last_undegraded":"2024-03-14T09:03:00.985915+0000","last_fullsized":"2024-03-14T09:03:00.985915+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986344+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:00.986344+0000","last_peered":"2024-03-14T09:03:00.986344+0000","last_clean":"2024-03-14T09:03:00.986344+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:00.986344+0000","last_undegraded":"2024-03-14T09:03:00.986344+0000","last_fullsized":"2024-03-14T09:03:00.986344+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153263+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:01.153263+0000","last_peered":"2024-03-14T09:03:01.153263+0000","last_clean":"2024-03-14T09:03:01.153263+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:01.153263+0000","last_undegraded":"2024-03-14T09:03:01.153263+0000","last_fullsized":"2024-03-14T09:03:01.153263+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153128+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:01.153128+0000","last_peered":"2024-03-14T09:03:01.153128+0000","last_clean":"2024-03-14T09:03:01.153128+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:01.153128+0000","last_undegraded":"2024-03-14T09:03:01.153128+0000","last_fullsized":"2024-03-14T09:03:01.153128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153184+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:01.153184+0000","last_peered":"2024-03-14T09:03:01.153184+0000","last_clean":"2024-03-14T09:03:01.153184+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:01.153184+0000","last_undegraded":"2024-03-14T09:03:01.153184+0000","last_fullsized":"2024-03-14T09:03:01.153184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982727+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:00.982727+0000","last_peered":"2024-03-14T09:03:00.982727+0000","last_clean":"2024-03-14T09:03:00.982727+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:00.982727+0000","last_undegraded":"2024-03-14T09:03:00.982727+0000","last_fullsized":"2024-03-14T09:03:00.982727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153352+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:01.153352+0000","last_peered":"2024-03-14T09:03:01.153352+0000","last_clean":"2024-03-14T09:03:01.153352+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:01.153352+0000","last_undegraded":"2024-03-14T09:03:01.153352+0000","last_fullsized":"2024-03-14T09:03:01.153352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153228+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:01.153228+0000","last_peered":"2024-03-14T09:03:01.153228+0000","last_clean":"2024-03-14T09:03:01.153228+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:01.153228+0000","last_undegraded":"2024-03-14T09:03:01.153228+0000","last_fullsized":"2024-03-14T09:03:01.153228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986521+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:00.986521+0000","last_peered":"2024-03-14T09:03:00.986521+0000","last_clean":"2024-03-14T09:03:00.986521+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:00.986521+0000","last_undegraded":"2024-03-14T09:03:00.986521+0000","last_fullsized":"2024-03-14T09:03:00.986521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985655+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:00.985655+0000","last_peered":"2024-03-14T09:03:00.985655+0000","last_clean":"2024-03-14T09:03:00.985655+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:00.985655+0000","last_undegraded":"2024-03-14T09:03:00.985655+0000","last_fullsized":"2024-03-14T09:03:00.985655+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152884+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:01.152884+0000","last_peered":"2024-03-14T09:03:01.152884+0000","last_clean":"2024-03-14T09:03:01.152884+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:01.152884+0000","last_undegraded":"2024-03-14T09:03:01.152884+0000","last_fullsized":"2024-03-14T09:03:01.152884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153743+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:01.153743+0000","last_peered":"2024-03-14T09:03:01.153743+0000","last_clean":"2024-03-14T09:03:01.153743+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:01.153743+0000","last_undegraded":"2024-03-14T09:03:01.153743+0000","last_fullsized":"2024-03-14T09:03:01.153743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152948+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:01.152948+0000","last_peered":"2024-03-14T09:03:01.152948+0000","last_clean":"2024-03-14T09:03:01.152948+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:01.152948+0000","last_undegraded":"2024-03-14T09:03:01.152948+0000","last_fullsized":"2024-03-14T09:03:01.152948+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985848+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:00.985848+0000","last_peered":"2024-03-14T09:03:00.985848+0000","last_clean":"2024-03-14T09:03:00.985848+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:00.985848+0000","last_undegraded":"2024-03-14T09:03:00.985848+0000","last_fullsized":"2024-03-14T09:03:00.985848+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985929+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:00.985929+0000","last_peered":"2024-03-14T09:03:00.985929+0000","last_clean":"2024-03-14T09:03:00.985929+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:00.985929+0000","last_undegraded":"2024-03-14T09:03:00.985929+0000","last_fullsized":"2024-03-14T09:03:00.985929+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152751+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:01.152751+0000","last_peered":"2024-03-14T09:03:01.152751+0000","last_clean":"2024-03-14T09:03:01.152751+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:01.152751+0000","last_undegraded":"2024-03-14T09:03:01.152751+0000","last_fullsized":"2024-03-14T09:03:01.152751+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152996+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:01.152996+0000","last_peered":"2024-03-14T09:03:01.152996+0000","last_clean":"2024-03-14T09:03:01.152996+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:01.152996+0000","last_undegraded":"2024-03-14T09:03:01.152996+0000","last_fullsized":"2024-03-14T09:03:01.152996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153141+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:01.153141+0000","last_peered":"2024-03-14T09:03:01.153141+0000","last_clean":"2024-03-14T09:03:01.153141+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:01.153141+0000","last_undegraded":"2024-03-14T09:03:01.153141+0000","last_fullsized":"2024-03-14T09:03:01.153141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153179+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:01.153179+0000","last_peered":"2024-03-14T09:03:01.153179+0000","last_clean":"2024-03-14T09:03:01.153179+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:01.153179+0000","last_undegraded":"2024-03-14T09:03:01.153179+0000","last_fullsized":"2024-03-14T09:03:01.153179+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986237+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:00.986237+0000","last_peered":"2024-03-14T09:03:00.986237+0000","last_clean":"2024-03-14T09:03:00.986237+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:00.986237+0000","last_undegraded":"2024-03-14T09:03:00.986237+0000","last_fullsized":"2024-03-14T09:03:00.986237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153439+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:01.153439+0000","last_peered":"2024-03-14T09:03:01.153439+0000","last_clean":"2024-03-14T09:03:01.153439+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:01.153439+0000","last_undegraded":"2024-03-14T09:03:01.153439+0000","last_fullsized":"2024-03-14T09:03:01.153439+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153048+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:01.153048+0000","last_peered":"2024-03-14T09:03:01.153048+0000","last_clean":"2024-03-14T09:03:01.153048+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:01.153048+0000","last_undegraded":"2024-03-14T09:03:01.153048+0000","last_fullsized":"2024-03-14T09:03:01.153048+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986629+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:00.986629+0000","last_peered":"2024-03-14T09:03:00.986629+0000","last_clean":"2024-03-14T09:03:00.986629+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:00.986629+0000","last_undegraded":"2024-03-14T09:03:00.986629+0000","last_fullsized":"2024-03-14T09:03:00.986629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153308+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:01.153308+0000","last_peered":"2024-03-14T09:03:01.153308+0000","last_clean":"2024-03-14T09:03:01.153308+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:01.153308+0000","last_undegraded":"2024-03-14T09:03:01.153308+0000","last_fullsized":"2024-03-14T09:03:01.153308+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153336+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:01.153336+0000","last_peered":"2024-03-14T09:03:01.153336+0000","last_clean":"2024-03-14T09:03:01.153336+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:01.153336+0000","last_undegraded":"2024-03-14T09:03:01.153336+0000","last_fullsized":"2024-03-14T09:03:01.153336+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985762+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:00.985762+0000","last_peered":"2024-03-14T09:03:00.985762+0000","last_clean":"2024-03-14T09:03:00.985762+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:00.985762+0000","last_undegraded":"2024-03-14T09:03:00.985762+0000","last_fullsized":"2024-03-14T09:03:00.985762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152927+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:01.152927+0000","last_peered":"2024-03-14T09:03:01.152927+0000","last_clean":"2024-03-14T09:03:01.152927+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:01.152927+0000","last_undegraded":"2024-03-14T09:03:01.152927+0000","last_fullsized":"2024-03-14T09:03:01.152927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152339+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:01.152339+0000","last_peered":"2024-03-14T09:03:01.152339+0000","last_clean":"2024-03-14T09:03:01.152339+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:01.152339+0000","last_undegraded":"2024-03-14T09:03:01.152339+0000","last_fullsized":"2024-03-14T09:03:01.152339+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982736+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:00.982736+0000","last_peered":"2024-03-14T09:03:00.982736+0000","last_clean":"2024-03-14T09:03:00.982736+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:00.982736+0000","last_undegraded":"2024-03-14T09:03:00.982736+0000","last_fullsized":"2024-03-14T09:03:00.982736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152100+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:01.152100+0000","last_peered":"2024-03-14T09:03:01.152100+0000","last_clean":"2024-03-14T09:03:01.152100+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:01.152100+0000","last_undegraded":"2024-03-14T09:03:01.152100+0000","last_fullsized":"2024-03-14T09:03:01.152100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985618+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:00.985618+0000","last_peered":"2024-03-14T09:03:00.985618+0000","last_clean":"2024-03-14T09:03:00.985618+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:00.985618+0000","last_undegraded":"2024-03-14T09:03:00.985618+0000","last_fullsized":"2024-03-14T09:03:00.985618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986706+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:00.986706+0000","last_peered":"2024-03-14T09:03:00.986706+0000","last_clean":"2024-03-14T09:03:00.986706+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:00.986706+0000","last_undegraded":"2024-03-14T09:03:00.986706+0000","last_fullsized":"2024-03-14T09:03:00.986706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.983029+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:00.983029+0000","last_peered":"2024-03-14T09:03:00.983029+0000","last_clean":"2024-03-14T09:03:00.983029+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:00.983029+0000","last_undegraded":"2024-03-14T09:03:00.983029+0000","last_fullsized":"2024-03-14T09:03:00.983029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986059+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:00.986059+0000","last_peered":"2024-03-14T09:03:00.986059+0000","last_clean":"2024-03-14T09:03:00.986059+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:00.986059+0000","last_undegraded":"2024-03-14T09:03:00.986059+0000","last_fullsized":"2024-03-14T09:03:00.986059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153461+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:01.153461+0000","last_peered":"2024-03-14T09:03:01.153461+0000","last_clean":"2024-03-14T09:03:01.153461+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:01.153461+0000","last_undegraded":"2024-03-14T09:03:01.153461+0000","last_fullsized":"2024-03-14T09:03:01.153461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982784+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:00.982784+0000","last_peered":"2024-03-14T09:03:00.982784+0000","last_clean":"2024-03-14T09:03:00.982784+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:00.982784+0000","last_undegraded":"2024-03-14T09:03:00.982784+0000","last_fullsized":"2024-03-14T09:03:00.982784+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153275+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:01.153275+0000","last_peered":"2024-03-14T09:03:01.153275+0000","last_clean":"2024-03-14T09:03:01.153275+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:01.153275+0000","last_undegraded":"2024-03-14T09:03:01.153275+0000","last_fullsized":"2024-03-14T09:03:01.153275+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153534+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:01.153534+0000","last_peered":"2024-03-14T09:03:01.153534+0000","last_clean":"2024-03-14T09:03:01.153534+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:01.153534+0000","last_undegraded":"2024-03-14T09:03:01.153534+0000","last_fullsized":"2024-03-14T09:03:01.153534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986710+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:00.986710+0000","last_peered":"2024-03-14T09:03:00.986710+0000","last_clean":"2024-03-14T09:03:00.986710+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:00.986710+0000","last_undegraded":"2024-03-14T09:03:00.986710+0000","last_fullsized":"2024-03-14T09:03:00.986710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153259+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:01.153259+0000","last_peered":"2024-03-14T09:03:01.153259+0000","last_clean":"2024-03-14T09:03:01.153259+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:01.153259+0000","last_undegraded":"2024-03-14T09:03:01.153259+0000","last_fullsized":"2024-03-14T09:03:01.153259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982681+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:00.982681+0000","last_peered":"2024-03-14T09:03:00.982681+0000","last_clean":"2024-03-14T09:03:00.982681+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:00.982681+0000","last_undegraded":"2024-03-14T09:03:00.982681+0000","last_fullsized":"2024-03-14T09:03:00.982681+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153839+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:01.153839+0000","last_peered":"2024-03-14T09:03:01.153839+0000","last_clean":"2024-03-14T09:03:01.153839+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:01.153839+0000","last_undegraded":"2024-03-14T09:03:01.153839+0000","last_fullsized":"2024-03-14T09:03:01.153839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986410+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:00.986410+0000","last_peered":"2024-03-14T09:03:00.986410+0000","last_clean":"2024-03-14T09:03:00.986410+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:00.986410+0000","last_undegraded":"2024-03-14T09:03:00.986410+0000","last_fullsized":"2024-03-14T09:03:00.986410+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982467+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:00.982467+0000","last_peered":"2024-03-14T09:03:00.982467+0000","last_clean":"2024-03-14T09:03:00.982467+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:00.982467+0000","last_undegraded":"2024-03-14T09:03:00.982467+0000","last_fullsized":"2024-03-14T09:03:00.982467+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982614+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:00.982614+0000","last_peered":"2024-03-14T09:03:00.982614+0000","last_clean":"2024-03-14T09:03:00.982614+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:00.982614+0000","last_undegraded":"2024-03-14T09:03:00.982614+0000","last_fullsized":"2024-03-14T09:03:00.982614+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153189+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:01.153189+0000","last_peered":"2024-03-14T09:03:01.153189+0000","last_clean":"2024-03-14T09:03:01.153189+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:01.153189+0000","last_undegraded":"2024-03-14T09:03:01.153189+0000","last_fullsized":"2024-03-14T09:03:01.153189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152921+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:01.152921+0000","last_peered":"2024-03-14T09:03:01.152921+0000","last_clean":"2024-03-14T09:03:01.152921+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:01.152921+0000","last_undegraded":"2024-03-14T09:03:01.152921+0000","last_fullsized":"2024-03-14T09:03:01.152921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152929+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:01.152929+0000","last_peered":"2024-03-14T09:03:01.152929+0000","last_clean":"2024-03-14T09:03:01.152929+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:01.152929+0000","last_undegraded":"2024-03-14T09:03:01.152929+0000","last_fullsized":"2024-03-14T09:03:01.152929+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153034+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:01.153034+0000","last_peered":"2024-03-14T09:03:01.153034+0000","last_clean":"2024-03-14T09:03:01.153034+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:01.153034+0000","last_undegraded":"2024-03-14T09:03:01.153034+0000","last_fullsized":"2024-03-14T09:03:01.153034+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-14T09:02:59.987475+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:02:59.987475+0000","last_peered":"2024-03-14T09:02:59.987475+0000","last_clean":"2024-03-14T09:02:59.987475+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:02:59.987475+0000","last_undegraded":"2024-03-14T09:02:59.987475+0000","last_fullsized":"2024-03-14T09:02:59.987475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1212416,"data_stored":847922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":610304,"data_stored":257394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.495},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":257484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.38}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.68899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1220608,"data_stored":849908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.624},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":257450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.254},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.308}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.498}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":618496,"data_stored":257450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.378},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.311}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.621}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.70699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":622592,"data_stored":258958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":258046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73499999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.752}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.878},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:04.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.580+0000 7f90ac509700 1 Processor -- start 2024-03-14T09:03:04.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.580+0000 7f90ac509700 1 -- start start 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90ac509700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 0x7f90a413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90ac509700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90a40c1780 0x7f90a40c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90ac509700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a413ade0 con 0x7f90a40a8540 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90ac509700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a406fb30 con 0x7f90a40c1780 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90ac509700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a406ec30 con 0x7f90a40c3250 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90aaaa6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90a40c1780 0x7f90a40c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90aaaa6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90a40c1780 0x7f90a40c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39822/0 (socket says 172.21.15.154:39822) 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.581+0000 7f90aaaa6700 1 -- 172.21.15.154:0/473046203 learned_addr learned my addr 172.21.15.154:0/473046203 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90a9aa4700 1 --2- 172.21.15.154:0/473046203 >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 0x7f90a413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90aaaa6700 1 -- 172.21.15.154:0/473046203 >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 msgr2=0x7f90a413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90aaaa6700 1 --2- 172.21.15.154:0/473046203 >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 0x7f90a413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.583 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90aaaa6700 1 -- 172.21.15.154:0/473046203 >> v1:172.21.15.191:6789/0 conn(0x7f90a40a8540 legacy=0x7f90a4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90aaaa6700 1 -- 172.21.15.154:0/473046203 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a406ef10 con 0x7f90a40c1780 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90a9aa4700 1 --2- 172.21.15.154:0/473046203 >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 0x7f90a413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90aaaa6700 1 --2- 172.21.15.154:0/473046203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40c1780 0x7f90a40c1c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f90a40ad180 tx=0x7f909c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f025f148d8f31e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90a92a3700 1 -- 172.21.15.154:0/473046203 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c010070 con 0x7f90a40c1780 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90a92a3700 1 -- 172.21.15.154:0/473046203 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c00dd50 con 0x7f90a40c1780 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.582+0000 7f90a92a3700 1 -- 172.21.15.154:0/473046203 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00b310 con 0x7f90a40c1780 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 -- 172.21.15.154:0/473046203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40c1780 msgr2=0x7f90a40c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 --2- 172.21.15.154:0/473046203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40c1780 0x7f90a40c1c10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f90a40ad180 tx=0x7f909c00d430 comp rx=0 tx=0).stop 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 -- 172.21.15.154:0/473046203 shutdown_connections 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 --2- 172.21.15.154:0/473046203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40c1780 0x7f90a40c1c10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 --2- 172.21.15.154:0/473046203 >> v2:172.21.15.191:3300/0 conn(0x7f90a40c3250 0x7f90a413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 -- 172.21.15.154:0/473046203 >> 172.21.15.154:0/473046203 conn(0x7f90a40c3880 msgr2=0x7f90a40c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 -- 172.21.15.154:0/473046203 shutdown_connections 2024-03-14T09:03:04.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.583+0000 7f90ac509700 1 -- 172.21.15.154:0/473046203 wait complete. 2024-03-14T09:03:04.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.584+0000 7f90ac509700 1 Processor -- start 2024-03-14T09:03:04.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.584+0000 7f90ac509700 1 -- start start 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a40c1780 0x7f90a40ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a40c3250 0x7f90a40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a41346d0 con 0x7f90a40a8540 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a41e97d0 con 0x7f90a40c1780 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90ac509700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40aa7f0 con 0x7f90a40c3250 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39824/0 (socket says 172.21.15.154:39824) 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 -- 172.21.15.154:0/781976389 learned_addr learned my addr 172.21.15.154:0/781976389 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90a9aa4700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a40c1780 0x7f90a40ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aaaa6700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a40c3250 0x7f90a40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 -- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a40c3250 msgr2=0x7f90a40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a40c3250 0x7f90a40ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 -- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a40c1780 msgr2=0x7f90a40ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a40c1780 0x7f90a40ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.585+0000 7f90aa2a5700 1 -- 172.21.15.154:0/781976389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c00bd40 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f90aa2a5700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f909800b4a0 tx=0x7f909800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098011830 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9098011990 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40aa510 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098011d40 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.586+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a41e9180 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.587+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a40697d0 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.587+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f909801a400 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.588+0000 7f9096ffd700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 0x7f908806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.588+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90980889d0 con 0x7f90a40a8540 2024-03-14T09:03:04.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.588+0000 7f90a9aa4700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 0x7f908806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.588+0000 7f90a9aa4700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 0x7f908806ccf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90a40a9f20 tx=0x7f90a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.592+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9098053de0 con 0x7f90a40a8540 2024-03-14T09:03:04.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.783+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90a4155900 con 0x7f908806a830 2024-03-14T09:03:04.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.792+0000 7f9096ffd700 1 -- 172.21.15.154:0/781976389 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f90a000c040 con 0x7f908806a830 2024-03-14T09:03:04.795 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:04.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 msgr2=0x7f908806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 0x7f908806ccf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90a40a9f20 tx=0x7f90a0008040 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 msgr2=0x7f90a4153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f909800b4a0 tx=0x7f909800b7b0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 shutdown_connections 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a40c3250 0x7f90a40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f908806a830 0x7f908806ccf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90a40a8540 0x7f90a4153ed0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 --2- 172.21.15.154:0/781976389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a40c1780 0x7f90a40ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 >> 172.21.15.154:0/781976389 conn(0x7f90a40c3880 msgr2=0x7f90a406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 shutdown_connections 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.795+0000 7f90ac509700 1 -- 172.21.15.154:0/781976389 wait complete. 2024-03-14T09:03:04.798 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:04.807 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-03-14T09:03:03.541978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222232,"kb_used_data":5992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752488,"statfs":{"total":773094113280,"available":772866547712,"internally_reserved":0,"allocated":6135808,"data_stored":3244612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985046"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152708+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:01.152708+0000","last_peered":"2024-03-14T09:03:01.152708+0000","last_clean":"2024-03-14T09:03:01.152708+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:01.152708+0000","last_undegraded":"2024-03-14T09:03:01.152708+0000","last_fullsized":"2024-03-14T09:03:01.152708+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153053+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:01.153053+0000","last_peered":"2024-03-14T09:03:01.153053+0000","last_clean":"2024-03-14T09:03:01.153053+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:01.153053+0000","last_undegraded":"2024-03-14T09:03:01.153053+0000","last_fullsized":"2024-03-14T09:03:01.153053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153111+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:01.153111+0000","last_peered":"2024-03-14T09:03:01.153111+0000","last_clean":"2024-03-14T09:03:01.153111+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:01.153111+0000","last_undegraded":"2024-03-14T09:03:01.153111+0000","last_fullsized":"2024-03-14T09:03:01.153111+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986319+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:00.986319+0000","last_peered":"2024-03-14T09:03:00.986319+0000","last_clean":"2024-03-14T09:03:00.986319+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:00.986319+0000","last_undegraded":"2024-03-14T09:03:00.986319+0000","last_fullsized":"2024-03-14T09:03:00.986319+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152588+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:01.152588+0000","last_peered":"2024-03-14T09:03:01.152588+0000","last_clean":"2024-03-14T09:03:01.152588+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:01.152588+0000","last_undegraded":"2024-03-14T09:03:01.152588+0000","last_fullsized":"2024-03-14T09:03:01.152588+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152125+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:01.152125+0000","last_peered":"2024-03-14T09:03:01.152125+0000","last_clean":"2024-03-14T09:03:01.152125+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:01.152125+0000","last_undegraded":"2024-03-14T09:03:01.152125+0000","last_fullsized":"2024-03-14T09:03:01.152125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986038+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:00.986038+0000","last_peered":"2024-03-14T09:03:00.986038+0000","last_clean":"2024-03-14T09:03:00.986038+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:00.986038+0000","last_undegraded":"2024-03-14T09:03:00.986038+0000","last_fullsized":"2024-03-14T09:03:00.986038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152829+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:01.152829+0000","last_peered":"2024-03-14T09:03:01.152829+0000","last_clean":"2024-03-14T09:03:01.152829+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:01.152829+0000","last_undegraded":"2024-03-14T09:03:01.152829+0000","last_fullsized":"2024-03-14T09:03:01.152829+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986250+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:00.986250+0000","last_peered":"2024-03-14T09:03:00.986250+0000","last_clean":"2024-03-14T09:03:00.986250+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:00.986250+0000","last_undegraded":"2024-03-14T09:03:00.986250+0000","last_fullsized":"2024-03-14T09:03:00.986250+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982948+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:00.982948+0000","last_peered":"2024-03-14T09:03:00.982948+0000","last_clean":"2024-03-14T09:03:00.982948+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:00.982948+0000","last_undegraded":"2024-03-14T09:03:00.982948+0000","last_fullsized":"2024-03-14T09:03:00.982948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982938+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:00.982938+0000","last_peered":"2024-03-14T09:03:00.982938+0000","last_clean":"2024-03-14T09:03:00.982938+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:00.982938+0000","last_undegraded":"2024-03-14T09:03:00.982938+0000","last_fullsized":"2024-03-14T09:03:00.982938+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986419+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:00.986419+0000","last_peered":"2024-03-14T09:03:00.986419+0000","last_clean":"2024-03-14T09:03:00.986419+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:00.986419+0000","last_undegraded":"2024-03-14T09:03:00.986419+0000","last_fullsized":"2024-03-14T09:03:00.986419+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153012+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:01.153012+0000","last_peered":"2024-03-14T09:03:01.153012+0000","last_clean":"2024-03-14T09:03:01.153012+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:01.153012+0000","last_undegraded":"2024-03-14T09:03:01.153012+0000","last_fullsized":"2024-03-14T09:03:01.153012+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986322+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:00.986322+0000","last_peered":"2024-03-14T09:03:00.986322+0000","last_clean":"2024-03-14T09:03:00.986322+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:00.986322+0000","last_undegraded":"2024-03-14T09:03:00.986322+0000","last_fullsized":"2024-03-14T09:03:00.986322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986134+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:00.986134+0000","last_peered":"2024-03-14T09:03:00.986134+0000","last_clean":"2024-03-14T09:03:00.986134+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:00.986134+0000","last_undegraded":"2024-03-14T09:03:00.986134+0000","last_fullsized":"2024-03-14T09:03:00.986134+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152062+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:01.152062+0000","last_peered":"2024-03-14T09:03:01.152062+0000","last_clean":"2024-03-14T09:03:01.152062+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:01.152062+0000","last_undegraded":"2024-03-14T09:03:01.152062+0000","last_fullsized":"2024-03-14T09:03:01.152062+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153102+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:01.153102+0000","last_peered":"2024-03-14T09:03:01.153102+0000","last_clean":"2024-03-14T09:03:01.153102+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:01.153102+0000","last_undegraded":"2024-03-14T09:03:01.153102+0000","last_fullsized":"2024-03-14T09:03:01.153102+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152864+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:01.152864+0000","last_peered":"2024-03-14T09:03:01.152864+0000","last_clean":"2024-03-14T09:03:01.152864+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:01.152864+0000","last_undegraded":"2024-03-14T09:03:01.152864+0000","last_fullsized":"2024-03-14T09:03:01.152864+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152546+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:01.152546+0000","last_peered":"2024-03-14T09:03:01.152546+0000","last_clean":"2024-03-14T09:03:01.152546+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:01.152546+0000","last_undegraded":"2024-03-14T09:03:01.152546+0000","last_fullsized":"2024-03-14T09:03:01.152546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152927+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:01.152927+0000","last_peered":"2024-03-14T09:03:01.152927+0000","last_clean":"2024-03-14T09:03:01.152927+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:01.152927+0000","last_undegraded":"2024-03-14T09:03:01.152927+0000","last_fullsized":"2024-03-14T09:03:01.152927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153645+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:01.153645+0000","last_peered":"2024-03-14T09:03:01.153645+0000","last_clean":"2024-03-14T09:03:01.153645+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:01.153645+0000","last_undegraded":"2024-03-14T09:03:01.153645+0000","last_fullsized":"2024-03-14T09:03:01.153645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152798+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:01.152798+0000","last_peered":"2024-03-14T09:03:01.152798+0000","last_clean":"2024-03-14T09:03:01.152798+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:01.152798+0000","last_undegraded":"2024-03-14T09:03:01.152798+0000","last_fullsized":"2024-03-14T09:03:01.152798+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152630+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:01.152630+0000","last_peered":"2024-03-14T09:03:01.152630+0000","last_clean":"2024-03-14T09:03:01.152630+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:01.152630+0000","last_undegraded":"2024-03-14T09:03:01.152630+0000","last_fullsized":"2024-03-14T09:03:01.152630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153421+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:01.153421+0000","last_peered":"2024-03-14T09:03:01.153421+0000","last_clean":"2024-03-14T09:03:01.153421+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:01.153421+0000","last_undegraded":"2024-03-14T09:03:01.153421+0000","last_fullsized":"2024-03-14T09:03:01.153421+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152756+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:01.152756+0000","last_peered":"2024-03-14T09:03:01.152756+0000","last_clean":"2024-03-14T09:03:01.152756+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:01.152756+0000","last_undegraded":"2024-03-14T09:03:01.152756+0000","last_fullsized":"2024-03-14T09:03:01.152756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986602+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:00.986602+0000","last_peered":"2024-03-14T09:03:00.986602+0000","last_clean":"2024-03-14T09:03:00.986602+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:00.986602+0000","last_undegraded":"2024-03-14T09:03:00.986602+0000","last_fullsized":"2024-03-14T09:03:00.986602+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153336+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:01.153336+0000","last_peered":"2024-03-14T09:03:01.153336+0000","last_clean":"2024-03-14T09:03:01.153336+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:01.153336+0000","last_undegraded":"2024-03-14T09:03:01.153336+0000","last_fullsized":"2024-03-14T09:03:01.153336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152558+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:01.152558+0000","last_peered":"2024-03-14T09:03:01.152558+0000","last_clean":"2024-03-14T09:03:01.152558+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:01.152558+0000","last_undegraded":"2024-03-14T09:03:01.152558+0000","last_fullsized":"2024-03-14T09:03:01.152558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152714+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:01.152714+0000","last_peered":"2024-03-14T09:03:01.152714+0000","last_clean":"2024-03-14T09:03:01.152714+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:01.152714+0000","last_undegraded":"2024-03-14T09:03:01.152714+0000","last_fullsized":"2024-03-14T09:03:01.152714+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152893+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:01.152893+0000","last_peered":"2024-03-14T09:03:01.152893+0000","last_clean":"2024-03-14T09:03:01.152893+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:01.152893+0000","last_undegraded":"2024-03-14T09:03:01.152893+0000","last_fullsized":"2024-03-14T09:03:01.152893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152403+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:01.152403+0000","last_peered":"2024-03-14T09:03:01.152403+0000","last_clean":"2024-03-14T09:03:01.152403+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:01.152403+0000","last_undegraded":"2024-03-14T09:03:01.152403+0000","last_fullsized":"2024-03-14T09:03:01.152403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152129+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:01.152129+0000","last_peered":"2024-03-14T09:03:01.152129+0000","last_clean":"2024-03-14T09:03:01.152129+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:01.152129+0000","last_undegraded":"2024-03-14T09:03:01.152129+0000","last_fullsized":"2024-03-14T09:03:01.152129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153415+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:01.153415+0000","last_peered":"2024-03-14T09:03:01.153415+0000","last_clean":"2024-03-14T09:03:01.153415+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:01.153415+0000","last_undegraded":"2024-03-14T09:03:01.153415+0000","last_fullsized":"2024-03-14T09:03:01.153415+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153519+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:01.153519+0000","last_peered":"2024-03-14T09:03:01.153519+0000","last_clean":"2024-03-14T09:03:01.153519+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:01.153519+0000","last_undegraded":"2024-03-14T09:03:01.153519+0000","last_fullsized":"2024-03-14T09:03:01.153519+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.983059+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:00.983059+0000","last_peered":"2024-03-14T09:03:00.983059+0000","last_clean":"2024-03-14T09:03:00.983059+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:00.983059+0000","last_undegraded":"2024-03-14T09:03:00.983059+0000","last_fullsized":"2024-03-14T09:03:00.983059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153115+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:01.153115+0000","last_peered":"2024-03-14T09:03:01.153115+0000","last_clean":"2024-03-14T09:03:01.153115+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:01.153115+0000","last_undegraded":"2024-03-14T09:03:01.153115+0000","last_fullsized":"2024-03-14T09:03:01.153115+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153272+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:01.153272+0000","last_peered":"2024-03-14T09:03:01.153272+0000","last_clean":"2024-03-14T09:03:01.153272+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:01.153272+0000","last_undegraded":"2024-03-14T09:03:01.153272+0000","last_fullsized":"2024-03-14T09:03:01.153272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153615+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:01.153615+0000","last_peered":"2024-03-14T09:03:01.153615+0000","last_clean":"2024-03-14T09:03:01.153615+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:01.153615+0000","last_undegraded":"2024-03-14T09:03:01.153615+0000","last_fullsized":"2024-03-14T09:03:01.153615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153440+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:01.153440+0000","last_peered":"2024-03-14T09:03:01.153440+0000","last_clean":"2024-03-14T09:03:01.153440+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:01.153440+0000","last_undegraded":"2024-03-14T09:03:01.153440+0000","last_fullsized":"2024-03-14T09:03:01.153440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986536+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:00.986536+0000","last_peered":"2024-03-14T09:03:00.986536+0000","last_clean":"2024-03-14T09:03:00.986536+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:00.986536+0000","last_undegraded":"2024-03-14T09:03:00.986536+0000","last_fullsized":"2024-03-14T09:03:00.986536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153055+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:01.153055+0000","last_peered":"2024-03-14T09:03:01.153055+0000","last_clean":"2024-03-14T09:03:01.153055+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:01.153055+0000","last_undegraded":"2024-03-14T09:03:01.153055+0000","last_fullsized":"2024-03-14T09:03:01.153055+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985798+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:00.985798+0000","last_peered":"2024-03-14T09:03:00.985798+0000","last_clean":"2024-03-14T09:03:00.985798+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:00.985798+0000","last_undegraded":"2024-03-14T09:03:00.985798+0000","last_fullsized":"2024-03-14T09:03:00.985798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152769+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:01.152769+0000","last_peered":"2024-03-14T09:03:01.152769+0000","last_clean":"2024-03-14T09:03:01.152769+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:01.152769+0000","last_undegraded":"2024-03-14T09:03:01.152769+0000","last_fullsized":"2024-03-14T09:03:01.152769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152229+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:01.152229+0000","last_peered":"2024-03-14T09:03:01.152229+0000","last_clean":"2024-03-14T09:03:01.152229+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:01.152229+0000","last_undegraded":"2024-03-14T09:03:01.152229+0000","last_fullsized":"2024-03-14T09:03:01.152229+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153480+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:01.153480+0000","last_peered":"2024-03-14T09:03:01.153480+0000","last_clean":"2024-03-14T09:03:01.153480+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:01.153480+0000","last_undegraded":"2024-03-14T09:03:01.153480+0000","last_fullsized":"2024-03-14T09:03:01.153480+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985830+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:00.985830+0000","last_peered":"2024-03-14T09:03:00.985830+0000","last_clean":"2024-03-14T09:03:00.985830+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:00.985830+0000","last_undegraded":"2024-03-14T09:03:00.985830+0000","last_fullsized":"2024-03-14T09:03:00.985830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153093+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:01.153093+0000","last_peered":"2024-03-14T09:03:01.153093+0000","last_clean":"2024-03-14T09:03:01.153093+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:01.153093+0000","last_undegraded":"2024-03-14T09:03:01.153093+0000","last_fullsized":"2024-03-14T09:03:01.153093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985915+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:00.985915+0000","last_peered":"2024-03-14T09:03:00.985915+0000","last_clean":"2024-03-14T09:03:00.985915+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:00.985915+0000","last_undegraded":"2024-03-14T09:03:00.985915+0000","last_fullsized":"2024-03-14T09:03:00.985915+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986344+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:00.986344+0000","last_peered":"2024-03-14T09:03:00.986344+0000","last_clean":"2024-03-14T09:03:00.986344+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:00.986344+0000","last_undegraded":"2024-03-14T09:03:00.986344+0000","last_fullsized":"2024-03-14T09:03:00.986344+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153263+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:01.153263+0000","last_peered":"2024-03-14T09:03:01.153263+0000","last_clean":"2024-03-14T09:03:01.153263+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:01.153263+0000","last_undegraded":"2024-03-14T09:03:01.153263+0000","last_fullsized":"2024-03-14T09:03:01.153263+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153128+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:01.153128+0000","last_peered":"2024-03-14T09:03:01.153128+0000","last_clean":"2024-03-14T09:03:01.153128+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:01.153128+0000","last_undegraded":"2024-03-14T09:03:01.153128+0000","last_fullsized":"2024-03-14T09:03:01.153128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153184+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:01.153184+0000","last_peered":"2024-03-14T09:03:01.153184+0000","last_clean":"2024-03-14T09:03:01.153184+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:01.153184+0000","last_undegraded":"2024-03-14T09:03:01.153184+0000","last_fullsized":"2024-03-14T09:03:01.153184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982727+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:00.982727+0000","last_peered":"2024-03-14T09:03:00.982727+0000","last_clean":"2024-03-14T09:03:00.982727+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:00.982727+0000","last_undegraded":"2024-03-14T09:03:00.982727+0000","last_fullsized":"2024-03-14T09:03:00.982727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153352+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:01.153352+0000","last_peered":"2024-03-14T09:03:01.153352+0000","last_clean":"2024-03-14T09:03:01.153352+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:01.153352+0000","last_undegraded":"2024-03-14T09:03:01.153352+0000","last_fullsized":"2024-03-14T09:03:01.153352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153228+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:01.153228+0000","last_peered":"2024-03-14T09:03:01.153228+0000","last_clean":"2024-03-14T09:03:01.153228+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:01.153228+0000","last_undegraded":"2024-03-14T09:03:01.153228+0000","last_fullsized":"2024-03-14T09:03:01.153228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986521+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:00.986521+0000","last_peered":"2024-03-14T09:03:00.986521+0000","last_clean":"2024-03-14T09:03:00.986521+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:00.986521+0000","last_undegraded":"2024-03-14T09:03:00.986521+0000","last_fullsized":"2024-03-14T09:03:00.986521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985655+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:00.985655+0000","last_peered":"2024-03-14T09:03:00.985655+0000","last_clean":"2024-03-14T09:03:00.985655+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:00.985655+0000","last_undegraded":"2024-03-14T09:03:00.985655+0000","last_fullsized":"2024-03-14T09:03:00.985655+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152884+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:01.152884+0000","last_peered":"2024-03-14T09:03:01.152884+0000","last_clean":"2024-03-14T09:03:01.152884+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:01.152884+0000","last_undegraded":"2024-03-14T09:03:01.152884+0000","last_fullsized":"2024-03-14T09:03:01.152884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153743+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:01.153743+0000","last_peered":"2024-03-14T09:03:01.153743+0000","last_clean":"2024-03-14T09:03:01.153743+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:01.153743+0000","last_undegraded":"2024-03-14T09:03:01.153743+0000","last_fullsized":"2024-03-14T09:03:01.153743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152948+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:01.152948+0000","last_peered":"2024-03-14T09:03:01.152948+0000","last_clean":"2024-03-14T09:03:01.152948+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:01.152948+0000","last_undegraded":"2024-03-14T09:03:01.152948+0000","last_fullsized":"2024-03-14T09:03:01.152948+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985848+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:00.985848+0000","last_peered":"2024-03-14T09:03:00.985848+0000","last_clean":"2024-03-14T09:03:00.985848+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:00.985848+0000","last_undegraded":"2024-03-14T09:03:00.985848+0000","last_fullsized":"2024-03-14T09:03:00.985848+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985929+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:00.985929+0000","last_peered":"2024-03-14T09:03:00.985929+0000","last_clean":"2024-03-14T09:03:00.985929+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:00.985929+0000","last_undegraded":"2024-03-14T09:03:00.985929+0000","last_fullsized":"2024-03-14T09:03:00.985929+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152751+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:01.152751+0000","last_peered":"2024-03-14T09:03:01.152751+0000","last_clean":"2024-03-14T09:03:01.152751+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:01.152751+0000","last_undegraded":"2024-03-14T09:03:01.152751+0000","last_fullsized":"2024-03-14T09:03:01.152751+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":19,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152996+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:01.152996+0000","last_peered":"2024-03-14T09:03:01.152996+0000","last_clean":"2024-03-14T09:03:01.152996+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:01.152996+0000","last_undegraded":"2024-03-14T09:03:01.152996+0000","last_fullsized":"2024-03-14T09:03:01.152996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153141+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:01.153141+0000","last_peered":"2024-03-14T09:03:01.153141+0000","last_clean":"2024-03-14T09:03:01.153141+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:01.153141+0000","last_undegraded":"2024-03-14T09:03:01.153141+0000","last_fullsized":"2024-03-14T09:03:01.153141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153179+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:01.153179+0000","last_peered":"2024-03-14T09:03:01.153179+0000","last_clean":"2024-03-14T09:03:01.153179+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:01.153179+0000","last_undegraded":"2024-03-14T09:03:01.153179+0000","last_fullsized":"2024-03-14T09:03:01.153179+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986237+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:00.986237+0000","last_peered":"2024-03-14T09:03:00.986237+0000","last_clean":"2024-03-14T09:03:00.986237+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:00.986237+0000","last_undegraded":"2024-03-14T09:03:00.986237+0000","last_fullsized":"2024-03-14T09:03:00.986237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153439+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:01.153439+0000","last_peered":"2024-03-14T09:03:01.153439+0000","last_clean":"2024-03-14T09:03:01.153439+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:01.153439+0000","last_undegraded":"2024-03-14T09:03:01.153439+0000","last_fullsized":"2024-03-14T09:03:01.153439+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153048+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:01.153048+0000","last_peered":"2024-03-14T09:03:01.153048+0000","last_clean":"2024-03-14T09:03:01.153048+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:01.153048+0000","last_undegraded":"2024-03-14T09:03:01.153048+0000","last_fullsized":"2024-03-14T09:03:01.153048+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986629+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:00.986629+0000","last_peered":"2024-03-14T09:03:00.986629+0000","last_clean":"2024-03-14T09:03:00.986629+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:00.986629+0000","last_undegraded":"2024-03-14T09:03:00.986629+0000","last_fullsized":"2024-03-14T09:03:00.986629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153308+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:01.153308+0000","last_peered":"2024-03-14T09:03:01.153308+0000","last_clean":"2024-03-14T09:03:01.153308+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:01.153308+0000","last_undegraded":"2024-03-14T09:03:01.153308+0000","last_fullsized":"2024-03-14T09:03:01.153308+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153336+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:01.153336+0000","last_peered":"2024-03-14T09:03:01.153336+0000","last_clean":"2024-03-14T09:03:01.153336+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:01.153336+0000","last_undegraded":"2024-03-14T09:03:01.153336+0000","last_fullsized":"2024-03-14T09:03:01.153336+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985762+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:00.985762+0000","last_peered":"2024-03-14T09:03:00.985762+0000","last_clean":"2024-03-14T09:03:00.985762+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:00.985762+0000","last_undegraded":"2024-03-14T09:03:00.985762+0000","last_fullsized":"2024-03-14T09:03:00.985762+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152927+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:01.152927+0000","last_peered":"2024-03-14T09:03:01.152927+0000","last_clean":"2024-03-14T09:03:01.152927+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:01.152927+0000","last_undegraded":"2024-03-14T09:03:01.152927+0000","last_fullsized":"2024-03-14T09:03:01.152927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152339+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:01.152339+0000","last_peered":"2024-03-14T09:03:01.152339+0000","last_clean":"2024-03-14T09:03:01.152339+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:01.152339+0000","last_undegraded":"2024-03-14T09:03:01.152339+0000","last_fullsized":"2024-03-14T09:03:01.152339+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982736+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:00.982736+0000","last_peered":"2024-03-14T09:03:00.982736+0000","last_clean":"2024-03-14T09:03:00.982736+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:00.982736+0000","last_undegraded":"2024-03-14T09:03:00.982736+0000","last_fullsized":"2024-03-14T09:03:00.982736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152100+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:01.152100+0000","last_peered":"2024-03-14T09:03:01.152100+0000","last_clean":"2024-03-14T09:03:01.152100+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:01.152100+0000","last_undegraded":"2024-03-14T09:03:01.152100+0000","last_fullsized":"2024-03-14T09:03:01.152100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.985618+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:00.985618+0000","last_peered":"2024-03-14T09:03:00.985618+0000","last_clean":"2024-03-14T09:03:00.985618+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:00.985618+0000","last_undegraded":"2024-03-14T09:03:00.985618+0000","last_fullsized":"2024-03-14T09:03:00.985618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986706+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:00.986706+0000","last_peered":"2024-03-14T09:03:00.986706+0000","last_clean":"2024-03-14T09:03:00.986706+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:00.986706+0000","last_undegraded":"2024-03-14T09:03:00.986706+0000","last_fullsized":"2024-03-14T09:03:00.986706+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":13,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.983029+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:00.983029+0000","last_peered":"2024-03-14T09:03:00.983029+0000","last_clean":"2024-03-14T09:03:00.983029+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:00.983029+0000","last_undegraded":"2024-03-14T09:03:00.983029+0000","last_fullsized":"2024-03-14T09:03:00.983029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986059+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:00.986059+0000","last_peered":"2024-03-14T09:03:00.986059+0000","last_clean":"2024-03-14T09:03:00.986059+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:00.986059+0000","last_undegraded":"2024-03-14T09:03:00.986059+0000","last_fullsized":"2024-03-14T09:03:00.986059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153461+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:01.153461+0000","last_peered":"2024-03-14T09:03:01.153461+0000","last_clean":"2024-03-14T09:03:01.153461+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:01.153461+0000","last_undegraded":"2024-03-14T09:03:01.153461+0000","last_fullsized":"2024-03-14T09:03:01.153461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982784+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:00.982784+0000","last_peered":"2024-03-14T09:03:00.982784+0000","last_clean":"2024-03-14T09:03:00.982784+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:00.982784+0000","last_undegraded":"2024-03-14T09:03:00.982784+0000","last_fullsized":"2024-03-14T09:03:00.982784+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153275+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:01.153275+0000","last_peered":"2024-03-14T09:03:01.153275+0000","last_clean":"2024-03-14T09:03:01.153275+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:01.153275+0000","last_undegraded":"2024-03-14T09:03:01.153275+0000","last_fullsized":"2024-03-14T09:03:01.153275+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153534+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:01.153534+0000","last_peered":"2024-03-14T09:03:01.153534+0000","last_clean":"2024-03-14T09:03:01.153534+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:01.153534+0000","last_undegraded":"2024-03-14T09:03:01.153534+0000","last_fullsized":"2024-03-14T09:03:01.153534+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986710+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:00.986710+0000","last_peered":"2024-03-14T09:03:00.986710+0000","last_clean":"2024-03-14T09:03:00.986710+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:00.986710+0000","last_undegraded":"2024-03-14T09:03:00.986710+0000","last_fullsized":"2024-03-14T09:03:00.986710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153259+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:01.153259+0000","last_peered":"2024-03-14T09:03:01.153259+0000","last_clean":"2024-03-14T09:03:01.153259+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:01.153259+0000","last_undegraded":"2024-03-14T09:03:01.153259+0000","last_fullsized":"2024-03-14T09:03:01.153259+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982681+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:00.982681+0000","last_peered":"2024-03-14T09:03:00.982681+0000","last_clean":"2024-03-14T09:03:00.982681+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:00.982681+0000","last_undegraded":"2024-03-14T09:03:00.982681+0000","last_fullsized":"2024-03-14T09:03:00.982681+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153839+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:01.153839+0000","last_peered":"2024-03-14T09:03:01.153839+0000","last_clean":"2024-03-14T09:03:01.153839+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:01.153839+0000","last_undegraded":"2024-03-14T09:03:01.153839+0000","last_fullsized":"2024-03-14T09:03:01.153839+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.986410+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:00.986410+0000","last_peered":"2024-03-14T09:03:00.986410+0000","last_clean":"2024-03-14T09:03:00.986410+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:00.986410+0000","last_undegraded":"2024-03-14T09:03:00.986410+0000","last_fullsized":"2024-03-14T09:03:00.986410+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982467+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:00.982467+0000","last_peered":"2024-03-14T09:03:00.982467+0000","last_clean":"2024-03-14T09:03:00.982467+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:00.982467+0000","last_undegraded":"2024-03-14T09:03:00.982467+0000","last_fullsized":"2024-03-14T09:03:00.982467+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:00.982614+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:00.982614+0000","last_peered":"2024-03-14T09:03:00.982614+0000","last_clean":"2024-03-14T09:03:00.982614+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:00.982614+0000","last_undegraded":"2024-03-14T09:03:00.982614+0000","last_fullsized":"2024-03-14T09:03:00.982614+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153189+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:01.153189+0000","last_peered":"2024-03-14T09:03:01.153189+0000","last_clean":"2024-03-14T09:03:01.153189+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:01.153189+0000","last_undegraded":"2024-03-14T09:03:01.153189+0000","last_fullsized":"2024-03-14T09:03:01.153189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152921+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:01.152921+0000","last_peered":"2024-03-14T09:03:01.152921+0000","last_clean":"2024-03-14T09:03:01.152921+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:01.152921+0000","last_undegraded":"2024-03-14T09:03:01.152921+0000","last_fullsized":"2024-03-14T09:03:01.152921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.152929+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:01.152929+0000","last_peered":"2024-03-14T09:03:01.152929+0000","last_clean":"2024-03-14T09:03:01.152929+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:01.152929+0000","last_undegraded":"2024-03-14T09:03:01.152929+0000","last_fullsized":"2024-03-14T09:03:01.152929+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-14T09:03:01.153034+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:01.153034+0000","last_peered":"2024-03-14T09:03:01.153034+0000","last_clean":"2024-03-14T09:03:01.153034+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:01.153034+0000","last_undegraded":"2024-03-14T09:03:01.153034+0000","last_fullsized":"2024-03-14T09:03:01.153034+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-03-14T09:02:59.987475+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:02:59.987475+0000","last_peered":"2024-03-14T09:02:59.987475+0000","last_clean":"2024-03-14T09:02:59.987475+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:02:59.987475+0000","last_undegraded":"2024-03-14T09:02:59.987475+0000","last_fullsized":"2024-03-14T09:02:59.987475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1212416,"data_stored":847922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.434},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":610304,"data_stored":257394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.495},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":257484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.38}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.68899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1220608,"data_stored":849908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.624},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":257450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.254},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.308}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.32600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.498}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.50800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":618496,"data_stored":257450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.378},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.311}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.621}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.70699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":622592,"data_stored":258958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.502}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":258046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73499999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.752}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.878},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:04.811 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:03:04.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T09:03:04.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.977+0000 7fbd9e6bc700 1 Processor -- start 2024-03-14T09:03:04.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.977+0000 7fbd9e6bc700 1 -- start start 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9e6bc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9814eaf0 0x7fbd9806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9e6bc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd9806acb0 0x7fbd98155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9e6bc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd980ac8e0 con 0x7fbd98156330 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9e6bc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9806b310 con 0x7fbd9806acb0 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9e6bc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9806de30 con 0x7fbd9814eaf0 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbd9806acb0 0x7fbd98155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9cc59700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbd98156330 0x7fbd98157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55108/0 (socket says 172.21.15.154:55108) 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd9cc59700 1 -- 172.21.15.154:0/3757777104 learned_addr learned my addr 172.21.15.154:0/3757777104 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 -- 172.21.15.154:0/3757777104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9814eaf0 msgr2=0x7fbd9806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 --2- 172.21.15.154:0/3757777104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9814eaf0 0x7fbd9806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 -- 172.21.15.154:0/3757777104 >> v1:172.21.15.154:6789/0 conn(0x7fbd98156330 legacy=0x7fbd98157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 -- 172.21.15.154:0/3757777104 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980acb30 con 0x7fbd9806acb0 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd96ffd700 1 -- 172.21.15.154:0/3757777104 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049826766 0 0) 0x7fbd980ac8e0 con 0x7fbd98156330 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd977fe700 1 --2- 172.21.15.154:0/3757777104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9806acb0 0x7fbd98155df0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009840 tx=0x7fbd8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d4fa238d911fca server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.978+0000 7fbd96ffd700 1 -- 172.21.15.154:0/3757777104 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c011830 con 0x7fbd9806acb0 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd96ffd700 1 -- 172.21.15.154:0/3757777104 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c011990 con 0x7fbd9806acb0 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd96ffd700 1 -- 172.21.15.154:0/3757777104 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c011d20 con 0x7fbd9806acb0 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3757777104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9806acb0 msgr2=0x7fbd98155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3757777104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9806acb0 0x7fbd98155df0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009840 tx=0x7fbd8c009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3757777104 shutdown_connections 2024-03-14T09:03:04.980 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3757777104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd9814eaf0 0x7fbd9806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3757777104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9806acb0 0x7fbd98155df0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3757777104 >> 172.21.15.154:0/3757777104 conn(0x7fbd980c3880 msgr2=0x7fbd980c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:04.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3757777104 shutdown_connections 2024-03-14T09:03:04.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.979+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3757777104 wait complete. 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.980+0000 7fbd9e6bc700 1 Processor -- start 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.980+0000 7fbd9e6bc700 1 -- start start 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9806acb0 0x7fbd981ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 0x7fbd98153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980c2480 con 0x7fbd9814eaf0 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980c1ec0 con 0x7fbd9806acb0 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9e6bc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980c21a0 con 0x7fbd98156330 2024-03-14T09:03:04.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39842/0 (socket says 172.21.15.154:39842) 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 -- 172.21.15.154:0/3197784951 learned_addr learned my addr 172.21.15.154:0/3197784951 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd97fff700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9806acb0 0x7fbd981ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9cc59700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 0x7fbd98153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 -- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 msgr2=0x7fbd98153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 0x7fbd98153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 -- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9806acb0 msgr2=0x7fbd981ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9806acb0 0x7fbd981ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd977fe700 1 -- 172.21.15.154:0/3197784951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0094a0 con 0x7fbd9814eaf0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.981+0000 7fbd9cc59700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 0x7fbd98153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd977fe700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000c00 tx=0x7fbd8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0036a0 con 0x7fbd9814eaf0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c01f070 con 0x7fbd9814eaf0 2024-03-14T09:03:04.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd98153300 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c003a60 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.982+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9814a7c0 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.983+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd980697d0 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.983+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbd8c003bc0 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.983+0000 7fbd94ff9700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 0x7fbd8406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.984+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd8c021070 con 0x7fbd9814eaf0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.984+0000 7fbd97fff700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 0x7fbd8406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:04.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.984+0000 7fbd97fff700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 0x7fbd8406ccf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806f180 tx=0x7fbd88009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:04.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:04.988+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbd8c055ba0 con 0x7fbd9814eaf0 2024-03-14T09:03:05.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.261+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbd9806b470 con 0x7fbd9814eaf0 2024-03-14T09:03:05.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.265+0000 7fbd94ff9700 1 -- 172.21.15.154:0/3197784951 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbd8c015020 con 0x7fbd9814eaf0 2024-03-14T09:03:05.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 msgr2=0x7fbd8406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 0x7fbd8406ccf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806f180 tx=0x7fbd88009190 comp rx=0 tx=0).stop 2024-03-14T09:03:05.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 msgr2=0x7fbd9814a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000c00 tx=0x7fbd8c004750 comp rx=0 tx=0).stop 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 shutdown_connections 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd98156330 0x7fbd98153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd9806acb0 0x7fbd981ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd8406a830 0x7fbd8406ccf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 --2- 172.21.15.154:0/3197784951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd9814eaf0 0x7fbd9814a280 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 >> 172.21.15.154:0/3197784951 conn(0x7fbd980c3880 msgr2=0x7fbd98157090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 shutdown_connections 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.267+0000 7fbd9e6bc700 1 -- 172.21.15.154:0/3197784951 wait complete. 2024-03-14T09:03:05.268 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2024-03-14T09:03:05.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:05.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.402+0000 7f98a558e700 1 Processor -- start 2024-03-14T09:03:05.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.402+0000 7f98a558e700 1 -- start start 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f98a558e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f98a00c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f98a558e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f98a00c4f30 0x7f98a015f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f98a558e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a0158240 con 0x7f98a00c7f70 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f98a558e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a014ed10 con 0x7f98a00c4f30 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f98a558e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cad00 con 0x7f98a01586d0 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989effd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f98a00c7f70 0x7f98a0155230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55124/0 (socket says 172.21.15.154:55124) 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989effd700 1 -- 172.21.15.154:0/2457486731 learned_addr learned my addr 172.21.15.154:0/2457486731 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989e7fc700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f98a00c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989f7fe700 1 --2- 172.21.15.154:0/2457486731 >> v2:172.21.15.191:3300/0 conn(0x7f98a00c4f30 0x7f98a015f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047373891 0 0) 0x7f98a0158240 con 0x7f98a00c7f70 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98880035b0 con 0x7f98a00c7f70 2024-03-14T09:03:05.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544945609 0 0) 0x7f98880035b0 con 0x7f98a00c7f70 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 msgr2=0x7f98a00c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f98a00c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 >> v2:172.21.15.191:3300/0 conn(0x7f98a00c4f30 msgr2=0x7f98a015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 --2- 172.21.15.154:0/2457486731 >> v2:172.21.15.191:3300/0 conn(0x7f98a00c4f30 0x7f98a015f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00ca740 con 0x7f98a00c7f70 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9890001670 con 0x7f98a00c7f70 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 >> v1:172.21.15.154:6789/0 conn(0x7f98a00c7f70 legacy=0x7f98a0155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9888003a00 0x7f9888003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f9888004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9888004d90 0x7f9888005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a014ed10 con 0x7f9888004d90 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cad00 con 0x7f9888003a00 2024-03-14T09:03:05.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.403+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988800ba90 con 0x7f98a01586d0 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9888004d90 0x7f9888005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989e7fc700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f9888004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989effd700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9888003a00 0x7f9888003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 -- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 msgr2=0x7f9888004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a01586d0 0x7f9888004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 -- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9888003a00 msgr2=0x7f9888003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9888003a00 0x7f9888003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 -- 172.21.15.154:0/2457486731 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880035b0 con 0x7f9888004d90 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989f7fe700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9888004d90 0x7f9888005210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9898009fc0 tx=0x7f9888005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6ca399af22390c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800ec30 con 0x7f9888004d90 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800f040 con 0x7f9888004d90 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f989dffb700 1 -- 172.21.15.154:0/2457486731 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898014480 con 0x7f9888004d90 2024-03-14T09:03:05.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f98a558e700 1 -- 172.21.15.154:0/2457486731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9888004d90 msgr2=0x7f9888005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.404+0000 7f98a558e700 1 --2- 172.21.15.154:0/2457486731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9888004d90 0x7f9888005210 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9898009fc0 tx=0x7f9888005880 comp rx=0 tx=0).stop 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.405+0000 7f989f7fe700 1 -- 172.21.15.154:0/2457486731 reap_dead start 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.405+0000 7f98a558e700 1 -- 172.21.15.154:0/2457486731 shutdown_connections 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.405+0000 7f98a558e700 1 -- 172.21.15.154:0/2457486731 >> 172.21.15.154:0/2457486731 conn(0x7f98a0072080 msgr2=0x7f98a0072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.405+0000 7f98a558e700 1 -- 172.21.15.154:0/2457486731 shutdown_connections 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.405+0000 7f98a558e700 1 -- 172.21.15.154:0/2457486731 wait complete. 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.406+0000 7f98a558e700 1 Processor -- start 2024-03-14T09:03:05.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.406+0000 7f98a558e700 1 -- start start 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 0x7f98a00acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a01ed400 0x7f98a01ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00af9c0 con 0x7f98a0146ba0 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00afc10 con 0x7f98a01ed400 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00aeaf0 con 0x7f98a0146f80 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39862/0 (socket says 172.21.15.154:39862) 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 -- 172.21.15.154:0/1837420098 learned_addr learned my addr 172.21.15.154:0/1837420098 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989e7fc700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 0x7f98a00acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989f7fe700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a01ed400 0x7f98a01ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 -- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 msgr2=0x7f98a00acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 0x7f98a00acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 -- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a01ed400 msgr2=0x7f98a01ed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a01ed400 0x7f98a01ed890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 -- 172.21.15.154:0/1837420098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00ad7d0 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989e7fc700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 0x7f98a00acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f989effd700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98900017a0 tx=0x7f98900076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900077c0 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989000f8d0 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a00ad1d0 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000fc80 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.407+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a00a9d00 con 0x7f98a0146ba0 2024-03-14T09:03:05.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.408+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a00697d0 con 0x7f98a0146ba0 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.409+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f989001d460 con 0x7f98a0146ba0 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.409+0000 7f98877fe700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 0x7f988c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.409+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9890088da0 con 0x7f98a0146ba0 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.409+0000 7f989e7fc700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 0x7f988c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.410+0000 7f989e7fc700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 0x7f988c06ccf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98a00acc10 tx=0x7f9894008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:05.414 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.414+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9890057ce0 con 0x7f98a0146ba0 2024-03-14T09:03:05.548 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-03-14T09:03:05.548+0000 7fe683d2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:03:05.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.652+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98a006cc40 con 0x7f98a0146ba0 2024-03-14T09:03:05.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.652+0000 7f98877fe700 1 -- 172.21.15.154:0/1837420098 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4746 (secure 0 0 0) 0x7f9890057870 con 0x7f98a0146ba0 2024-03-14T09:03:05.652 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:05.652 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-14T09:03:05.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 msgr2=0x7f988c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 0x7f988c06ccf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f98a00acc10 tx=0x7f9894008040 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 msgr2=0x7f98a00ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98900017a0 tx=0x7f98900076a0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 shutdown_connections 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98a01ed400 0x7f98a01ed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f988c06a830 0x7f988c06ccf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98a0146ba0 0x7f98a00ae5b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 --2- 172.21.15.154:0/1837420098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98a0146f80 0x7f98a00acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 >> 172.21.15.154:0/1837420098 conn(0x7f98a0072080 msgr2=0x7f98a0156280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 shutdown_connections 2024-03-14T09:03:05.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:05.654+0000 7f98a558e700 1 -- 172.21.15.154:0/1837420098 wait complete. 2024-03-14T09:03:05.656 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 30 2024-03-14T09:03:06.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T09:02:59.971830+0000', 'modified': '2024-03-14T09:03:03.317866+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5727': {'gid': 5727, 'name': 'b', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6837/1333349284', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1333349284}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1333349284}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T09:03:06.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T09:03:06.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-14T09:03:06.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.192+0000 7f3859197700 1 Processor -- start 2024-03-14T09:03:06.192 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.192+0000 7f3859197700 1 -- start start 2024-03-14T09:03:06.194 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f3859197700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f385414e550 0x7f3854150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f3859197700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3854151400 0x7f385415b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f3859197700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385406d940 con 0x7f385406c740 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f3859197700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385414e9d0 con 0x7f385414e550 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f3859197700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38540aa800 con 0x7f3854151400 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f385259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f385414e550 0x7f3854150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f385259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f385414e550 0x7f3854150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39878/0 (socket says 172.21.15.154:39878) 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f385259c700 1 -- 172.21.15.154:0/2729815994 learned_addr learned my addr 172.21.15.154:0/2729815994 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.193+0000 7f385359e700 1 --2- 172.21.15.154:0/2729815994 >> v2:172.21.15.191:3300/0 conn(0x7f3854151400 0x7f385415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f385259c700 1 -- 172.21.15.154:0/2729815994 >> v2:172.21.15.191:3300/0 conn(0x7f3854151400 msgr2=0x7f385415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f385259c700 1 --2- 172.21.15.154:0/2729815994 >> v2:172.21.15.191:3300/0 conn(0x7f3854151400 0x7f385415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f385259c700 1 -- 172.21.15.154:0/2729815994 >> v1:172.21.15.191:6789/0 conn(0x7f385406c740 legacy=0x7f38540c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.195 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f385259c700 1 -- 172.21.15.154:0/2729815994 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385406cd30 con 0x7f385414e550 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f385259c700 1 --2- 172.21.15.154:0/2729815994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385414e550 0x7f3854150ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f385406bd70 tx=0x7f384800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59889f65cb10a7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3851d9b700 1 -- 172.21.15.154:0/2729815994 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800dc70 con 0x7f385414e550 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3851d9b700 1 -- 172.21.15.154:0/2729815994 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384800ddd0 con 0x7f385414e550 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3851d9b700 1 -- 172.21.15.154:0/2729815994 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848010690 con 0x7f385414e550 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 -- 172.21.15.154:0/2729815994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385414e550 msgr2=0x7f3854150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 --2- 172.21.15.154:0/2729815994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385414e550 0x7f3854150ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f385406bd70 tx=0x7f384800be70 comp rx=0 tx=0).stop 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 -- 172.21.15.154:0/2729815994 shutdown_connections 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 --2- 172.21.15.154:0/2729815994 >> v2:172.21.15.191:3300/0 conn(0x7f3854151400 0x7f385415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 --2- 172.21.15.154:0/2729815994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385414e550 0x7f3854150ec0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.194+0000 7f3859197700 1 -- 172.21.15.154:0/2729815994 >> 172.21.15.154:0/2729815994 conn(0x7f38540c5e70 msgr2=0x7f38540c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.195+0000 7f3859197700 1 -- 172.21.15.154:0/2729815994 shutdown_connections 2024-03-14T09:03:06.196 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.195+0000 7f3859197700 1 -- 172.21.15.154:0/2729815994 wait complete. 2024-03-14T09:03:06.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.196+0000 7f3859197700 1 Processor -- start 2024-03-14T09:03:06.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.196+0000 7f3859197700 1 -- start start 2024-03-14T09:03:06.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f385414e550 0x7f38541e6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 0x7f38541dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541e6b80 con 0x7f385406c740 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541dbcc0 con 0x7f3854151400 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38541da690 con 0x7f385414e550 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39880/0 (socket says 172.21.15.154:39880) 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 -- 172.21.15.154:0/2364487701 learned_addr learned my addr 172.21.15.154:0/2364487701 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f385359e700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 0x7f38541dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 -- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f385414e550 msgr2=0x7f38541e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f385414e550 0x7f38541e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 -- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 msgr2=0x7f38541dcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 0x7f38541dcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 -- 172.21.15.154:0/2364487701 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384800b7f0 con 0x7f385406c740 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f385359e700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 0x7f38541dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:06.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3852d9d700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f384400c450 tx=0x7f384400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f383b7fe700 1 -- 172.21.15.154:0/2364487701 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38440052d0 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f383b7fe700 1 -- 172.21.15.154:0/2364487701 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844005430 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f383b7fe700 1 -- 172.21.15.154:0/2364487701 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38440038c0 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38541da970 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.197+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38541dd460 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.199+0000 7f383b7fe700 1 -- 172.21.15.154:0/2364487701 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3844003a20 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.199+0000 7f383b7fe700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 0x7f384006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.199+0000 7f383b7fe700 1 -- 172.21.15.154:0/2364487701 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3844088080 con 0x7f385406c740 2024-03-14T09:03:06.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.200+0000 7f385259c700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 0x7f384006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.200+0000 7f385259c700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 0x7f384006ccf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3854150bb0 tx=0x7f3848012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.201+0000 7f3865350b80 1 Processor -- start 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.201+0000 7f3865350b80 1 -- start start 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 0x55c48059d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c48059d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4804d7940 con 0x55c4805b7180 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4804d5300 con 0x55c48057dfe0 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4804d0bf0 con 0x55c48057e3c0 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f385359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f385359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39886/0 (socket says 172.21.15.154:39886) 2024-03-14T09:03:06.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f385359e700 1 -- 172.21.15.154:0/1078798514 learned_addr learned my addr 172.21.15.154:0/1078798514 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f385259c700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c48059d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f3852d9d700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 0x55c48059d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.202+0000 7f385359e700 1 -- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 msgr2=0x55c48059d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f385359e700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c48059d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f385359e700 1 -- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 msgr2=0x55c48059d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f385359e700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 0x55c48059d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f385359e700 1 -- 172.21.15.154:0/1078798514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c480352360 con 0x55c4805b7180 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3852d9d700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 0x55c48059d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f385359e700 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f384c007520 tx=0x7f384c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70baf305c0680d9d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3838ff9700 1 -- 172.21.15.154:0/1078798514 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c015670 con 0x55c4805b7180 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3838ff9700 1 -- 172.21.15.154:0/1078798514 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00f050 con 0x55c4805b7180 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3838ff9700 1 -- 172.21.15.154:0/1078798514 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0159d0 con 0x55c4805b7180 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3865350b80 1 -- 172.21.15.154:0/1078798514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 msgr2=0x55c4805b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.203+0000 7f3865350b80 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f384c007520 tx=0x7f384c009750 comp rx=0 tx=0).stop 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 -- 172.21.15.154:0/1078798514 shutdown_connections 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c4805b7180 0x55c4805b7600 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057dfe0 0x55c48059d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 --2- 172.21.15.154:0/1078798514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c48059d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 -- 172.21.15.154:0/1078798514 >> 172.21.15.154:0/1078798514 conn(0x55c4804f41d0 msgr2=0x55c48057ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 -- 172.21.15.154:0/1078798514 shutdown_connections 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 -- 172.21.15.154:0/1078798514 wait complete. 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 -- start start 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.204+0000 7f3865350b80 1 Processor -- start 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057e3c0 0x55c4805be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c4805b7180 0x55c4805bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48046cd30 con 0x55c48057dfe0 2024-03-14T09:03:06.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c48046c9d0 con 0x55c48057e3c0 2024-03-14T09:03:06.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c480354650 con 0x55c4805b7180 2024-03-14T09:03:06.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3865350b80 10 client.0 _ll_register_callbacks cb 0x55c480476420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:03:06.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.212 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39898/0 (socket says 172.21.15.154:39898) 2024-03-14T09:03:06.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 -- 172.21.15.154:0/1222760090 learned_addr learned my addr 172.21.15.154:0/1222760090 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f385259c700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057e3c0 0x55c4805be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.213 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f385359e700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c4805b7180 0x55c4805bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 -- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c4805b7180 msgr2=0x55c4805bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c4805b7180 0x55c4805bc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 -- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057e3c0 msgr2=0x55c4805be9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057e3c0 0x55c4805be9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 -- 172.21.15.154:0/1222760090 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c009df0 con 0x55c48057dfe0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3852d9d700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f38440536f0 tx=0x7f38440571f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af764fbac3027e3a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.205+0000 7f3831ffb700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384405d2e0 con 0x55c48057dfe0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3844081e30 con 0x55c48057dfe0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c480571450 con 0x55c48057dfe0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f384405b080 con 0x55c48057dfe0 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 10 client.5868 fetch_fsmap learned FSMap version 30 2024-03-14T09:03:06.214 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55c4802feb10 con 0x55c48057dfe0 2024-03-14T09:03:06.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c480571450 con 0x55c48057dfe0 2024-03-14T09:03:06.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f38440566e0 con 0x55c48057dfe0 2024-03-14T09:03:06.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 10 client.5868 fetch_fsmap finished waiting for FSMap version 30 2024-03-14T09:03:06.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 10 client.5868 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T09:03:06.215 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 20 client.5868 populate_metadata read hostname 'smithi154' 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c4805ce8e0 0x55c4805cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 4 client.5868 mds_command: new command op to 5727 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 --> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4804c96b0 con 0x55c4805ce8e0 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.206+0000 7f385259c700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c4805ce8e0 0x55c4805cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3831ffb700 10 client.5868 ms_handle_connect on v2:172.21.15.191:6836/1333349284 2024-03-14T09:03:06.216 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f385259c700 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c4805ce8e0 0x55c4805cecc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f8d1af613c24b8 server_cookie=68ca140eb2aeeb6c in_seq=0 out_seq=0 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3831ffb700 1 -- 172.21.15.154:0/1222760090 <== mds.0 v2:172.21.15.191:6836/1333349284 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c4804c96b0 con 0x55c4805ce8e0 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3831ffb700 10 client.5868 handle_command_reply: tid=0 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3865350b80 1 client.5868 shutdown 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 msgr2=0x55c48057d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f38440536f0 tx=0x7f38440571f0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 shutdown_connections 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.207+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c4805b7180 0x55c4805bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c48057d210 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c4805ce8e0 0x55c4805cecc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 --2- 172.21.15.154:0/1222760090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c48057e3c0 0x55c4805be9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 >> 172.21.15.154:0/1222760090 conn(0x55c4804f41d0 msgr2=0x55c4805b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3831ffb700 0 client.5868 ms_handle_reset on v2:172.21.15.191:6836/1333349284 2024-03-14T09:03:06.217 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 shutdown_connections 2024-03-14T09:03:06.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 1 -- 172.21.15.154:0/1222760090 wait complete. 2024-03-14T09:03:06.218 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.208+0000 7f3865350b80 20 client.5868 trim_cache size 0 max 16384 2024-03-14T09:03:06.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.225+0000 7f3865350b80 1 Processor -- start 2024-03-14T09:03:06.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.225+0000 7f3865350b80 1 -- start start 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057dfe0 0x55c4805bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48059d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4805bb6d0 con 0x55c48057e3c0 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4805bba10 con 0x55c4805b7180 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4805758b0 con 0x55c48057dfe0 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39902/0 (socket says 172.21.15.154:39902) 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 -- 172.21.15.154:0/2706832664 learned_addr learned my addr 172.21.15.154:0/2706832664 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 -- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057dfe0 msgr2=0x55c4805bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057dfe0 0x55c4805bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 -- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 msgr2=0x55c48059d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48059d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 -- 172.21.15.154:0/2706832664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c480496e50 con 0x55c48057e3c0 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f385259c700 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3848006e70 tx=0x7f3848014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df31bdcef897b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f38397fa700 1 -- 172.21.15.154:0/2706832664 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384802d040 con 0x55c48057e3c0 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f38397fa700 1 -- 172.21.15.154:0/2706832664 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848034050 con 0x55c48057e3c0 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.226+0000 7f38397fa700 1 -- 172.21.15.154:0/2706832664 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384801f070 con 0x55c48057e3c0 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- 172.21.15.154:0/2706832664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 msgr2=0x55c48059cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3848006e70 tx=0x7f3848014000 comp rx=0 tx=0).stop 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- 172.21.15.154:0/2706832664 shutdown_connections 2024-03-14T09:03:06.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48059d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057dfe0 0x55c4805bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 --2- 172.21.15.154:0/2706832664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057e3c0 0x55c48059cf50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- 172.21.15.154:0/2706832664 >> 172.21.15.154:0/2706832664 conn(0x55c4804f41d0 msgr2=0x55c4805797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- 172.21.15.154:0/2706832664 shutdown_connections 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- 172.21.15.154:0/2706832664 wait complete. 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 -- start start 2024-03-14T09:03:06.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.227+0000 7f3865350b80 1 Processor -- start 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c480624150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48061dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480624be0 con 0x55c48057dfe0 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4805b7b60 con 0x55c4805b7180 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4805b7e40 con 0x55c48057e3c0 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3865350b80 10 client.0 _ll_register_callbacks cb 0x55c480476420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:03:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39914/0 (socket says 172.21.15.154:39914) 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 -- 172.21.15.154:0/4172440807 learned_addr learned my addr 172.21.15.154:0/4172440807 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f385359e700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48061dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f385259c700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c480624150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 -- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 msgr2=0x55c480624150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c480624150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 -- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 msgr2=0x55c48061dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48061dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f3852d9d700 1 -- 172.21.15.154:0/4172440807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385406cd30 con 0x55c48057dfe0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.228+0000 7f385359e700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48061dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3852d9d700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f384406c040 tx=0x7f3844001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3c3a1c0525e234 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:03:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384405b080 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3844081e30 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f384c00f050 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3844086390 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 10 client.5874 fetch_fsmap learned FSMap version 30 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55c480356c80 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00f050 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3844075070 con 0x55c48057dfe0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 10 client.5874 fetch_fsmap finished waiting for FSMap version 30 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 10 client.5874 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 20 client.5874 populate_metadata read hostname 'smithi154' 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c48061e420 0x55c4805b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 4 client.5874 mds_command: new command op to 5727 tid=0[{"prefix": "status"}] 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 --> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c48031fca0 con 0x55c48061e420 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.229+0000 7f385259c700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c48061e420 0x55c4805b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3831ffb700 10 client.5874 ms_handle_connect on v2:172.21.15.191:6836/1333349284 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f385259c700 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c48061e420 0x55c4805b4900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5996fddb298e1672 server_cookie=601b5bdd47ec4fb5 in_seq=0 out_seq=0 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3831ffb700 1 -- 172.21.15.154:0/4172440807 <== mds.0 v2:172.21.15.191:6836/1333349284 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c48031fca0 con 0x55c48061e420 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3831ffb700 10 client.5874 handle_command_reply: tid=0 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 client.5874 shutdown 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 msgr2=0x55c480599a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f384406c040 tx=0x7f3844001080 comp rx=0 tx=0).stop 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 shutdown_connections 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:6836/1333349284,v1:172.21.15.191:6837/1333349284] conn(0x55c48061e420 0x55c4805b4900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c4805b7180 0x55c48061dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3831ffb700 0 client.5874 ms_handle_reset on v2:172.21.15.191:6836/1333349284 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c48057dfe0 0x55c480599a60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 --2- 172.21.15.154:0/4172440807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55c48057e3c0 0x55c480624150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 >> 172.21.15.154:0/4172440807 conn(0x55c4804f41d0 msgr2=0x55c4805d4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 shutdown_connections 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 1 -- 172.21.15.154:0/4172440807 wait complete. 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.230+0000 7f3865350b80 20 client.5874 trim_cache size 0 max 16384 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "661f6023-01e4-4d6e-b4b3-5b5701ce7cc5", 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5727, 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2024-03-14T09:03:06.237 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 6.2572254889999996, 2024-03-14T09:03:06.238 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 30, 2024-03-14T09:03:06.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 51, 2024-03-14T09:03:06.238 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 51, 2024-03-14T09:03:06.238 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 11.628870108999999 2024-03-14T09:03:06.238 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 msgr2=0x7f384006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 0x7f384006ccf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3854150bb0 tx=0x7f3848012000 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 msgr2=0x7f38541db670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f384400c450 tx=0x7f384400c810 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 shutdown_connections 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3854151400 0x7f38541dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f384006a830 0x7f384006ccf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f385406c740 0x7f38541db670 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 --2- 172.21.15.154:0/2364487701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f385414e550 0x7f38541e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 >> 172.21.15.154:0/2364487701 conn(0x7f38540c5e70 msgr2=0x7f38540c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 shutdown_connections 2024-03-14T09:03:06.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.231+0000 7f3859197700 1 -- 172.21.15.154:0/2364487701 wait complete. 2024-03-14T09:03:06.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:03:06.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.366+0000 7fc9fe7f2700 1 Processor -- start 2024-03-14T09:03:06.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9fe7f2700 1 -- start start 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9fe7f2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc9f814eaf0 0x7fc9f806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9fe7f2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80ac8e0 con 0x7fc9f806acb0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9fe7f2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f806de30 con 0x7fc9f8156500 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9fe7f2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f806b310 con 0x7fc9f814eaf0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9f77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc9f806acb0 0x7fc9f8155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55130/0 (socket says 172.21.15.154:55130) 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.367+0000 7fc9f77fe700 1 -- 172.21.15.154:0/4021720503 learned_addr learned my addr 172.21.15.154:0/4021720503 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999880337 0 0) 0x7fc9f80ac8e0 con 0x7fc9f806acb0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/4021720503 >> v2:172.21.15.191:3300/0 conn(0x7fc9f814eaf0 0x7fc9f806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e40035b0 con 0x7fc9f806acb0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1926339982 0 0) 0x7fc9e40035b0 con 0x7fc9f806acb0 2024-03-14T09:03:06.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 >> v2:172.21.15.191:3300/0 conn(0x7fc9f814eaf0 msgr2=0x7fc9f806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 --2- 172.21.15.154:0/4021720503 >> v2:172.21.15.191:3300/0 conn(0x7fc9f814eaf0 0x7fc9f806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 >> v1:172.21.15.191:6789/0 conn(0x7fc9f8156500 legacy=0x7fc9f815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8156910 con 0x7fc9f806acb0 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fc9e0001670 con 0x7fc9f806acb0 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 >> v1:172.21.15.154:6789/0 conn(0x7fc9f806acb0 legacy=0x7fc9f8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4003a00 0x7fc9e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 0x7fc9e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4005330 0x7fc9e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f806b310 con 0x7fc9e4004850 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e400c200 con 0x7fc9e4005330 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e400c3c0 con 0x7fc9e4003a00 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 0x7fc9e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4003a00 0x7fc9e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9fcd8f700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4005330 0x7fc9e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 -- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4003a00 msgr2=0x7fc9e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4003a00 0x7fc9e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 -- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4005330 msgr2=0x7fc9e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4005330 0x7fc9e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f77fe700 1 -- 172.21.15.154:0/4021720503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40035b0 con 0x7fc9e4004850 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9fcd8f700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4005330 0x7fc9e400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4003a00 0x7fc9e4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.368+0000 7fc9fcd8f700 1 -- 172.21.15.154:0/4021720503 reap_dead start 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9f77fe700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 0x7fc9e4004cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c210 tx=0x7fc9e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1f322e21b41588 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0007c10 con 0x7fc9e4004850 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e000cdb0 con 0x7fc9e4004850 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9f6ffd700 1 -- 172.21.15.154:0/4021720503 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0005ba0 con 0x7fc9e4004850 2024-03-14T09:03:06.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 msgr2=0x7fc9e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 0x7fc9e4004cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c210 tx=0x7fc9e4005880 comp rx=0 tx=0).stop 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/4021720503 shutdown_connections 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/4021720503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e4004850 0x7fc9e4004cd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/4021720503 >> 172.21.15.154:0/4021720503 conn(0x7fc9f80c3880 msgr2=0x7fc9f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/4021720503 shutdown_connections 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.369+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/4021720503 wait complete. 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.371+0000 7fc9fe7f2700 1 Processor -- start 2024-03-14T09:03:06.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.371+0000 7fc9fe7f2700 1 -- start start 2024-03-14T09:03:06.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4004850 0x7fc9f81ee570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9f80c2f20 0x7fc9f8146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c5e20 con 0x7fc9f80c4b80 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81edb20 con 0x7fc9f80c2f20 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fe7f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c6060 con 0x7fc9e4004850 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39930/0 (socket says 172.21.15.154:39930) 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 -- 172.21.15.154:0/292116946 learned_addr learned my addr 172.21.15.154:0/292116946 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4004850 0x7fc9f81ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9f77fe700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9f80c2f20 0x7fc9f8146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 -- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4004850 msgr2=0x7fc9f81ee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4004850 0x7fc9f81ee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 -- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9f80c2f20 msgr2=0x7fc9f8146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9f80c2f20 0x7fc9f8146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 -- 172.21.15.154:0/292116946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80c6340 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.372+0000 7fc9fcd8f700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc9f806d840 tx=0x7fc9f000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.373+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000eee0 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.373+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f0003c10 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.373+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f81edd70 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.373+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00050d0 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.373+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8146bc0 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.374+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f81ee290 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.374+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc9f0005230 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.374+0000 7fc9f4ff9700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 0x7fc9d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.374+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9f0088bc0 con 0x7fc9f80c4b80 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.375+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 0x7fc9d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:06.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.375+0000 7fc9f7fff700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 0x7fc9d806ccf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8157b30 tx=0x7fc9e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:06.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.379+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc9f81ee290 con 0x7fc9f80c4b80 2024-03-14T09:03:06.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.617+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9f81ee290 con 0x7fc9f80c4b80 2024-03-14T09:03:06.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.617+0000 7fc9f4ff9700 1 -- 172.21.15.154:0/292116946 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc9f81ee290 con 0x7fc9f80c4b80 2024-03-14T09:03:06.617 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 msgr2=0x7fc9d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 0x7fc9d806ccf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8157b30 tx=0x7fc9e8008040 comp rx=0 tx=0).stop 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 msgr2=0x7fc9f813e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc9f806d840 tx=0x7fc9f000ed50 comp rx=0 tx=0).stop 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 shutdown_connections 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc9d806a830 0x7fc9d806ccf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9f80c4b80 0x7fc9f813e120 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4004850 0x7fc9f81ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 --2- 172.21.15.154:0/292116946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9f80c2f20 0x7fc9f8146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:06.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 >> 172.21.15.154:0/292116946 conn(0x7fc9f80c3880 msgr2=0x7fc9f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:06.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 shutdown_connections 2024-03-14T09:03:06.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:06.620+0000 7fc9fe7f2700 1 -- 172.21.15.154:0/292116946 wait complete. 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:03:06.632 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:03:06.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.633 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:03:06.650 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:06.651 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:06.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.652 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:03:06.716 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:03:06.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.718 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:03:06.776 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:03:06.776 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:03:06.776 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:03:06.776 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:03:06.776 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:03:06.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:03:06.777 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:03:06.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.883 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:03:06.899 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:03:06.900 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:03:06.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:06.900 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:03:06.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:06.967 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:07.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:07.025 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:07.081 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T09:03:07.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:07.168 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:07.168 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:07.188 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:07.188 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:07.206 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:07.206 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:07.206 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:03:07.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:03:07.209 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:03:07.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.239 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:03:07.327 INFO:teuthology.orchestra.run.smithi154.stdout:client5168 2024-03-14T09:03:07.328 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:07.328 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:07.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:03:07.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.506+0000 7f208d7af700 1 Processor -- start 2024-03-14T09:03:07.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f208d7af700 1 -- start start 2024-03-14T09:03:07.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f208d7af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f20880c4ed0 0x7f2088160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f208d7af700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880c5340 con 0x7f208806ba10 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f208d7af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20880c55e0 con 0x7f20880c4af0 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f208d7af700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20880c5ac0 con 0x7f20880c4ed0 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f20867fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f208806ba10 0x7f208815d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55142/0 (socket says 172.21.15.154:55142) 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.507+0000 7f20867fc700 1 -- 172.21.15.154:0/3089298199 learned_addr learned my addr 172.21.15.154:0/3089298199 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20877fe700 1 --2- 172.21.15.154:0/3089298199 >> v2:172.21.15.191:3300/0 conn(0x7f20880c4ed0 0x7f2088160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995588077 0 0) 0x7f20880c5340 con 0x7f208806ba10 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20700035b0 con 0x7f208806ba10 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652224266 0 0) 0x7f20700035b0 con 0x7f208806ba10 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 >> v2:172.21.15.191:3300/0 conn(0x7f20880c4ed0 msgr2=0x7f2088160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 --2- 172.21.15.154:0/3089298199 >> v2:172.21.15.191:3300/0 conn(0x7f20880c4ed0 0x7f2088160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 >> v1:172.21.15.191:6789/0 conn(0x7f20880c4af0 legacy=0x7f208806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880c8290 con 0x7f208806ba10 2024-03-14T09:03:07.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2078001670 con 0x7f208806ba10 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 >> v1:172.21.15.154:6789/0 conn(0x7f208806ba10 legacy=0x7f208815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070003a00 0x7f2070003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 0x7f2070004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2070005210 0x7f207000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880c5ac0 con 0x7f2070005210 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207000c0b0 con 0x7f2070004850 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207000c270 con 0x7f2070003a00 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20877fe700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2070005210 0x7f207000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20867fc700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 0x7f2070004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f2086ffd700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070003a00 0x7f2070003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20867fc700 1 -- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070003a00 msgr2=0x7f2070003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20867fc700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070003a00 0x7f2070003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20867fc700 1 -- 172.21.15.154:0/3089298199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2070005210 msgr2=0x7f207000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:07.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.508+0000 7f20867fc700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2070005210 0x7f207000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f20867fc700 1 -- 172.21.15.154:0/3089298199 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700035b0 con 0x7f2070004850 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f20877fe700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2070005210 0x7f207000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f20877fe700 1 -- 172.21.15.154:0/3089298199 reap_dead start 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f20867fc700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 0x7f2070004cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f207800c2a0 tx=0x7f2070005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321074e0e491a2b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078007c10 con 0x7f2070004850 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800cde0 con 0x7f2070004850 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f2085ffb700 1 -- 172.21.15.154:0/3089298199 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078005ba0 con 0x7f2070004850 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 -- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 msgr2=0x7f2070004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 0x7f2070004cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f207800c2a0 tx=0x7f2070005880 comp rx=0 tx=0).stop 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 -- 172.21.15.154:0/3089298199 shutdown_connections 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 --2- 172.21.15.154:0/3089298199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2070004850 0x7f2070004cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 -- 172.21.15.154:0/3089298199 >> 172.21.15.154:0/3089298199 conn(0x7f20880abc60 msgr2=0x7f20880ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:07.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 -- 172.21.15.154:0/3089298199 shutdown_connections 2024-03-14T09:03:07.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.509+0000 7f208d7af700 1 -- 172.21.15.154:0/3089298199 wait complete. 2024-03-14T09:03:07.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.511+0000 7f208d7af700 1 Processor -- start 2024-03-14T09:03:07.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.511+0000 7f208d7af700 1 -- start start 2024-03-14T09:03:07.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070004850 0x7f20880c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088157110 0x7f20881eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20881eef50 con 0x7f20881eaf80 2024-03-14T09:03:07.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20880c2010 con 0x7f2088157110 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f208d7af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20881ee190 con 0x7f2070004850 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39946/0 (socket says 172.21.15.154:39946) 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070004850 0x7f20880c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 -- 172.21.15.154:0/2124955670 learned_addr learned my addr 172.21.15.154:0/2124955670 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20867fc700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088157110 0x7f20881eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 -- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070004850 msgr2=0x7f20880c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070004850 0x7f20880c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 -- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088157110 msgr2=0x7f20881eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088157110 0x7f20881eaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.512+0000 7f20877fe700 1 -- 172.21.15.154:0/2124955670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20881ee470 con 0x7f20881eaf80 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f20877fe700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f20880c52b0 tx=0x7f208000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000e040 con 0x7f20881eaf80 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208000f040 con 0x7f20881eaf80 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20881ee750 con 0x7f20881eaf80 2024-03-14T09:03:07.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080009e70 con 0x7f20881eaf80 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.513+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2088157c00 con 0x7f20881eaf80 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.514+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20880c5ea0 con 0x7f20881eaf80 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.514+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2080015430 con 0x7f20881eaf80 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.515+0000 7f206f7fe700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 0x7f207406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.515+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2080088ed0 con 0x7f20881eaf80 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.515+0000 7f2086ffd700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 0x7f207406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:07.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.515+0000 7f2086ffd700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 0x7f207406ccf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20881ebbb0 tx=0x7f207c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:07.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.519+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2080054380 con 0x7f20881eaf80 2024-03-14T09:03:07.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.764+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2088067bd0 con 0x7f20881eaf80 2024-03-14T09:03:07.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.765+0000 7f206f7fe700 1 -- 172.21.15.154:0/2124955670 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2080009c10 con 0x7f20881eaf80 2024-03-14T09:03:07.765 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 msgr2=0x7f207406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 0x7f207406ccf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20881ebbb0 tx=0x7f207c009040 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 msgr2=0x7f20880bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f20880c52b0 tx=0x7f208000f490 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 shutdown_connections 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f207406a830 0x7f207406ccf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20881eaf80 0x7f20880bdca0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2070004850 0x7f20880c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 --2- 172.21.15.154:0/2124955670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088157110 0x7f20881eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:07.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 >> 172.21.15.154:0/2124955670 conn(0x7f20880abc60 msgr2=0x7f20881e0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:07.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 shutdown_connections 2024-03-14T09:03:07.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:07.767+0000 7f208d7af700 1 -- 172.21.15.154:0/2124955670 wait complete. 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:03:07.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:03:07.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:03:07.780 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:07.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.780 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:03:07.801 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:03:07.801 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:07.802 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:03:07.803 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:07.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.804 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:03:07.873 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:03:07.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.875 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:03:07.936 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:03:07.936 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:03:07.936 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:03:07.936 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:03:07.936 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:03:07.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:03:07.937 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:03:08.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:08.049 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:03:08.067 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:03:08.067 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:08.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:08.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:03:08.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:08.135 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:08.192 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:08.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:08.192 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:08.248 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T09:03:08.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:08.249 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:08.334 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:08.334 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:08.354 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:08.355 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:08.372 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:08.372 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:08.372 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:03:08.373 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:03:08.375 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:03:08.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:08.375 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:08.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:08.402 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:03:08.474 INFO:teuthology.orchestra.run.smithi191.stdout:client5174 2024-03-14T09:03:08.475 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:08.475 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:08.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-03-14T09:03:08.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.660+0000 7f5696c29700 1 Processor -- start 2024-03-14T09:03:08.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f5696c29700 1 -- start start 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f5696c29700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5690155df0 0x7f5690156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f5696c29700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56900ac8e0 con 0x7f569014eaf0 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f5696c29700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569006de30 con 0x7f569015d270 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f5696c29700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569006b310 con 0x7f5690155df0 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f568ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5690155df0 0x7f5690156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f56949c5700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f569014eaf0 0x7f569006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55156/0 (socket says 172.21.15.154:55156) 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f56949c5700 1 -- 172.21.15.154:0/256422403 learned_addr learned my addr 172.21.15.154:0/256422403 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:08.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.661+0000 7f568ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5690155df0 0x7f5690156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39962/0 (socket says 172.21.15.154:39962) 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568ffff700 1 -- 172.21.15.154:0/256422403 >> v1:172.21.15.191:6789/0 conn(0x7f569015d270 legacy=0x7f569015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568ffff700 1 -- 172.21.15.154:0/256422403 >> v1:172.21.15.154:6789/0 conn(0x7f569014eaf0 legacy=0x7f569006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568ffff700 1 -- 172.21.15.154:0/256422403 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56900acb30 con 0x7f5690155df0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568f7fe700 1 -- 172.21.15.154:0/256422403 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038174728 0 0) 0x7f56900ac8e0 con 0x7f569014eaf0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568ffff700 1 --2- 172.21.15.154:0/256422403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5690155df0 0x7f5690156250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f567c009840 tx=0x7f567c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863644251090f6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568f7fe700 1 -- 172.21.15.154:0/256422403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011830 con 0x7f5690155df0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568f7fe700 1 -- 172.21.15.154:0/256422403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c011990 con 0x7f5690155df0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.662+0000 7f568f7fe700 1 -- 172.21.15.154:0/256422403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011d20 con 0x7f5690155df0 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 -- 172.21.15.154:0/256422403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5690155df0 msgr2=0x7f5690156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 --2- 172.21.15.154:0/256422403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5690155df0 0x7f5690156250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f567c009840 tx=0x7f567c009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 -- 172.21.15.154:0/256422403 shutdown_connections 2024-03-14T09:03:08.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 --2- 172.21.15.154:0/256422403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5690155df0 0x7f5690156250 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 -- 172.21.15.154:0/256422403 >> 172.21.15.154:0/256422403 conn(0x7f56900c3880 msgr2=0x7f56900c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:08.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 -- 172.21.15.154:0/256422403 shutdown_connections 2024-03-14T09:03:08.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.663+0000 7f5696c29700 1 -- 172.21.15.154:0/256422403 wait complete. 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.664+0000 7f5696c29700 1 Processor -- start 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.664+0000 7f5696c29700 1 -- start start 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5690155df0 0x7f5690141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f569015d270 0x7f56901421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5690149450 con 0x7f569014eaf0 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569014d1f0 con 0x7f5690155df0 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569013f380 con 0x7f569015d270 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39968/0 (socket says 172.21.15.154:39968) 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 -- 172.21.15.154:0/3861516219 learned_addr learned my addr 172.21.15.154:0/3861516219 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f568ffff700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5690155df0 0x7f5690141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56951c6700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f569015d270 0x7f56901421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:08.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 -- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f569015d270 msgr2=0x7f56901421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f569015d270 0x7f56901421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 -- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5690155df0 msgr2=0x7f5690141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5690155df0 0x7f5690141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 -- 172.21.15.154:0/3861516219 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c0094a0 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f56949c5700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f568400c550 tx=0x7f568400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684004c80 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684004de0 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5690149170 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.665+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684007310 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.666+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569013f010 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.667+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569013eb90 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.667+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5684007470 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.667+0000 7f568d7fa700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 0x7f567406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.667+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5684088870 con 0x7f569014eaf0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.668+0000 7f568ffff700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 0x7f567406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:08.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.668+0000 7f568ffff700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 0x7f567406ccf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f567c0090b0 tx=0x7f567c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:08.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.672+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f569013eb90 con 0x7f569014eaf0 2024-03-14T09:03:08.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.943+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7f569013eb90 con 0x7f569014eaf0 2024-03-14T09:03:08.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.943+0000 7f568d7fa700 1 -- 172.21.15.154:0/3861516219 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v19) v1 ==== 84+0+0 (secure 0 0 0) 0x7f569013eb90 con 0x7f569014eaf0 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 msgr2=0x7f567406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 0x7f567406ccf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f567c0090b0 tx=0x7f567c008030 comp rx=0 tx=0).stop 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 msgr2=0x7f56901539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f568400c550 tx=0x7f568400c860 comp rx=0 tx=0).stop 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 shutdown_connections 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f569015d270 0x7f56901421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f567406a830 0x7f567406ccf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f569014eaf0 0x7f56901539c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 --2- 172.21.15.154:0/3861516219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5690155df0 0x7f5690141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:08.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 >> 172.21.15.154:0/3861516219 conn(0x7f56900c3880 msgr2=0x7f569006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:08.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 shutdown_connections 2024-03-14T09:03:08.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:08.946+0000 7f5696c29700 1 -- 172.21.15.154:0/3861516219 wait complete. 2024-03-14T09:03:08.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:09.079 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.079+0000 7f8b5c6b6700 1 Processor -- start 2024-03-14T09:03:09.079 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.079+0000 7f8b5c6b6700 1 -- start start 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 0x7f8b540c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b540c4f30 0x7f8b541547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8b540c8350 0x7f8b54154d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b54158ac0 con 0x7f8b540c4f30 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b54157e80 con 0x7f8b540c8350 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b5c6b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b54158120 con 0x7f8b540c7f70 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b59c51700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b540c4f30 0x7f8b541547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b59c51700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b540c4f30 0x7f8b541547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39984/0 (socket says 172.21.15.154:39984) 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.080+0000 7f8b59c51700 1 -- 172.21.15.154:0/1340769008 learned_addr learned my addr 172.21.15.154:0/1340769008 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b5a452700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 0x7f8b540c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b5ac53700 1 --2- 172.21.15.154:0/1340769008 >> v2:172.21.15.191:3300/0 conn(0x7f8b540c8350 0x7f8b54154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 -- 172.21.15.154:0/1340769008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 msgr2=0x7f8b540c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 0x7f8b540c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 -- 172.21.15.154:0/1340769008 >> v2:172.21.15.191:3300/0 conn(0x7f8b540c8350 msgr2=0x7f8b54154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1340769008 >> v2:172.21.15.191:3300/0 conn(0x7f8b540c8350 0x7f8b54154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 -- 172.21.15.154:0/1340769008 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54158400 con 0x7f8b540c4f30 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b5a452700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 0x7f8b540c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c4f30 0x7f8b541547f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c0fa0 tx=0x7f8b50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20933a2fc155c4f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59450700 1 -- 172.21.15.154:0/1340769008 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50011830 con 0x7f8b540c4f30 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59450700 1 -- 172.21.15.154:0/1340769008 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b50011990 con 0x7f8b540c4f30 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.081+0000 7f8b59450700 1 -- 172.21.15.154:0/1340769008 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50011d20 con 0x7f8b540c4f30 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1340769008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c4f30 msgr2=0x7f8b541547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c4f30 0x7f8b541547f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c0fa0 tx=0x7f8b50009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1340769008 shutdown_connections 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1340769008 >> v2:172.21.15.191:3300/0 conn(0x7f8b540c8350 0x7f8b54154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c7f70 0x7f8b540c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1340769008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c4f30 0x7f8b541547f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1340769008 >> 172.21.15.154:0/1340769008 conn(0x7f8b54072080 msgr2=0x7f8b54072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1340769008 shutdown_connections 2024-03-14T09:03:09.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.082+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1340769008 wait complete. 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.083+0000 7f8b5c6b6700 1 Processor -- start 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.083+0000 7f8b5c6b6700 1 -- start start 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.083+0000 7f8b5c6b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c4f30 0x7f8b541535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.083+0000 7f8b5c6b6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 0x7f8b540c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540aeea0 con 0x7f8b540c7f70 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5414e000 con 0x7f8b540c8350 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b541ea010 con 0x7f8b540c4f30 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39988/0 (socket says 172.21.15.154:39988) 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 -- 172.21.15.154:0/1504270576 learned_addr learned my addr 172.21.15.154:0/1504270576 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5a452700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c4f30 0x7f8b541535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5ac53700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 0x7f8b540c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 -- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c4f30 msgr2=0x7f8b541535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c4f30 0x7f8b541535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 -- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 msgr2=0x7f8b540c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 0x7f8b540c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 -- 172.21.15.154:0/1504270576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440094a0 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5ac53700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 0x7f8b540c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b59c51700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50004300 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5001f070 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50009550 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500036a0 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.084+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5414e8c0 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.085+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b540697d0 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.086+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8b50021030 con 0x7f8b540c7f70 2024-03-14T09:03:09.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.086+0000 7f8b4affd700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 0x7f8b3806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.086+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b50089ef0 con 0x7f8b540c7f70 2024-03-14T09:03:09.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.086+0000 7f8b5a452700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 0x7f8b3806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.087+0000 7f8b5a452700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 0x7f8b3806ccf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c6320 tx=0x7f8b44009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.091+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8b500553a0 con 0x7f8b540c7f70 2024-03-14T09:03:09.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.283+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b5406cc40 con 0x7f8b540c7f70 2024-03-14T09:03:09.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.284+0000 7f8b4affd700 1 -- 172.21.15.154:0/1504270576 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16145 (secure 0 0 0) 0x7f8b50015090 con 0x7f8b540c7f70 2024-03-14T09:03:09.284 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:09.284 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":51,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:00.973845+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.286+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 msgr2=0x7f8b3806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.286+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 0x7f8b3806ccf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c6320 tx=0x7f8b44009210 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 msgr2=0x7f8b541ea740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b50004750 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 shutdown_connections 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b540c8350 0x7f8b540c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b540c4f30 0x7f8b541535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8b3806a830 0x7f8b3806ccf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 --2- 172.21.15.154:0/1504270576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b540c7f70 0x7f8b541ea740 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 >> 172.21.15.154:0/1504270576 conn(0x7f8b54072080 msgr2=0x7f8b5414aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 shutdown_connections 2024-03-14T09:03:09.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.287+0000 7f8b5c6b6700 1 -- 172.21.15.154:0/1504270576 wait complete. 2024-03-14T09:03:09.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:57.920434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:58.934033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:09.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:09.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.426+0000 7f2e87773700 1 Processor -- start 2024-03-14T09:03:09.427 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 -- start start 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e80150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2e80151400 0x7f2e8015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8006d940 con 0x7f2e8006c740 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8014e9d0 con 0x7f2e80151400 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.427+0000 7f2e87773700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800aa800 con 0x7f2e8014e550 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e8550f700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2e8006c740 0x7f2e800c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35132/0 (socket says 172.21.15.154:35132) 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e8550f700 1 -- 172.21.15.154:0/3593151343 learned_addr learned my addr 172.21.15.154:0/3593151343 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3593151343 >> v2:172.21.15.191:3300/0 conn(0x7f2e80151400 0x7f2e8015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e84d0e700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e80150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593831900 0 0) 0x7f2e8006d940 con 0x7f2e8006c740 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e680035b0 con 0x7f2e8006c740 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811224567 0 0) 0x7f2e680035b0 con 0x7f2e8006c740 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 msgr2=0x7f2e80150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e80150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 >> v2:172.21.15.191:3300/0 conn(0x7f2e80151400 msgr2=0x7f2e8015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 --2- 172.21.15.154:0/3593151343 >> v2:172.21.15.191:3300/0 conn(0x7f2e80151400 0x7f2e8015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8006cd30 con 0x7f2e8006c740 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2e700019f0 con 0x7f2e8006c740 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3593151343 >> v2:172.21.15.191:3300/0 conn(0x7f2e80151400 0x7f2e8015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 >> v1:172.21.15.154:6789/0 conn(0x7f2e8006c740 legacy=0x7f2e800c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e68003a00 0x7f2e68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e68004850 0x7f2e68004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8014e9d0 con 0x7f2e68003a00 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800aa800 con 0x7f2e68004850 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6800ba90 con 0x7f2e8014e550 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e68003a00 0x7f2e68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e84d0e700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e68004850 0x7f2e68004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.428+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 -- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 msgr2=0x7f2e68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e68005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 -- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e68004850 msgr2=0x7f2e68004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e68004850 0x7f2e68004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 -- 172.21.15.154:0/3593151343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680035b0 con 0x7f2e68003a00 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e8014e550 0x7f2e68005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e68003a00 0x7f2e68003e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000a860 tx=0x7f2e68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b38578f132353b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70012600 con 0x7f2e68003a00 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70012760 con 0x7f2e68003a00 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e7ffff700 1 -- 172.21.15.154:0/3593151343 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70012b10 con 0x7f2e68003a00 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e87773700 1 -- 172.21.15.154:0/3593151343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e68003a00 msgr2=0x7f2e68003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e87773700 1 --2- 172.21.15.154:0/3593151343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e68003a00 0x7f2e68003e20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000a860 tx=0x7f2e68005880 comp rx=0 tx=0).stop 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e85d10700 1 -- 172.21.15.154:0/3593151343 reap_dead start 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e87773700 1 -- 172.21.15.154:0/3593151343 shutdown_connections 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e87773700 1 -- 172.21.15.154:0/3593151343 >> 172.21.15.154:0/3593151343 conn(0x7f2e800c5e70 msgr2=0x7f2e800c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.429+0000 7f2e87773700 1 -- 172.21.15.154:0/3593151343 shutdown_connections 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.430+0000 7f2e87773700 1 -- 172.21.15.154:0/3593151343 wait complete. 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.431+0000 7f2e87773700 1 Processor -- start 2024-03-14T09:03:09.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.431+0000 7f2e87773700 1 -- start start 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e68003a00 0x7f2e80149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e80154490 0x7f2e801e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801e7930 con 0x7f2e801e82c0 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800bbae0 con 0x7f2e80154490 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e87773700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800bf9d0 con 0x7f2e68003a00 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56084/0 (socket says 172.21.15.154:56084) 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 -- 172.21.15.154:0/3484135716 learned_addr learned my addr 172.21.15.154:0/3484135716 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:09.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e68003a00 0x7f2e80149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e84d0e700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e80154490 0x7f2e801e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 -- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e68003a00 msgr2=0x7f2e80149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e68003a00 0x7f2e80149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 -- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e80154490 msgr2=0x7f2e801e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e80154490 0x7f2e801e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 -- 172.21.15.154:0/3484135716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800bfcb0 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.432+0000 7f2e85d10700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2e801552b0 tx=0x7f2e78011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.433+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004440 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.433+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e780036a0 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.433+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e800bff90 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.433+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78003a50 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.433+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80154dd0 con 0x7f2e801e82c0 2024-03-14T09:03:09.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.434+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e801e7aa0 con 0x7f2e801e82c0 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.434+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2e78003bb0 con 0x7f2e801e82c0 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.434+0000 7f2e7dffb700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 0x7f2e6c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.435+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e78088fd0 con 0x7f2e801e82c0 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.435+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 0x7f2e6c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.435+0000 7f2e8550f700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 0x7f2e6c06ccf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e70015040 tx=0x7f2e70009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.439+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2e801e7aa0 con 0x7f2e801e82c0 2024-03-14T09:03:09.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.630+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e80073f10 con 0x7f2e801e82c0 2024-03-14T09:03:09.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.631+0000 7f2e7dffb700 1 -- 172.21.15.154:0/3484135716 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e78054480 con 0x7f2e801e82c0 2024-03-14T09:03:09.631 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 msgr2=0x7f2e6c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 0x7f2e6c06ccf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e70015040 tx=0x7f2e70009030 comp rx=0 tx=0).stop 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 msgr2=0x7f2e80140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2e801552b0 tx=0x7f2e78011ae0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 shutdown_connections 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2e6c06a830 0x7f2e6c06ccf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e801e82c0 0x7f2e80140430 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e68003a00 0x7f2e80149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 --2- 172.21.15.154:0/3484135716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e80154490 0x7f2e801e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 >> 172.21.15.154:0/3484135716 conn(0x7f2e800c5e70 msgr2=0x7f2e800c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 shutdown_connections 2024-03-14T09:03:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.633+0000 7f2e87773700 1 -- 172.21.15.154:0/3484135716 wait complete. 2024-03-14T09:03:09.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:03:09.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.769+0000 7effe3a36700 1 Processor -- start 2024-03-14T09:03:09.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.769+0000 7effe3a36700 1 -- start start 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe3a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effdc13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe3a36700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc13ade0 con 0x7effdc0c1780 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe3a36700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc06fb30 con 0x7effdc0c3250 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe3a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc06ec30 con 0x7effdc0a8540 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe1fd3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7effdc0c1780 0x7effdc0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35136/0 (socket says 172.21.15.154:35136) 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe1fd3700 1 -- 172.21.15.154:0/416602588 learned_addr learned my addr 172.21.15.154:0/416602588 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effe17d2700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effdc13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.770+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509656738 0 0) 0x7effd4001490 con 0x7effdc0c1780 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc80035b0 con 0x7effdc0c1780 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226029226 0 0) 0x7effc80035b0 con 0x7effdc0c1780 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 msgr2=0x7effdc13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effdc13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 >> v1:172.21.15.191:6789/0 conn(0x7effdc0c3250 legacy=0x7effdc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc06ef10 con 0x7effdc0c1780 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7effd4001b80 con 0x7effdc0c1780 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 >> v1:172.21.15.154:6789/0 conn(0x7effdc0c1780 legacy=0x7effdc0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8003a30 0x7effc8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effc8004850 0x7effc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effc8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc06ec30 con 0x7effc8004850 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc800bb40 con 0x7effc8003a30 2024-03-14T09:03:09.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc800bd00 con 0x7effdc0a8540 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effc8004850 0x7effc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe17d2700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8003a30 0x7effc8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe1fd3700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effc8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 -- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 msgr2=0x7effc8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc0a8540 0x7effc8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 -- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8003a30 msgr2=0x7effc8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8003a30 0x7effc8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 -- 172.21.15.154:0/416602588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80035b0 con 0x7effc8004850 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.771+0000 7effe0fd1700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effc8004850 0x7effc8004cd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effd80018b0 tx=0x7effc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c06802e0d7947c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8001370 con 0x7effc8004850 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd800c6b0 con 0x7effc8004850 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effd3fff700 1 -- 172.21.15.154:0/416602588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800ca60 con 0x7effc8004850 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 -- 172.21.15.154:0/416602588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effc8004850 msgr2=0x7effc8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 --2- 172.21.15.154:0/416602588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effc8004850 0x7effc8004cd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effd80018b0 tx=0x7effc8005880 comp rx=0 tx=0).stop 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe1fd3700 1 -- 172.21.15.154:0/416602588 reap_dead start 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 -- 172.21.15.154:0/416602588 shutdown_connections 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 -- 172.21.15.154:0/416602588 >> 172.21.15.154:0/416602588 conn(0x7effdc0c3880 msgr2=0x7effdc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 -- 172.21.15.154:0/416602588 shutdown_connections 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.772+0000 7effe3a36700 1 -- 172.21.15.154:0/416602588 wait complete. 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.773+0000 7effe3a36700 1 Processor -- start 2024-03-14T09:03:09.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.773+0000 7effe3a36700 1 -- start start 2024-03-14T09:03:09.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8004850 0x7effdc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc14e820 0x7effdc1dee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc1519b0 con 0x7effdc14f3f0 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc14f880 con 0x7effc8004850 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe3a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc151010 con 0x7effdc14e820 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe0fd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe0fd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56114/0 (socket says 172.21.15.154:56114) 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe0fd1700 1 -- 172.21.15.154:0/2623825672 learned_addr learned my addr 172.21.15.154:0/2623825672 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe1fd3700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc14e820 0x7effdc1dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.774+0000 7effe17d2700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8004850 0x7effdc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 -- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc14e820 msgr2=0x7effdc1dee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc14e820 0x7effdc1dee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 -- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8004850 msgr2=0x7effdc14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8004850 0x7effdc14e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 -- 172.21.15.154:0/2623825672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc1512f0 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe0fd1700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effc800bb10 tx=0x7effd8001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8001370 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd8005570 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc1515d0 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd800ce90 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.775+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc1df550 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.776+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc151af0 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.776+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7effd8023070 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.777+0000 7effd1ffb700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 0x7effc006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.777+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effd80899a0 con 0x7effdc14f3f0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.777+0000 7effe17d2700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 0x7effc006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:09.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.777+0000 7effe17d2700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 0x7effc006ccf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effdc150c60 tx=0x7effcc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:09.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.781+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7effdc151af0 con 0x7effdc14f3f0 2024-03-14T09:03:09.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.973+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effdc151af0 con 0x7effdc14f3f0 2024-03-14T09:03:09.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.973+0000 7effd1ffb700 1 -- 172.21.15.154:0/2623825672 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7effd802a020 con 0x7effdc14f3f0 2024-03-14T09:03:09.973 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:09.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 msgr2=0x7effc006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 0x7effc006ccf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effdc150c60 tx=0x7effcc009210 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 msgr2=0x7effdc14dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effc800bb10 tx=0x7effd8001110 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 shutdown_connections 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effdc14e820 0x7effdc1dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effc8004850 0x7effdc14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7effc006a830 0x7effc006ccf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 --2- 172.21.15.154:0/2623825672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc14f3f0 0x7effdc14dcb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.975+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 >> 172.21.15.154:0/2623825672 conn(0x7effdc0c3880 msgr2=0x7effdc0c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.976+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 shutdown_connections 2024-03-14T09:03:09.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:09.976+0000 7effe3a36700 1 -- 172.21.15.154:0/2623825672 wait complete. 2024-03-14T09:03:09.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:03:10.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.113+0000 7f3649cfc700 1 Processor -- start 2024-03-14T09:03:10.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.113+0000 7f3649cfc700 1 -- start start 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3649cfc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f364414eaf0 0x7f364406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3649cfc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 0x7f3644155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3649cfc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36440ac8e0 con 0x7f3644156330 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3649cfc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364406b310 con 0x7f364414eaf0 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3649cfc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364406de30 con 0x7f364406acb0 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3643fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3644156330 0x7f3644157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35148/0 (socket says 172.21.15.154:35148) 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3643fff700 1 -- 172.21.15.154:0/3002137790 learned_addr learned my addr 172.21.15.154:0/3002137790 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3002137790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 0x7f3644155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f36437fe700 1 --2- 172.21.15.154:0/3002137790 >> v2:172.21.15.191:3300/0 conn(0x7f364414eaf0 0x7f364406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.114+0000 7f36427fc700 1 -- 172.21.15.154:0/3002137790 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271529410 0 0) 0x7f36440ac8e0 con 0x7f3644156330 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f36427fc700 1 -- 172.21.15.154:0/3002137790 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36300035b0 con 0x7f3644156330 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f3642ffd700 1 -- 172.21.15.154:0/3002137790 >> v2:172.21.15.191:3300/0 conn(0x7f364414eaf0 msgr2=0x7f364406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:10.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3002137790 >> v2:172.21.15.191:3300/0 conn(0x7f364414eaf0 0x7f364406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f3642ffd700 1 -- 172.21.15.154:0/3002137790 >> v1:172.21.15.154:6789/0 conn(0x7f3644156330 legacy=0x7f3644157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f3642ffd700 1 -- 172.21.15.154:0/3002137790 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440acb30 con 0x7f364406acb0 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3002137790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 0x7f3644155df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3634009840 tx=0x7f3634009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e9cbbaced5a24d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f36427fc700 1 -- 172.21.15.154:0/3002137790 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634011830 con 0x7f364406acb0 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f36427fc700 1 -- 172.21.15.154:0/3002137790 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634011990 con 0x7f364406acb0 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.115+0000 7f36427fc700 1 -- 172.21.15.154:0/3002137790 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634011d20 con 0x7f364406acb0 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 -- 172.21.15.154:0/3002137790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 msgr2=0x7f3644155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3002137790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 0x7f3644155df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3634009840 tx=0x7f3634009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 -- 172.21.15.154:0/3002137790 shutdown_connections 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3002137790 >> v2:172.21.15.191:3300/0 conn(0x7f364414eaf0 0x7f364406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3002137790 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364406acb0 0x7f3644155df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 -- 172.21.15.154:0/3002137790 >> 172.21.15.154:0/3002137790 conn(0x7f36440c3880 msgr2=0x7f36440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 -- 172.21.15.154:0/3002137790 shutdown_connections 2024-03-14T09:03:10.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.116+0000 7f3649cfc700 1 -- 172.21.15.154:0/3002137790 wait complete. 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.117+0000 7f3649cfc700 1 Processor -- start 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.117+0000 7f3649cfc700 1 -- start start 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 0x7f36440c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644156330 0x7f36440c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644154760 con 0x7f364406acb0 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644153f50 con 0x7f3644156330 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36441532f0 con 0x7f364414eaf0 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56116/0 (socket says 172.21.15.154:56116) 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 -- 172.21.15.154:0/3335478205 learned_addr learned my addr 172.21.15.154:0/3335478205 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 0x7f36440c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3643fff700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644156330 0x7f36440c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 -- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 msgr2=0x7f36440c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 0x7f36440c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 -- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644156330 msgr2=0x7f36440c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644156330 0x7f36440c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 -- 172.21.15.154:0/3335478205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340094a0 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 0x7f36440c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f36437fe700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e870 tx=0x7f362c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c009590 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00ee70 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36441541a0 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00ca50 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.118+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3644153780 con 0x7f364406acb0 2024-03-14T09:03:10.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.119+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644157a60 con 0x7f364406acb0 2024-03-14T09:03:10.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.120+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f362c00cbb0 con 0x7f364406acb0 2024-03-14T09:03:10.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.120+0000 7f363bfff700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 0x7f362406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.120+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f362c088d30 con 0x7f364406acb0 2024-03-14T09:03:10.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.120+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 0x7f362406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.121+0000 7f3642ffd700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 0x7f362406cd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36440c3850 tx=0x7f3634008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.125+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f362c054150 con 0x7f364406acb0 2024-03-14T09:03:10.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.316+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36440697d0 con 0x7f364406acb0 2024-03-14T09:03:10.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.316+0000 7f363bfff700 1 -- 172.21.15.154:0/3335478205 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f362c00ce60 con 0x7f364406acb0 2024-03-14T09:03:10.316 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 msgr2=0x7f362406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 0x7f362406cd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36440c3850 tx=0x7f3634008030 comp rx=0 tx=0).stop 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 msgr2=0x7f36440c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f362c00e870 tx=0x7f362c00ec30 comp rx=0 tx=0).stop 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 shutdown_connections 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644156330 0x7f36440c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f362406a860 0x7f362406cd20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f364406acb0 0x7f36440c19e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 --2- 172.21.15.154:0/3335478205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f364414eaf0 0x7f36440c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 >> 172.21.15.154:0/3335478205 conn(0x7f36440c3880 msgr2=0x7f364414c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 shutdown_connections 2024-03-14T09:03:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.318+0000 7f3649cfc700 1 -- 172.21.15.154:0/3335478205 wait complete. 2024-03-14T09:03:10.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:03:10.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.452+0000 7fd30fc30700 1 Processor -- start 2024-03-14T09:03:10.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 -- start start 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3080c26c0 0x7fd30814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3080c55a0 0x7fd3080c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd308155a40 con 0x7fd3081518c0 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3080c2f10 con 0x7fd3080c26c0 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.453+0000 7fd30fc30700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd308072a90 con 0x7fd3080c55a0 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3080c26c0 0x7fd30814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3080c26c0 0x7fd30814edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56124/0 (socket says 172.21.15.154:56124) 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 -- 172.21.15.154:0/73819892 learned_addr learned my addr 172.21.15.154:0/73819892 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d1cb700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3080c55a0 0x7fd3080c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 -- 172.21.15.154:0/73819892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3080c55a0 msgr2=0x7fd3080c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3080c55a0 0x7fd3080c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 -- 172.21.15.154:0/73819892 >> v1:172.21.15.191:6789/0 conn(0x7fd3081518c0 legacy=0x7fd308151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 -- 172.21.15.154:0/73819892 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3081556d0 con 0x7fd3080c26c0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30d9cc700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd30814edf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00bb90 tx=0x7fd2fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e0a5013d8fa17 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30c9ca700 1 -- 172.21.15.154:0/73819892 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc00dd30 con 0x7fd3080c26c0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30c9ca700 1 -- 172.21.15.154:0/73819892 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00de90 con 0x7fd3080c26c0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.454+0000 7fd30c9ca700 1 -- 172.21.15.154:0/73819892 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc010690 con 0x7fd3080c26c0 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 -- 172.21.15.154:0/73819892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 msgr2=0x7fd30814edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd30814edf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00bb90 tx=0x7fd2fc00bea0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 -- 172.21.15.154:0/73819892 shutdown_connections 2024-03-14T09:03:10.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd30814edf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 --2- 172.21.15.154:0/73819892 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3080c55a0 0x7fd3080c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 -- 172.21.15.154:0/73819892 >> 172.21.15.154:0/73819892 conn(0x7fd30806a770 msgr2=0x7fd30806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:10.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 -- 172.21.15.154:0/73819892 shutdown_connections 2024-03-14T09:03:10.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.455+0000 7fd30fc30700 1 -- 172.21.15.154:0/73819892 wait complete. 2024-03-14T09:03:10.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.456+0000 7fd30fc30700 1 Processor -- start 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.456+0000 7fd30fc30700 1 -- start start 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3080c55a0 0x7fd3080a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3081518c0 0x7fd3080ab830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308151410 con 0x7fd3080c26c0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081532b0 con 0x7fd3080c55a0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30fc30700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd308150e90 con 0x7fd3081518c0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56136/0 (socket says 172.21.15.154:56136) 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 -- 172.21.15.154:0/564935312 learned_addr learned my addr 172.21.15.154:0/564935312 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d1cb700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3080c55a0 0x7fd3080a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30e1cd700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3081518c0 0x7fd3080ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 -- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3081518c0 msgr2=0x7fd3080ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3081518c0 0x7fd3080ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 -- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3080c55a0 msgr2=0x7fd3080a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3080c55a0 0x7fd3080a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 -- 172.21.15.154:0/564935312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc00b7f0 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd30d9cc700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00bee0 tx=0x7fd2fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc003b70 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.457+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc003cd0 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.458+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0106b0 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.458+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308153740 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.458+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd308152c60 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.459+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd308153590 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.459+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd2fc019400 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.459+0000 7fd2fa7fc700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 0x7fd2ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.459+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2fc08a2c0 con 0x7fd3080c26c0 2024-03-14T09:03:10.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.459+0000 7fd30d1cb700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 0x7fd2ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.460+0000 7fd30d1cb700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 0x7fd2ec06ccf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3080c0ee0 tx=0x7fd304009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.464+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd2fc055820 con 0x7fd3080c26c0 2024-03-14T09:03:10.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.660+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd3080697d0 con 0x7fd3080c26c0 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.660+0000 7fd2fa7fc700 1 -- 172.21.15.154:0/564935312 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+207 (secure 0 0 0) 0x7fd2fc058e40 con 0x7fd3080c26c0 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:03:10.660 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 8, 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_metadata" 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 9, 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_data" 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:10.661 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:03:10.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 msgr2=0x7fd2ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 0x7fd2ec06ccf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd3080c0ee0 tx=0x7fd304009210 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 msgr2=0x7fd308150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00bee0 tx=0x7fd2fc004750 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 shutdown_connections 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd3081518c0 0x7fd3080ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2ec06a830 0x7fd2ec06ccf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3080c26c0 0x7fd308150950 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 --2- 172.21.15.154:0/564935312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3080c55a0 0x7fd3080a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 >> 172.21.15.154:0/564935312 conn(0x7fd30806a770 msgr2=0x7fd30806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 shutdown_connections 2024-03-14T09:03:10.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.662+0000 7fd30fc30700 1 -- 172.21.15.154:0/564935312 wait complete. 2024-03-14T09:03:10.674 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-14T09:03:10.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-14T09:03:10.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 Processor -- start 2024-03-14T09:03:10.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 -- start start 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0c180c52f0 0x7f0c1806bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c180c65a0 0x7f0c180c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180c4cc0 con 0x7f0c180c7f80 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1806c260 con 0x7f0c180c65a0 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.801+0000 7f0c1d63c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180c4fa0 con 0x7f0c180c52f0 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c180c65a0 0x7f0c180c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c180c65a0 0x7f0c180c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56152/0 (socket says 172.21.15.154:56152) 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 -- 172.21.15.154:0/2560126127 learned_addr learned my addr 172.21.15.154:0/2560126127 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:10.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c16ffd700 1 --2- 172.21.15.154:0/2560126127 >> v2:172.21.15.191:3300/0 conn(0x7f0c180c52f0 0x7f0c1806bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 -- 172.21.15.154:0/2560126127 >> v2:172.21.15.191:3300/0 conn(0x7f0c180c52f0 msgr2=0x7f0c1806bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 --2- 172.21.15.154:0/2560126127 >> v2:172.21.15.191:3300/0 conn(0x7f0c180c52f0 0x7f0c1806bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 -- 172.21.15.154:0/2560126127 >> v1:172.21.15.191:6789/0 conn(0x7f0c180c7f80 legacy=0x7f0c180c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 -- 172.21.15.154:0/2560126127 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18155580 con 0x7f0c180c65a0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c177fe700 1 --2- 172.21.15.154:0/2560126127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0c1808b650 tx=0x7f0c1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b684cf0ab8594297 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c15ffb700 1 -- 172.21.15.154:0/2560126127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000e070 con 0x7f0c180c65a0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c15ffb700 1 -- 172.21.15.154:0/2560126127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000b9c0 con 0x7f0c180c65a0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.802+0000 7f0c15ffb700 1 -- 172.21.15.154:0/2560126127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000bd70 con 0x7f0c180c65a0 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 -- 172.21.15.154:0/2560126127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 msgr2=0x7f0c180c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/2560126127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0c1808b650 tx=0x7f0c1000b330 comp rx=0 tx=0).stop 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 -- 172.21.15.154:0/2560126127 shutdown_connections 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/2560126127 >> v2:172.21.15.191:3300/0 conn(0x7f0c180c52f0 0x7f0c1806bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/2560126127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8400 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 -- 172.21.15.154:0/2560126127 >> 172.21.15.154:0/2560126127 conn(0x7f0c180ca260 msgr2=0x7f0c180ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:10.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 -- 172.21.15.154:0/2560126127 shutdown_connections 2024-03-14T09:03:10.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.803+0000 7f0c1d63c700 1 -- 172.21.15.154:0/2560126127 wait complete. 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.804+0000 7f0c1d63c700 1 Processor -- start 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.804+0000 7f0c1d63c700 1 -- start start 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 0x7f0c180c0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c180c7f80 0x7f0c180bdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180c0220 con 0x7f0c180c65a0 2024-03-14T09:03:10.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18155e70 con 0x7f0c180c52f0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c1d63c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181eb400 con 0x7f0c180c7f80 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56162/0 (socket says 172.21.15.154:56162) 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 -- 172.21.15.154:0/4261667755 learned_addr learned my addr 172.21.15.154:0/4261667755 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c16ffd700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 0x7f0c180c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c177fe700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c180c7f80 0x7f0c180bdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 -- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c180c7f80 msgr2=0x7f0c180bdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c180c7f80 0x7f0c180bdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 -- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 msgr2=0x7f0c180c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 0x7f0c180c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 -- 172.21.15.154:0/4261667755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10009b30 con 0x7f0c180c65a0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c16ffd700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 0x7f0c180c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0c167fc700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c4e0 tx=0x7f0c0c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.805+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0052d0 con 0x7f0c180c65a0 2024-03-14T09:03:10.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.806+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c005430 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.806+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c18156130 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.806+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0038c0 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.806+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c181566f0 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.806+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c180697d0 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.807+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0c0c003a20 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.807+0000 7f0bff7fe700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 0x7f0c0406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.807+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c0c088230 con 0x7f0c180c65a0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.808+0000 7f0c16ffd700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 0x7f0c0406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:10.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.808+0000 7f0c16ffd700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 0x7f0c0406ccf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009820 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:10.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:10.812+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0c0c053650 con 0x7f0c180c65a0 2024-03-14T09:03:11.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.007+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c180c9d90 con 0x7f0c180c65a0 2024-03-14T09:03:11.347 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.347+0000 7f0bff7fe700 1 -- 172.21.15.154:0/4261667755 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v52) v1 ==== 140+0+0 (secure 0 0 0) 0x7f0c0c056c70 con 0x7f0c180c65a0 2024-03-14T09:03:11.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 msgr2=0x7f0c0406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 0x7f0c0406ccf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009820 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-03-14T09:03:11.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 msgr2=0x7f0c180c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c4e0 tx=0x7f0c0c00c8a0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 shutdown_connections 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c180c7f80 0x7f0c180bdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c180c52f0 0x7f0c180c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0c0406a830 0x7f0c0406ccf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 --2- 172.21.15.154:0/4261667755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c180c65a0 0x7f0c180c8b80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 >> 172.21.15.154:0/4261667755 conn(0x7f0c180ca260 msgr2=0x7f0c180c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 shutdown_connections 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.352+0000 7f0c1d63c700 1 -- 172.21.15.154:0/4261667755 wait complete. 2024-03-14T09:03:11.353 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' created 2024-03-14T09:03:11.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:03:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.496+0000 7f18e061e700 1 Processor -- start 2024-03-14T09:03:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.496+0000 7f18e061e700 1 -- start start 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.497+0000 7f18e061e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18d814eaf0 0x7f18d806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.497+0000 7f18e061e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 0x7f18d815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.497+0000 7f18e061e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d80ac8e0 con 0x7f18d806acb0 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.497+0000 7f18e061e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d806b310 con 0x7f18d814eaf0 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.497+0000 7f18e061e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d806de30 con 0x7f18d8156500 2024-03-14T09:03:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f18d814eaf0 0x7f18d806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18ddbb9700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f18d806acb0 0x7f18d8155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35156/0 (socket says 172.21.15.154:35156) 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18ddbb9700 1 -- 172.21.15.154:0/803684448 learned_addr learned my addr 172.21.15.154:0/803684448 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 -- 172.21.15.154:0/803684448 >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 msgr2=0x7f18d815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18debbb700 1 --2- 172.21.15.154:0/803684448 >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 0x7f18d815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18dd3b8700 1 -- 172.21.15.154:0/803684448 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624017132 0 0) 0x7f18d80ac8e0 con 0x7f18d806acb0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 --2- 172.21.15.154:0/803684448 >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 0x7f18d815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 -- 172.21.15.154:0/803684448 >> v1:172.21.15.154:6789/0 conn(0x7f18d806acb0 legacy=0x7f18d8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 -- 172.21.15.154:0/803684448 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d8156910 con 0x7f18d814eaf0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18debbb700 1 --2- 172.21.15.154:0/803684448 >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 0x7f18d815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18de3ba700 1 --2- 172.21.15.154:0/803684448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d814eaf0 0x7f18d806a770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18d80c4920 tx=0x7f18d4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f502c9733ffe312 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18dd3b8700 1 -- 172.21.15.154:0/803684448 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18d814eaf0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18dd3b8700 1 -- 172.21.15.154:0/803684448 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4011990 con 0x7f18d814eaf0 2024-03-14T09:03:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.498+0000 7f18dd3b8700 1 -- 172.21.15.154:0/803684448 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011d20 con 0x7f18d814eaf0 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 -- 172.21.15.154:0/803684448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d814eaf0 msgr2=0x7f18d806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 --2- 172.21.15.154:0/803684448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d814eaf0 0x7f18d806a770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18d80c4920 tx=0x7f18d4009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 -- 172.21.15.154:0/803684448 shutdown_connections 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 --2- 172.21.15.154:0/803684448 >> v2:172.21.15.191:3300/0 conn(0x7f18d8156500 0x7f18d815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 --2- 172.21.15.154:0/803684448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d814eaf0 0x7f18d806a770 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 -- 172.21.15.154:0/803684448 >> 172.21.15.154:0/803684448 conn(0x7f18d80c3880 msgr2=0x7f18d80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 -- 172.21.15.154:0/803684448 shutdown_connections 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.499+0000 7f18e061e700 1 -- 172.21.15.154:0/803684448 wait complete. 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.500+0000 7f18e061e700 1 Processor -- start 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.500+0000 7f18e061e700 1 -- start start 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 0x7f18d8143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 0x7f18d8141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d814b7b0 con 0x7f18d806acb0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81497a0 con 0x7f18d814eaf0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8149290 con 0x7f18d8156500 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56188/0 (socket says 172.21.15.154:56188) 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 -- 172.21.15.154:0/1113625122 learned_addr learned my addr 172.21.15.154:0/1113625122 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18debbb700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 0x7f18d8141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18ddbb9700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 0x7f18d8143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 -- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 msgr2=0x7f18d8141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 0x7f18d8141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 -- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 msgr2=0x7f18d8143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 0x7f18d8143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 -- 172.21.15.154:0/1113625122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d40094a0 con 0x7f18d806acb0 2024-03-14T09:03:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18debbb700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 0x7f18d8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18ddbb9700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 0x7f18d8143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18de3ba700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d40036a0 con 0x7f18d806acb0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d401f070 con 0x7f18d806acb0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d8149a40 con 0x7f18d806acb0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003a60 con 0x7f18d806acb0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.501+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d80c2290 con 0x7f18d806acb0 2024-03-14T09:03:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.502+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d80c2450 con 0x7f18d806acb0 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.503+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f18d4021030 con 0x7f18d806acb0 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.503+0000 7f18ceffd700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 0x7f18bc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.503+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6080+0+0 (secure 0 0 0) 0x7f18d408a200 con 0x7f18d806acb0 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.504+0000 7f18ddbb9700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 0x7f18bc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.504+0000 7f18ddbb9700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 0x7f18bc06ccf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18d8154ad0 tx=0x7f18c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.508+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f18d4055540 con 0x7f18d806acb0 2024-03-14T09:03:11.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:11.703+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f18d80697d0 con 0x7f18d806acb0 2024-03-14T09:03:12.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.352+0000 7f18ceffd700 1 -- 172.21.15.154:0/1113625122 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v53) v1 ==== 172+0+0 (secure 0 0 0) 0x7f18d4058b60 con 0x7f18d806acb0 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 msgr2=0x7f18bc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 0x7f18bc06ccf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18d8154ad0 tx=0x7f18c8009040 comp rx=0 tx=0).stop 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 msgr2=0x7f18d80c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d4004750 comp rx=0 tx=0).stop 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 shutdown_connections 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18d8156500 0x7f18d8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f18bc06a830 0x7f18bc06ccf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18d806acb0 0x7f18d80c3070 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 --2- 172.21.15.154:0/1113625122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18d814eaf0 0x7f18d8143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.364+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 >> 172.21.15.154:0/1113625122 conn(0x7f18d80c3880 msgr2=0x7f18d80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:12.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.365+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 shutdown_connections 2024-03-14T09:03:12.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.365+0000 7f18e061e700 1 -- 172.21.15.154:0/1113625122 wait complete. 2024-03-14T09:03:12.366 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' created 2024-03-14T09:03:12.382 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-14T09:03:12.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.509+0000 7f86c3acb700 1 Processor -- start 2024-03-14T09:03:12.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.509+0000 7f86c3acb700 1 -- start start 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c3acb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86bc14eaf0 0x7f86bc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c3acb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0ac8e0 con 0x7f86bc156500 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c3acb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc06de30 con 0x7f86bc06acb0 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c3acb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc06b310 con 0x7f86bc14eaf0 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c1867700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86bc14eaf0 0x7f86bc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c2068700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f86bc156500 0x7f86bc15f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35166/0 (socket says 172.21.15.154:35166) 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c2068700 1 -- 172.21.15.154:0/16924498 learned_addr learned my addr 172.21.15.154:0/16924498 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c1867700 1 -- 172.21.15.154:0/16924498 >> v1:172.21.15.191:6789/0 conn(0x7f86bc06acb0 legacy=0x7f86bc155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:12.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c1867700 1 -- 172.21.15.154:0/16924498 >> v1:172.21.15.154:6789/0 conn(0x7f86bc156500 legacy=0x7f86bc15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c1867700 1 -- 172.21.15.154:0/16924498 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc156910 con 0x7f86bc14eaf0 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c1867700 1 --2- 172.21.15.154:0/16924498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc14eaf0 0x7f86bc06a770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009840 tx=0x7f86b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3741b0bd166e899c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c0865700 1 -- 172.21.15.154:0/16924498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8004230 con 0x7f86bc14eaf0 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c0865700 1 -- 172.21.15.154:0/16924498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b8004390 con 0x7f86bc14eaf0 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.510+0000 7f86c0865700 1 -- 172.21.15.154:0/16924498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80036a0 con 0x7f86bc14eaf0 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 -- 172.21.15.154:0/16924498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc14eaf0 msgr2=0x7f86bc06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 --2- 172.21.15.154:0/16924498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc14eaf0 0x7f86bc06a770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009840 tx=0x7f86b8009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 -- 172.21.15.154:0/16924498 shutdown_connections 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 --2- 172.21.15.154:0/16924498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc14eaf0 0x7f86bc06a770 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 -- 172.21.15.154:0/16924498 >> 172.21.15.154:0/16924498 conn(0x7f86bc0c3880 msgr2=0x7f86bc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 -- 172.21.15.154:0/16924498 shutdown_connections 2024-03-14T09:03:12.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.511+0000 7f86c3acb700 1 -- 172.21.15.154:0/16924498 wait complete. 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.512+0000 7f86c3acb700 1 Processor -- start 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.512+0000 7f86c3acb700 1 -- start start 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86bc14eaf0 0x7f86bc1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc156500 0x7f86bc14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc14ade0 con 0x7f86bc06acb0 2024-03-14T09:03:12.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc149810 con 0x7f86bc156500 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c3acb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc149a60 con 0x7f86bc14eaf0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56210/0 (socket says 172.21.15.154:56210) 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 -- 172.21.15.154:0/2117961307 learned_addr learned my addr 172.21.15.154:0/2117961307 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c2068700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc156500 0x7f86bc14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1066700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86bc14eaf0 0x7f86bc1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 -- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86bc14eaf0 msgr2=0x7f86bc1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86bc14eaf0 0x7f86bc1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 -- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc156500 msgr2=0x7f86bc14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc156500 0x7f86bc14b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 -- 172.21.15.154:0/2117961307 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80094a0 con 0x7f86bc06acb0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.513+0000 7f86c1867700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f86b80090b0 tx=0x7f86b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.514+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8003ea0 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.514+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b801e070 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.514+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc149d40 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.514+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8011850 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.514+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc154250 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.515+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc154440 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.515+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f86b80119b0 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.515+0000 7f86b27fc700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 0x7f86a006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.516+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6463+0+0 (secure 0 0 0) 0x7f86b808a1f0 con 0x7f86bc06acb0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.516+0000 7f86c1066700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 0x7f86a006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.516+0000 7f86c1066700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 0x7f86a006ccf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f86ac007fd0 tx=0x7f86ac007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.520+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f86b80553b0 con 0x7f86bc06acb0 2024-03-14T09:03:12.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.758+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f86bc0697d0 con 0x7f86bc06acb0 2024-03-14T09:03:12.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.764+0000 7f86b27fc700 1 -- 172.21.15.154:0/2117961307 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 10 and data pool 11 v31) v1 ==== 177+0+0 (secure 0 0 0) 0x7f86b80589d0 con 0x7f86bc06acb0 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 msgr2=0x7f86a006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 0x7f86a006ccf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f86ac007fd0 tx=0x7f86ac007ec0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 msgr2=0x7f86bc154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f86b80090b0 tx=0x7f86b8004750 comp rx=0 tx=0).stop 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 shutdown_connections 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc156500 0x7f86bc14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86a006a830 0x7f86a006ccf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86bc06acb0 0x7f86bc154770 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 --2- 172.21.15.154:0/2117961307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86bc14eaf0 0x7f86bc1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.772+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 >> 172.21.15.154:0/2117961307 conn(0x7f86bc0c3880 msgr2=0x7f86bc14c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:12.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.774+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 shutdown_connections 2024-03-14T09:03:12.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.775+0000 7f86c3acb700 1 -- 172.21.15.154:0/2117961307 wait complete. 2024-03-14T09:03:12.775 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 10 and data pool 11 2024-03-14T09:03:12.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:12.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.916+0000 7fe128350700 1 Processor -- start 2024-03-14T09:03:12.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.916+0000 7fe128350700 1 -- start start 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe128350700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe12014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe128350700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe1200c55a0 0x7fe1200c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe128350700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe120155a40 con 0x7fe1201518c0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe128350700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1200c2f10 con 0x7fe1200c55a0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe128350700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe120072a90 con 0x7fe1200c26c0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe1268ed700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe1201518c0 0x7fe120151d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35180/0 (socket says 172.21.15.154:35180) 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe1268ed700 1 -- 172.21.15.154:0/136575446 learned_addr learned my addr 172.21.15.154:0/136575446 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe1260ec700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe12014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe1258eb700 1 --2- 172.21.15.154:0/136575446 >> v2:172.21.15.191:3300/0 conn(0x7fe1200c55a0 0x7fe1200c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.917+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142535339 0 0) 0x7fe120155a40 con 0x7fe1201518c0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c0035b0 con 0x7fe1201518c0 2024-03-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 256183916 0 0) 0x7fe10c0035b0 con 0x7fe1201518c0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1258eb700 1 -- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 msgr2=0x7fe12014edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1258eb700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe12014edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1258eb700 1 -- 172.21.15.154:0/136575446 >> v1:172.21.15.154:6789/0 conn(0x7fe1201518c0 legacy=0x7fe120151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1258eb700 1 -- 172.21.15.154:0/136575446 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1201556d0 con 0x7fe1200c55a0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1260ec700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe12014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1258eb700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200c5a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe110009840 tx=0x7fe110009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1920e3c71848adb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1180019f0 con 0x7fe1200c55a0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110004030 con 0x7fe1200c55a0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.918+0000 7fe1250ea700 1 -- 172.21.15.154:0/136575446 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100043a0 con 0x7fe1200c55a0 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 -- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 msgr2=0x7fe1200c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200c5a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe110009840 tx=0x7fe110009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 -- 172.21.15.154:0/136575446 shutdown_connections 2024-03-14T09:03:12.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe12014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 --2- 172.21.15.154:0/136575446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200c5a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 -- 172.21.15.154:0/136575446 >> 172.21.15.154:0/136575446 conn(0x7fe12006a770 msgr2=0x7fe12006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:12.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 -- 172.21.15.154:0/136575446 shutdown_connections 2024-03-14T09:03:12.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.919+0000 7fe128350700 1 -- 172.21.15.154:0/136575446 wait complete. 2024-03-14T09:03:12.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.920+0000 7fe128350700 1 Processor -- start 2024-03-14T09:03:12.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.920+0000 7fe128350700 1 -- start start 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe1200ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12014c710 con 0x7fe1201518c0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1201542a0 con 0x7fe1200c55a0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe128350700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe120153ce0 con 0x7fe1200c26c0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe1268ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe1268ed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56214/0 (socket says 172.21.15.154:56214) 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe1268ed700 1 -- 172.21.15.154:0/3446740414 learned_addr learned my addr 172.21.15.154:0/3446740414 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe1260ec700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe1200ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.921+0000 7fe1258eb700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 -- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 msgr2=0x7fe1200ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe1200ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 -- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 msgr2=0x7fe1200a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:12.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 -- 172.21.15.154:0/3446740414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0094a0 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe1268ed700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe11800ee80 tx=0x7fe11800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11800cd00 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11800ce80 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1100094a0 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118007310 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.922+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1200a9420 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.923+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12014edf0 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.923+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe118007470 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.924+0000 7fe116ffd700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 0x7fe10406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.924+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fe11808bfa0 con 0x7fe1201518c0 2024-03-14T09:03:12.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.924+0000 7fe1260ec700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 0x7fe10406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:12.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.925+0000 7fe1260ec700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 0x7fe10406ccf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009d70 tx=0x7fe11c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:12.929+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe1180bf4f0 con 0x7fe1201518c0 2024-03-14T09:03:13.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.123+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe12014c8b0 con 0x7fe1201518c0 2024-03-14T09:03:13.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.124+0000 7fe116ffd700 1 -- 172.21.15.154:0/3446740414 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+19873 (secure 0 0 0) 0x7fe118015090 con 0x7fe1201518c0 2024-03-14T09:03:13.124 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:13.125 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:12.760108+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.126+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 msgr2=0x7fe10406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.126+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 0x7fe10406ccf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009d70 tx=0x7fe11c009210 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.126+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 msgr2=0x7fe1201545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.126+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe11800ee80 tx=0x7fe11800c550 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 shutdown_connections 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe10406a830 0x7fe10406ccf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1201518c0 0x7fe1201545a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe1200c26c0 0x7fe1200ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 --2- 172.21.15.154:0/3446740414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1200c55a0 0x7fe1200a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 >> 172.21.15.154:0/3446740414 conn(0x7fe12006a770 msgr2=0x7fe12015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 shutdown_connections 2024-03-14T09:03:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.127+0000 7fe128350700 1 -- 172.21.15.154:0/3446740414 wait complete. 2024-03-14T09:03:13.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:13.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.260+0000 7f471098f700 1 Processor -- start 2024-03-14T09:03:13.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.260+0000 7f471098f700 1 -- start start 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f471098f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 0x7f470c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f471098f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f470c0c1780 0x7f470c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f471098f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c13ade0 con 0x7f470c0c3250 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f471098f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c06fb30 con 0x7f470c0c1780 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f471098f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c06ec30 con 0x7f470c0a8540 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f470ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f470c0c1780 0x7f470c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f4709d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f470c0c3250 0x7f470c154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35196/0 (socket says 172.21.15.154:35196) 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.261+0000 7f4709d9b700 1 -- 172.21.15.154:0/287476420 learned_addr learned my addr 172.21.15.154:0/287476420 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470a59c700 1 --2- 172.21.15.154:0/287476420 >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 0x7f470c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470959a700 1 -- 172.21.15.154:0/287476420 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313945589 0 0) 0x7f46fc001110 con 0x7f470c0c3250 2024-03-14T09:03:13.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470959a700 1 -- 172.21.15.154:0/287476420 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f40057c0 con 0x7f470c0c3250 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470ad9d700 1 -- 172.21.15.154:0/287476420 >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 msgr2=0x7f470c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470ad9d700 1 --2- 172.21.15.154:0/287476420 >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 0x7f470c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470ad9d700 1 -- 172.21.15.154:0/287476420 >> v1:172.21.15.154:6789/0 conn(0x7f470c0c3250 legacy=0x7f470c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470ad9d700 1 -- 172.21.15.154:0/287476420 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c06ef10 con 0x7f470c0c1780 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470a59c700 1 --2- 172.21.15.154:0/287476420 >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 0x7f470c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470ad9d700 1 --2- 172.21.15.154:0/287476420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c1780 0x7f470c0c1c10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f470c070da0 tx=0x7f470400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32411fd9da2079dd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470959a700 1 -- 172.21.15.154:0/287476420 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470400e070 con 0x7f470c0c1780 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470959a700 1 -- 172.21.15.154:0/287476420 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400bb20 con 0x7f470c0c1780 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.262+0000 7f470959a700 1 -- 172.21.15.154:0/287476420 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704012430 con 0x7f470c0c1780 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 -- 172.21.15.154:0/287476420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c1780 msgr2=0x7f470c0c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 --2- 172.21.15.154:0/287476420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c1780 0x7f470c0c1c10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f470c070da0 tx=0x7f470400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 -- 172.21.15.154:0/287476420 shutdown_connections 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 --2- 172.21.15.154:0/287476420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c1780 0x7f470c0c1c10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 --2- 172.21.15.154:0/287476420 >> v2:172.21.15.191:3300/0 conn(0x7f470c0a8540 0x7f470c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 -- 172.21.15.154:0/287476420 >> 172.21.15.154:0/287476420 conn(0x7f470c0c3880 msgr2=0x7f470c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:13.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 -- 172.21.15.154:0/287476420 shutdown_connections 2024-03-14T09:03:13.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.263+0000 7f471098f700 1 -- 172.21.15.154:0/287476420 wait complete. 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.264+0000 7f471098f700 1 Processor -- start 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.264+0000 7f471098f700 1 -- start start 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c0a8540 0x7f470c1ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f470c0c1780 0x7f470c0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c1ea1d0 con 0x7f470c0c3250 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0a9ea0 con 0x7f470c0c1780 2024-03-14T09:03:13.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f471098f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c05d0 con 0x7f470c0a8540 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56228/0 (socket says 172.21.15.154:56228) 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 -- 172.21.15.154:0/3057893799 learned_addr learned my addr 172.21.15.154:0/3057893799 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470a59c700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c0a8540 0x7f470c1ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f4709d9b700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f470c0c1780 0x7f470c0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 -- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c0a8540 msgr2=0x7f470c1ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c0a8540 0x7f470c1ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 -- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f470c0c1780 msgr2=0x7f470c0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f470c0c1780 0x7f470c0c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 -- 172.21.15.154:0/3057893799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704009a70 con 0x7f470c0c3250 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.265+0000 7f470ad9d700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f470400b370 tx=0x7f4704004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704012430 con 0x7f470c0c3250 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704020070 con 0x7f470c0c3250 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c0aa160 con 0x7f470c0c3250 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004280 con 0x7f470c0c3250 2024-03-14T09:03:13.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c0c6850 con 0x7f470c0c3250 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.266+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c0697d0 con 0x7f470c0c3250 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.267+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f47040043e0 con 0x7f470c0c3250 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.267+0000 7f46faffd700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 0x7f46ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.267+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4704022070 con 0x7f470c0c3250 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.267+0000 7f470a59c700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 0x7f46ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.268+0000 7f470a59c700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 0x7f46ec06ccf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.272+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4704055f80 con 0x7f470c0c3250 2024-03-14T09:03:13.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.465+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f470c0c29e0 con 0x7f470c0c3250 2024-03-14T09:03:13.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.466+0000 7f46faffd700 1 -- 172.21.15.154:0/3057893799 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+19873 (secure 0 0 0) 0x7f470402c090 con 0x7f470c0c3250 2024-03-14T09:03:13.466 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:13.467 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:12.760108+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:13.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 msgr2=0x7f46ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 0x7f46ec06ccf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4700006fd0 tx=0x7f4700008040 comp rx=0 tx=0).stop 2024-03-14T09:03:13.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 msgr2=0x7f470c1ea6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.468 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f470400b370 tx=0x7f4704004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 shutdown_connections 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f46ec06a830 0x7f46ec06ccf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f470c0c3250 0x7f470c1ea6b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c0a8540 0x7f470c1ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 --2- 172.21.15.154:0/3057893799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f470c0c1780 0x7f470c0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 >> 172.21.15.154:0/3057893799 conn(0x7f470c0c3880 msgr2=0x7f470c139700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 shutdown_connections 2024-03-14T09:03:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.468+0000 7f471098f700 1 -- 172.21.15.154:0/3057893799 wait complete. 2024-03-14T09:03:13.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-14T09:03:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.605+0000 7fa392969700 1 Processor -- start 2024-03-14T09:03:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa392969700 1 -- start start 2024-03-14T09:03:13.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa392969700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa38c14eaf0 0x7fa38c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa392969700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38c0ac8e0 con 0x7fa38c06acb0 2024-03-14T09:03:13.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa392969700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa38c06de30 con 0x7fa38c156500 2024-03-14T09:03:13.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa392969700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38c06b310 con 0x7fa38c14eaf0 2024-03-14T09:03:13.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa38b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa38c06acb0 0x7fa38c155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35204/0 (socket says 172.21.15.154:35204) 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.606+0000 7fa38b7fe700 1 -- 172.21.15.154:0/751222825 learned_addr learned my addr 172.21.15.154:0/751222825 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38bfff700 1 --2- 172.21.15.154:0/751222825 >> v2:172.21.15.191:3300/0 conn(0x7fa38c14eaf0 0x7fa38c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335265860 0 0) 0x7fa38c0ac8e0 con 0x7fa38c06acb0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3740035b0 con 0x7fa38c06acb0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100737928 0 0) 0x7fa3740035b0 con 0x7fa38c06acb0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 >> v2:172.21.15.191:3300/0 conn(0x7fa38c14eaf0 msgr2=0x7fa38c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 --2- 172.21.15.154:0/751222825 >> v2:172.21.15.191:3300/0 conn(0x7fa38c14eaf0 0x7fa38c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 >> v1:172.21.15.191:6789/0 conn(0x7fa38c156500 legacy=0x7fa38c15f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c156910 con 0x7fa38c06acb0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fa380001670 con 0x7fa38c06acb0 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 >> v1:172.21.15.154:6789/0 conn(0x7fa38c06acb0 legacy=0x7fa38c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374003a00 0x7fa374003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa374004850 0x7fa374004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 0x7fa37400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c06b310 con 0x7fa374005330 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400c200 con 0x7fa374004850 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400c3c0 con 0x7fa374003a00 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 0x7fa37400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38bfff700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374003a00 0x7fa374003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa38b7fe700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa374004850 0x7fa374004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 -- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374003a00 msgr2=0x7fa374003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374003a00 0x7fa374003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 -- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa374004850 msgr2=0x7fa374004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa374004850 0x7fa374004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.607+0000 7fa390f06700 1 -- 172.21.15.154:0/751222825 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740035b0 con 0x7fa374005330 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa390f06700 1 -- 172.21.15.154:0/751222825 reap_dead start 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa390f06700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 0x7fa37400ba10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c160 tx=0x7fa374005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e3db78c12e1f33 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384001db0 con 0x7fa374005330 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38400cca0 con 0x7fa374005330 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa38affd700 1 -- 172.21.15.154:0/751222825 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384003c00 con 0x7fa374005330 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 -- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 msgr2=0x7fa37400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 0x7fa37400ba10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c160 tx=0x7fa374005880 comp rx=0 tx=0).stop 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 -- 172.21.15.154:0/751222825 shutdown_connections 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 --2- 172.21.15.154:0/751222825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa374005330 0x7fa37400ba10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 -- 172.21.15.154:0/751222825 >> 172.21.15.154:0/751222825 conn(0x7fa38c0c3880 msgr2=0x7fa38c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 -- 172.21.15.154:0/751222825 shutdown_connections 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.608+0000 7fa392969700 1 -- 172.21.15.154:0/751222825 wait complete. 2024-03-14T09:03:13.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.609+0000 7fa392969700 1 Processor -- start 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 -- start start 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374005330 0x7fa38c141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 0x7fa38c144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c141580 con 0x7fa38c13b700 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c0c3500 con 0x7fa38c0c1470 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa392969700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38c1ede60 con 0x7fa374005330 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa390f06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa390f06700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56250/0 (socket says 172.21.15.154:56250) 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.610+0000 7fa390f06700 1 -- 172.21.15.154:0/457795021 learned_addr learned my addr 172.21.15.154:0/457795021 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa38bfff700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374005330 0x7fa38c141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa38b7fe700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 0x7fa38c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 -- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374005330 msgr2=0x7fa38c141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374005330 0x7fa38c141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 -- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 msgr2=0x7fa38c144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 0x7fa38c144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 -- 172.21.15.154:0/457795021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c1ee140 con 0x7fa38c13b700 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa38b7fe700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 0x7fa38c144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa390f06700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa384005040 tx=0x7fa3840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384005070 con 0x7fa38c13b700 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3840051d0 con 0x7fa38c13b700 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384005560 con 0x7fa38c13b700 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38c0c2eb0 con 0x7fa38c13b700 2024-03-14T09:03:13.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.611+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38c1446a0 con 0x7fa38c13b700 2024-03-14T09:03:13.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.612+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c1ee2d0 con 0x7fa38c13b700 2024-03-14T09:03:13.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.612+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa3840056c0 con 0x7fa38c13b700 2024-03-14T09:03:13.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.613+0000 7fa388ff9700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 0x7fa37806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:13.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.613+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa384089f30 con 0x7fa38c13b700 2024-03-14T09:03:13.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.613+0000 7fa38bfff700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 0x7fa37806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:13.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.613+0000 7fa38bfff700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 0x7fa37806ccf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009be0 tx=0x7fa37c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:13.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.617+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa384055080 con 0x7fa38c13b700 2024-03-14T09:03:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:13.860+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa38c0c3670 con 0x7fa38c13b700 2024-03-14T09:03:14.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.773+0000 7fa388ff9700 1 -- 172.21.15.154:0/457795021 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 119+0+0 (secure 0 0 0) 0x7fa384016020 con 0x7fa38c13b700 2024-03-14T09:03:14.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 msgr2=0x7fa37806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:14.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 0x7fa37806ccf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009be0 tx=0x7fa37c009210 comp rx=0 tx=0).stop 2024-03-14T09:03:14.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 msgr2=0x7fa38c13bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:14.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa384005040 tx=0x7fa3840014a0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 shutdown_connections 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa37806a830 0x7fa37806ccf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa38c13b700 0x7fa38c13bb80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa374005330 0x7fa38c141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 --2- 172.21.15.154:0/457795021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38c0c1470 0x7fa38c144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 >> 172.21.15.154:0/457795021 conn(0x7fa38c0c3880 msgr2=0x7fa38c155cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 shutdown_connections 2024-03-14T09:03:14.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.776+0000 7fa392969700 1 -- 172.21.15.154:0/457795021 wait complete. 2024-03-14T09:03:14.791 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:03:14.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-14T09:03:14.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.922+0000 7f835dc61700 1 Processor -- start 2024-03-14T09:03:14.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.922+0000 7f835dc61700 1 -- start start 2024-03-14T09:03:14.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f835dc61700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83580c80e0 0x7f83580c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f835dc61700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 0x7f83580c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f835dc61700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83580c6720 con 0x7f83580c7570 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f835dc61700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580c57b0 con 0x7f83580c80e0 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f835dc61700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83580c4f10 con 0x7f83580c6bb0 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f83577fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83580c80e0 0x7f83580c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f83577fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83580c80e0 0x7f83580c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56262/0 (socket says 172.21.15.154:56262) 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.923+0000 7f83577fe700 1 -- 172.21.15.154:0/404985701 learned_addr learned my addr 172.21.15.154:0/404985701 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f8356ffd700 1 --2- 172.21.15.154:0/404985701 >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 0x7f83580c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83577fe700 1 -- 172.21.15.154:0/404985701 >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 msgr2=0x7f83580c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83577fe700 1 --2- 172.21.15.154:0/404985701 >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 0x7f83580c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83577fe700 1 -- 172.21.15.154:0/404985701 >> v1:172.21.15.191:6789/0 conn(0x7f83580c7570 legacy=0x7f8358155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83577fe700 1 -- 172.21.15.154:0/404985701 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83580c51f0 con 0x7f83580c80e0 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f8356ffd700 1 --2- 172.21.15.154:0/404985701 >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 0x7f83580c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:14.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83577fe700 1 --2- 172.21.15.154:0/404985701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f83580c45e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8358070570 tx=0x7f8348009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526f937fecbc7e28 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83567fc700 1 -- 172.21.15.154:0/404985701 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348004230 con 0x7f83580c80e0 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83567fc700 1 -- 172.21.15.154:0/404985701 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348004390 con 0x7f83580c80e0 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f83567fc700 1 -- 172.21.15.154:0/404985701 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480036a0 con 0x7f83580c80e0 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 -- 172.21.15.154:0/404985701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 msgr2=0x7f83580c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 --2- 172.21.15.154:0/404985701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f83580c45e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8358070570 tx=0x7f8348009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 -- 172.21.15.154:0/404985701 shutdown_connections 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 --2- 172.21.15.154:0/404985701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f83580c45e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 --2- 172.21.15.154:0/404985701 >> v2:172.21.15.191:3300/0 conn(0x7f83580c6bb0 0x7f83580c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.924+0000 7f835dc61700 1 -- 172.21.15.154:0/404985701 >> 172.21.15.154:0/404985701 conn(0x7f835806a770 msgr2=0x7f835806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.925+0000 7f835dc61700 1 -- 172.21.15.154:0/404985701 shutdown_connections 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.925+0000 7f835dc61700 1 -- 172.21.15.154:0/404985701 wait complete. 2024-03-14T09:03:14.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.926+0000 7f835dc61700 1 Processor -- start 2024-03-14T09:03:14.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.926+0000 7f835dc61700 1 -- start start 2024-03-14T09:03:14.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.926+0000 7f835dc61700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 0x7f83580c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.926+0000 7f835dc61700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580c7570 0x7f835809b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.926+0000 7f835dc61700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f835dc61700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83580c3ac0 con 0x7f83580c80e0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f835dc61700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835809dab0 con 0x7f83580c6bb0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f835dc61700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835809e7f0 con 0x7f83580c7570 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56268/0 (socket says 172.21.15.154:56268) 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 -- 172.21.15.154:0/2831018338 learned_addr learned my addr 172.21.15.154:0/2831018338 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f83577fe700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 0x7f83580c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8356ffd700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580c7570 0x7f835809b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 -- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580c7570 msgr2=0x7f835809b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580c7570 0x7f835809b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 -- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 msgr2=0x7f83580c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 0x7f83580c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 -- 172.21.15.154:0/2831018338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009550 con 0x7f83580c80e0 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f83577fe700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 0x7f83580c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:14.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f8357fff700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f835000cc10 tx=0x7f835000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000abb0 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835000ad10 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350012660 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83580c2bf0 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.927+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83580c4470 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.928+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8358067bd0 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.929+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f83500127c0 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.929+0000 7f833ffff700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 0x7f834406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.929+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f8350089800 con 0x7f83580c80e0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.929+0000 7f83577fe700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 0x7f834406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:14.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.930+0000 7f83577fe700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 0x7f834406ccf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8348009130 tx=0x7f8348008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:14.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:14.934+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8350054950 con 0x7f83580c80e0 2024-03-14T09:03:15.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.170+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f83580c3c00 con 0x7f83580c80e0 2024-03-14T09:03:15.547 INFO:tasks.ceph.mds.a.smithi154.stderr:2024-03-14T09:03:15.547+0000 7f2a5ea9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:03:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.776+0000 7f833ffff700 1 -- 172.21.15.154:0/2831018338 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 123+0+0 (secure 0 0 0) 0x7f83580c3c00 con 0x7f83580c80e0 2024-03-14T09:03:15.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 msgr2=0x7f834406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:15.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 0x7f834406ccf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8348009130 tx=0x7f8348008030 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 msgr2=0x7f835809a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f835000cc10 tx=0x7f835000a060 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 shutdown_connections 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f834406a830 0x7f834406ccf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83580c80e0 0x7f835809a820 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83580c6bb0 0x7f83580c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 --2- 172.21.15.154:0/2831018338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83580c7570 0x7f835809b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 >> 172.21.15.154:0/2831018338 conn(0x7f835806a770 msgr2=0x7f83580a9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 shutdown_connections 2024-03-14T09:03:15.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.778+0000 7f835dc61700 1 -- 172.21.15.154:0/2831018338 wait complete. 2024-03-14T09:03:15.791 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-14T09:03:15.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:15.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.914+0000 7fc2658b4700 1 Processor -- start 2024-03-14T09:03:15.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.914+0000 7fc2658b4700 1 -- start start 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc2658b4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc26014e550 0x7fc260150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc2658b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 0x7fc26015b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc2658b4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26006d940 con 0x7fc26006c740 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc2658b4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26014e9d0 con 0x7fc26014e550 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc2658b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600aa800 con 0x7fc260151400 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc26014e550 0x7fc260150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc26014e550 0x7fc260150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56280/0 (socket says 172.21.15.154:56280) 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 -- 172.21.15.154:0/1465992776 learned_addr learned my addr 172.21.15.154:0/1465992776 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 -- 172.21.15.154:0/1465992776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 msgr2=0x7fc26015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25f7fe700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 0x7fc26015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 0x7fc26015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 -- 172.21.15.154:0/1465992776 >> v1:172.21.15.191:6789/0 conn(0x7fc26006c740 legacy=0x7fc2600c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 -- 172.21.15.154:0/1465992776 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26006cd30 con 0x7fc26014e550 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25f7fe700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 0x7fc26015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25e7fc700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc26014e550 0x7fc260150ec0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098f0 tx=0x7fc254009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81d63c2c28e696a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25dffb700 1 -- 172.21.15.154:0/1465992776 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254004230 con 0x7fc26014e550 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25dffb700 1 -- 172.21.15.154:0/1465992776 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254004390 con 0x7fc26014e550 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.915+0000 7fc25dffb700 1 -- 172.21.15.154:0/1465992776 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540036a0 con 0x7fc26014e550 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 -- 172.21.15.154:0/1465992776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc26014e550 msgr2=0x7fc260150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc26014e550 0x7fc260150ec0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098f0 tx=0x7fc254009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 -- 172.21.15.154:0/1465992776 shutdown_connections 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc260151400 0x7fc26015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 --2- 172.21.15.154:0/1465992776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc26014e550 0x7fc260150ec0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 -- 172.21.15.154:0/1465992776 >> 172.21.15.154:0/1465992776 conn(0x7fc2600c5e70 msgr2=0x7fc2600c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:15.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 -- 172.21.15.154:0/1465992776 shutdown_connections 2024-03-14T09:03:15.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.916+0000 7fc2658b4700 1 -- 172.21.15.154:0/1465992776 wait complete. 2024-03-14T09:03:15.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.917+0000 7fc2658b4700 1 Processor -- start 2024-03-14T09:03:15.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.917+0000 7fc2658b4700 1 -- start start 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc26006c740 0x7fc260153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc26014e550 0x7fc2601ea340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600be130 con 0x7fc260151400 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600bba30 con 0x7fc26006c740 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc2658b4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc260153fa0 con 0x7fc26014e550 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56284/0 (socket says 172.21.15.154:56284) 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 -- 172.21.15.154:0/3341127908 learned_addr learned my addr 172.21.15.154:0/3341127908 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:15.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25e7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc26014e550 0x7fc2601ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25effd700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc26006c740 0x7fc260153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 -- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc26014e550 msgr2=0x7fc2601ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc26014e550 0x7fc2601ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 -- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc26006c740 msgr2=0x7fc260153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc26006c740 0x7fc260153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 -- 172.21.15.154:0/3341127908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009550 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.918+0000 7fc25f7fe700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc25800cdb0 tx=0x7fc25800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.919+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f480 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.919+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800f600 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.919+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800f9b0 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.919+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc260150230 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.919+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2601ea880 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.920+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc260073f10 con 0x7fc260151400 2024-03-14T09:03:15.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.920+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc25800b040 con 0x7fc260151400 2024-03-14T09:03:15.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.920+0000 7fc247fff700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 0x7fc24c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:15.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.921+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fc258089730 con 0x7fc260151400 2024-03-14T09:03:15.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.921+0000 7fc25effd700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 0x7fc24c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:15.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.921+0000 7fc25effd700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 0x7fc24c06cd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc2600a9610 tx=0x7fc2500090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:15.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:15.925+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc2580547e0 con 0x7fc260151400 2024-03-14T09:03:16.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.165+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2601ecf00 con 0x7fc260151400 2024-03-14T09:03:16.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.166+0000 7fc247fff700 1 -- 172.21.15.154:0/3341127908 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5956 (secure 0 0 0) 0x7fc2601ecf00 con 0x7fc260151400 2024-03-14T09:03:16.165 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:16.166 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 msgr2=0x7fc24c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 0x7fc24c06cd20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc2600a9610 tx=0x7fc2500090e0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 msgr2=0x7fc2601ed860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc25800cdb0 tx=0x7fc25800a560 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 shutdown_connections 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc24c06a860 0x7fc24c06cd20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc260151400 0x7fc2601ed860 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc26006c740 0x7fc260153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 --2- 172.21.15.154:0/3341127908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc26014e550 0x7fc2601ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 >> 172.21.15.154:0/3341127908 conn(0x7fc2600c5e70 msgr2=0x7fc26014cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:16.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 shutdown_connections 2024-03-14T09:03:16.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.168+0000 7fc2658b4700 1 -- 172.21.15.154:0/3341127908 wait complete. 2024-03-14T09:03:16.169 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 35 2024-03-14T09:03:16.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:16.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 Processor -- start 2024-03-14T09:03:16.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 -- start start 2024-03-14T09:03:16.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00600c5f10 con 0x7f006006c6e0 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00600c7d50 con 0x7f00600c31c0 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.303+0000 7f00686d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00600c8030 con 0x7f0060070a10 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0066c6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:54614/0 (socket says 172.21.15.154:54614) 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0066c6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:47744/0 (socket says 172.21.15.154:47744) 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 -- 172.21.15.154:0/300324998 learned_addr learned my addr 172.21.15.154:0/300324998 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f006546b700 1 -- 172.21.15.154:0/300324998 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233016757 0 0) 0x7f00600c5f10 con 0x7f006006c6e0 2024-03-14T09:03:16.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f006546b700 1 -- 172.21.15.154:0/300324998 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004c0035b0 con 0x7f006006c6e0 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 -- 172.21.15.154:0/300324998 >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 msgr2=0x7f00600c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 --2- 172.21.15.154:0/300324998 >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 -- 172.21.15.154:0/300324998 >> v1:172.21.15.191:6789/0 conn(0x7f006006c6e0 legacy=0x7f006006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 -- 172.21.15.154:0/300324998 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600c8310 con 0x7f0060070a10 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0066c6e700 1 --2- 172.21.15.154:0/300324998 >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.304+0000 7f0065c6c700 1 --2- 172.21.15.154:0/300324998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098f0 tx=0x7f005c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbcaeb811f348f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f006546b700 1 -- 172.21.15.154:0/300324998 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011830 con 0x7f0060070a10 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f006546b700 1 -- 172.21.15.154:0/300324998 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c011990 con 0x7f0060070a10 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f006546b700 1 -- 172.21.15.154:0/300324998 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c011d00 con 0x7f0060070a10 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 -- 172.21.15.154:0/300324998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 msgr2=0x7f00600c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 --2- 172.21.15.154:0/300324998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098f0 tx=0x7f005c009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 -- 172.21.15.154:0/300324998 shutdown_connections 2024-03-14T09:03:16.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 --2- 172.21.15.154:0/300324998 >> v2:172.21.15.191:3300/0 conn(0x7f00600c31c0 0x7f00600c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 --2- 172.21.15.154:0/300324998 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0060070a10 0x7f00600c63a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 -- 172.21.15.154:0/300324998 >> 172.21.15.154:0/300324998 conn(0x7f00600a9bf0 msgr2=0x7f00600a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:16.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 -- 172.21.15.154:0/300324998 shutdown_connections 2024-03-14T09:03:16.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.305+0000 7f00686d1700 1 -- 172.21.15.154:0/300324998 wait complete. 2024-03-14T09:03:16.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.306+0000 7f00686d1700 1 Processor -- start 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.306+0000 7f00686d1700 1 -- start start 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006006c6e0 0x7f00601e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00600c31c0 0x7f00600abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006009a640 con 0x7f0060070a10 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0060099c10 con 0x7f006006c6e0 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f00686d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0060099ef0 con 0x7f00600c31c0 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0065c6c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0065c6c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56296/0 (socket says 172.21.15.154:56296) 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0065c6c700 1 -- 172.21.15.154:0/2360054690 learned_addr learned my addr 172.21.15.154:0/2360054690 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0066c6e700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00600c31c0 0x7f00600abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f006646d700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006006c6e0 0x7f00601e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0065c6c700 1 -- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00600c31c0 msgr2=0x7f00600abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.307+0000 7f0065c6c700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00600c31c0 0x7f00600abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0065c6c700 1 -- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006006c6e0 msgr2=0x7f00601e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0065c6c700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006006c6e0 0x7f00601e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0065c6c700 1 -- 172.21.15.154:0/2360054690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058009a70 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0065c6c700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f005c009c40 tx=0x7f005c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0043c0 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c01f070 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c009550 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0036a0 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.308+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00601ddb10 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.309+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00600697d0 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.309+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f005c021030 con 0x7f0060070a10 2024-03-14T09:03:16.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.310+0000 7f0052ffd700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 0x7f004406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.310+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f005c08a2b0 con 0x7f0060070a10 2024-03-14T09:03:16.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.310+0000 7f006646d700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 0x7f004406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.310+0000 7f006646d700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 0x7f004406ccf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0054001830 tx=0x7f0054009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.314+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f005c055520 con 0x7f0060070a10 2024-03-14T09:03:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.512+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00600c2d90 con 0x7f0060070a10 2024-03-14T09:03:16.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.514+0000 7f0052ffd700 1 -- 172.21.15.154:0/2360054690 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7f005c058b40 con 0x7f0060070a10 2024-03-14T09:03:16.514 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:16.514 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 msgr2=0x7f004406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 0x7f004406ccf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0054001830 tx=0x7f0054009040 comp rx=0 tx=0).stop 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 msgr2=0x7f00600ab920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f005c009c40 tx=0x7f005c004750 comp rx=0 tx=0).stop 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 shutdown_connections 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00600c31c0 0x7f00600abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f006006c6e0 0x7f00601e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f004406a830 0x7f004406ccf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 --2- 172.21.15.154:0/2360054690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0060070a10 0x7f00600ab920 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 >> 172.21.15.154:0/2360054690 conn(0x7f00600a9bf0 msgr2=0x7f00600a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 shutdown_connections 2024-03-14T09:03:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.516+0000 7f00686d1700 1 -- 172.21.15.154:0/2360054690 wait complete. 2024-03-14T09:03:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:03:16.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:16.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 Processor -- start 2024-03-14T09:03:16.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 -- start start 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fafc014c5c0 0x7fafc0152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc0072290 0x7fafc0072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fafc00c1fd0 0x7fafc00c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00c1bb0 con 0x7fafc00c1fd0 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00c6170 con 0x7fafc014c5c0 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc7249700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0070e10 con 0x7fafc0072290 2024-03-14T09:03:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.651+0000 7fafc57e6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fafc00c1fd0 0x7fafc00c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fafc00c1fd0 0x7fafc00c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56300/0 (socket says 172.21.15.154:56300) 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 -- 172.21.15.154:0/2168662641 learned_addr learned my addr 172.21.15.154:0/2168662641 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafbffff700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc0072290 0x7fafc0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc4fe5700 1 --2- 172.21.15.154:0/2168662641 >> v2:172.21.15.191:3300/0 conn(0x7fafc014c5c0 0x7fafc0152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 -- 172.21.15.154:0/2168662641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc0072290 msgr2=0x7fafc0072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc0072290 0x7fafc0072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 -- 172.21.15.154:0/2168662641 >> v2:172.21.15.191:3300/0 conn(0x7fafc014c5c0 msgr2=0x7fafc0152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2168662641 >> v2:172.21.15.191:3300/0 conn(0x7fafc014c5c0 0x7fafc0152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 -- 172.21.15.154:0/2168662641 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc014c9a0 con 0x7fafc00c1fd0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc00c1fd0 0x7fafc00c2460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafc00a9340 tx=0x7fafb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df95abbc382cebe server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafbf7fe700 1 -- 172.21.15.154:0/2168662641 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800e070 con 0x7fafc00c1fd0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafbf7fe700 1 -- 172.21.15.154:0/2168662641 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800b910 con 0x7fafc00c1fd0 2024-03-14T09:03:16.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.652+0000 7fafbf7fe700 1 -- 172.21.15.154:0/2168662641 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800bcc0 con 0x7fafc00c1fd0 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 -- 172.21.15.154:0/2168662641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc00c1fd0 msgr2=0x7fafc00c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc00c1fd0 0x7fafc00c2460 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafc00a9340 tx=0x7fafb800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 -- 172.21.15.154:0/2168662641 shutdown_connections 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc00c1fd0 0x7fafc00c2460 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 --2- 172.21.15.154:0/2168662641 >> v2:172.21.15.191:3300/0 conn(0x7fafc014c5c0 0x7fafc0152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 --2- 172.21.15.154:0/2168662641 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc0072290 0x7fafc0072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 -- 172.21.15.154:0/2168662641 >> 172.21.15.154:0/2168662641 conn(0x7fafc006a770 msgr2=0x7fafc006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 -- 172.21.15.154:0/2168662641 shutdown_connections 2024-03-14T09:03:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.653+0000 7fafc7249700 1 -- 172.21.15.154:0/2168662641 wait complete. 2024-03-14T09:03:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.654+0000 7fafc7249700 1 Processor -- start 2024-03-14T09:03:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.654+0000 7fafc7249700 1 -- start start 2024-03-14T09:03:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc0072290 0x7fafc00aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00c1fd0 0x7fafc01ea470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0149950 con 0x7fafc014c5c0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00abc80 con 0x7fafc0072290 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc7249700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00aa590 con 0x7fafc00c1fd0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56310/0 (socket says 172.21.15.154:56310) 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 -- 172.21.15.154:0/2274216182 learned_addr learned my addr 172.21.15.154:0/2274216182 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafbffff700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00c1fd0 0x7fafc01ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc4fe5700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc0072290 0x7fafc00aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 -- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00c1fd0 msgr2=0x7fafc01ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00c1fd0 0x7fafc01ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 -- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc0072290 msgr2=0x7fafc00aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc0072290 0x7fafc00aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 -- 172.21.15.154:0/2274216182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009b30 con 0x7fafc014c5c0 2024-03-14T09:03:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.655+0000 7fafc57e6700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fafc0074020 tx=0x7fafb8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.656+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb80092e0 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.656+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800ce80 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.656+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc00ac750 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.656+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb80044b0 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.656+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00abeb0 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.657+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc01ed1b0 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.657+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fafb8004610 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.658+0000 7fafbd7fa700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 0x7fafa406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.658+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fafb808a670 con 0x7fafc014c5c0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.658+0000 7fafc4fe5700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 0x7fafa406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.659+0000 7fafc4fe5700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 0x7fafa406ccf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009b40 tx=0x7fafb4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:16.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.662+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fafb80557a0 con 0x7fafc014c5c0 2024-03-14T09:03:16.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.853+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fafc0072d80 con 0x7fafa406a830 2024-03-14T09:03:16.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.867+0000 7fafbd7fa700 1 -- 172.21.15.154:0/2274216182 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524924 (secure 0 0 0) 0x7fafb4011810 con 0x7fafa406a830 2024-03-14T09:03:16.874 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 msgr2=0x7fafa406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 0x7fafa406ccf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009b40 tx=0x7fafb4009210 comp rx=0 tx=0).stop 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 msgr2=0x7fafc00bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fafc0074020 tx=0x7fafb8004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 shutdown_connections 2024-03-14T09:03:16.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fafa406a830 0x7fafa406ccf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafc014c5c0 0x7fafc00bf9c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafc0072290 0x7fafc00aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 --2- 172.21.15.154:0/2274216182 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafc00c1fd0 0x7fafc01ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 >> 172.21.15.154:0/2274216182 conn(0x7fafc006a770 msgr2=0x7fafc014fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 shutdown_connections 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:16.872+0000 7fafc7249700 1 -- 172.21.15.154:0/2274216182 wait complete. 2024-03-14T09:03:16.880 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:16.884 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-03-14T09:03:15.544985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222312,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752408,"statfs":{"total":773094113280,"available":772866465792,"internally_reserved":0,"allocated":6283264,"data_stored":3329522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.196195"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353917+0000","last_change":"2024-03-14T09:03:12.353917+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353917+0000","last_undegraded":"2024-03-14T09:03:12.353917+0000","last_fullsized":"2024-03-14T09:03:12.353917+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353360+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:11.353360+0000","last_peered":"2024-03-14T09:03:11.353360+0000","last_clean":"2024-03-14T09:03:11.353360+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:11.353360+0000","last_undegraded":"2024-03-14T09:03:11.353360+0000","last_fullsized":"2024-03-14T09:03:11.353360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361638+0000","last_change":"2024-03-14T09:03:12.361638+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361638+0000","last_undegraded":"2024-03-14T09:03:12.361638+0000","last_fullsized":"2024-03-14T09:03:12.361638+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353404+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:11.353404+0000","last_peered":"2024-03-14T09:03:11.353404+0000","last_clean":"2024-03-14T09:03:11.353404+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:11.353404+0000","last_undegraded":"2024-03-14T09:03:11.353404+0000","last_fullsized":"2024-03-14T09:03:11.353404+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358323+0000","last_change":"2024-03-14T09:03:12.358323+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358323+0000","last_undegraded":"2024-03-14T09:03:12.358323+0000","last_fullsized":"2024-03-14T09:03:12.358323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358248+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:11.358248+0000","last_peered":"2024-03-14T09:03:11.358248+0000","last_clean":"2024-03-14T09:03:11.358248+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:11.358248+0000","last_undegraded":"2024-03-14T09:03:11.358248+0000","last_fullsized":"2024-03-14T09:03:11.358248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354698+0000","last_change":"2024-03-14T09:03:12.354698+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354698+0000","last_undegraded":"2024-03-14T09:03:12.354698+0000","last_fullsized":"2024-03-14T09:03:12.354698+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355314+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:11.355314+0000","last_peered":"2024-03-14T09:03:11.355314+0000","last_clean":"2024-03-14T09:03:11.355314+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:11.355314+0000","last_undegraded":"2024-03-14T09:03:11.355314+0000","last_fullsized":"2024-03-14T09:03:11.355314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362720+0000","last_change":"2024-03-14T09:03:12.362720+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362720+0000","last_undegraded":"2024-03-14T09:03:12.362720+0000","last_fullsized":"2024-03-14T09:03:12.362720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353370+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:11.353370+0000","last_peered":"2024-03-14T09:03:11.353370+0000","last_clean":"2024-03-14T09:03:11.353370+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:11.353370+0000","last_undegraded":"2024-03-14T09:03:11.353370+0000","last_fullsized":"2024-03-14T09:03:11.353370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353623+0000","last_change":"2024-03-14T09:03:12.353623+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353623+0000","last_undegraded":"2024-03-14T09:03:12.353623+0000","last_fullsized":"2024-03-14T09:03:12.353623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349110+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:11.349110+0000","last_peered":"2024-03-14T09:03:11.349110+0000","last_clean":"2024-03-14T09:03:11.349110+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:11.349110+0000","last_undegraded":"2024-03-14T09:03:11.349110+0000","last_fullsized":"2024-03-14T09:03:11.349110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360227+0000","last_change":"2024-03-14T09:03:12.360227+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360227+0000","last_undegraded":"2024-03-14T09:03:12.360227+0000","last_fullsized":"2024-03-14T09:03:12.360227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355387+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:11.355387+0000","last_peered":"2024-03-14T09:03:11.355387+0000","last_clean":"2024-03-14T09:03:11.355387+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:11.355387+0000","last_undegraded":"2024-03-14T09:03:11.355387+0000","last_fullsized":"2024-03-14T09:03:11.355387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353748+0000","last_change":"2024-03-14T09:03:12.353748+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353748+0000","last_undegraded":"2024-03-14T09:03:12.353748+0000","last_fullsized":"2024-03-14T09:03:12.353748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351097+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:11.351097+0000","last_peered":"2024-03-14T09:03:11.351097+0000","last_clean":"2024-03-14T09:03:11.351097+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:11.351097+0000","last_undegraded":"2024-03-14T09:03:11.351097+0000","last_fullsized":"2024-03-14T09:03:11.351097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357388+0000","last_change":"2024-03-14T09:03:12.357388+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357388+0000","last_undegraded":"2024-03-14T09:03:12.357388+0000","last_fullsized":"2024-03-14T09:03:12.357388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356978+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:11.356978+0000","last_peered":"2024-03-14T09:03:11.356978+0000","last_clean":"2024-03-14T09:03:11.356978+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:11.356978+0000","last_undegraded":"2024-03-14T09:03:11.356978+0000","last_fullsized":"2024-03-14T09:03:11.356978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362629+0000","last_change":"2024-03-14T09:03:12.362629+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362629+0000","last_undegraded":"2024-03-14T09:03:12.362629+0000","last_fullsized":"2024-03-14T09:03:12.362629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357377+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:11.357377+0000","last_peered":"2024-03-14T09:03:11.357377+0000","last_clean":"2024-03-14T09:03:11.357377+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:11.357377+0000","last_undegraded":"2024-03-14T09:03:11.357377+0000","last_fullsized":"2024-03-14T09:03:11.357377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359448+0000","last_change":"2024-03-14T09:03:12.359448+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359448+0000","last_undegraded":"2024-03-14T09:03:12.359448+0000","last_fullsized":"2024-03-14T09:03:12.359448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357336+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:11.357336+0000","last_peered":"2024-03-14T09:03:11.357336+0000","last_clean":"2024-03-14T09:03:11.357336+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:11.357336+0000","last_undegraded":"2024-03-14T09:03:11.357336+0000","last_fullsized":"2024-03-14T09:03:11.357336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364654+0000","last_change":"2024-03-14T09:03:12.364654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364654+0000","last_undegraded":"2024-03-14T09:03:12.364654+0000","last_fullsized":"2024-03-14T09:03:12.364654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355433+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:11.355433+0000","last_peered":"2024-03-14T09:03:11.355433+0000","last_clean":"2024-03-14T09:03:11.355433+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:11.355433+0000","last_undegraded":"2024-03-14T09:03:11.355433+0000","last_fullsized":"2024-03-14T09:03:11.355433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356123+0000","last_change":"2024-03-14T09:03:12.356123+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356123+0000","last_undegraded":"2024-03-14T09:03:12.356123+0000","last_fullsized":"2024-03-14T09:03:12.356123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353947+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:11.353947+0000","last_peered":"2024-03-14T09:03:11.353947+0000","last_clean":"2024-03-14T09:03:11.353947+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:11.353947+0000","last_undegraded":"2024-03-14T09:03:11.353947+0000","last_fullsized":"2024-03-14T09:03:11.353947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361949+0000","last_change":"2024-03-14T09:03:12.361949+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361949+0000","last_undegraded":"2024-03-14T09:03:12.361949+0000","last_fullsized":"2024-03-14T09:03:12.361949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355064+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:11.355064+0000","last_peered":"2024-03-14T09:03:11.355064+0000","last_clean":"2024-03-14T09:03:11.355064+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:11.355064+0000","last_undegraded":"2024-03-14T09:03:11.355064+0000","last_fullsized":"2024-03-14T09:03:11.355064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355497+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:11.355497+0000","last_peered":"2024-03-14T09:03:11.355497+0000","last_clean":"2024-03-14T09:03:11.355497+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:11.355497+0000","last_undegraded":"2024-03-14T09:03:11.355497+0000","last_fullsized":"2024-03-14T09:03:11.355497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364020+0000","last_change":"2024-03-14T09:03:12.364020+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364020+0000","last_undegraded":"2024-03-14T09:03:12.364020+0000","last_fullsized":"2024-03-14T09:03:12.364020+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.348935+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:11.348935+0000","last_peered":"2024-03-14T09:03:11.348935+0000","last_clean":"2024-03-14T09:03:11.348935+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:11.348935+0000","last_undegraded":"2024-03-14T09:03:11.348935+0000","last_fullsized":"2024-03-14T09:03:11.348935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353377+0000","last_change":"2024-03-14T09:03:12.353377+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353377+0000","last_undegraded":"2024-03-14T09:03:12.353377+0000","last_fullsized":"2024-03-14T09:03:12.353377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353394+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:11.353394+0000","last_peered":"2024-03-14T09:03:11.353394+0000","last_clean":"2024-03-14T09:03:11.353394+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:11.353394+0000","last_undegraded":"2024-03-14T09:03:11.353394+0000","last_fullsized":"2024-03-14T09:03:11.353394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356427+0000","last_change":"2024-03-14T09:03:12.356427+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356427+0000","last_undegraded":"2024-03-14T09:03:12.356427+0000","last_fullsized":"2024-03-14T09:03:12.356427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358442+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:11.358442+0000","last_peered":"2024-03-14T09:03:11.358442+0000","last_clean":"2024-03-14T09:03:11.358442+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:11.358442+0000","last_undegraded":"2024-03-14T09:03:11.358442+0000","last_fullsized":"2024-03-14T09:03:11.358442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360390+0000","last_change":"2024-03-14T09:03:12.360390+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360390+0000","last_undegraded":"2024-03-14T09:03:12.360390+0000","last_fullsized":"2024-03-14T09:03:12.360390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353260+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:11.353260+0000","last_peered":"2024-03-14T09:03:11.353260+0000","last_clean":"2024-03-14T09:03:11.353260+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:11.353260+0000","last_undegraded":"2024-03-14T09:03:11.353260+0000","last_fullsized":"2024-03-14T09:03:11.353260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359276+0000","last_change":"2024-03-14T09:03:12.359276+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359276+0000","last_undegraded":"2024-03-14T09:03:12.359276+0000","last_fullsized":"2024-03-14T09:03:12.359276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351650+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:11.351650+0000","last_peered":"2024-03-14T09:03:11.351650+0000","last_clean":"2024-03-14T09:03:11.351650+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:11.351650+0000","last_undegraded":"2024-03-14T09:03:11.351650+0000","last_fullsized":"2024-03-14T09:03:11.351650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356216+0000","last_change":"2024-03-14T09:03:12.356216+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356216+0000","last_undegraded":"2024-03-14T09:03:12.356216+0000","last_fullsized":"2024-03-14T09:03:12.356216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358150+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:11.358150+0000","last_peered":"2024-03-14T09:03:11.358150+0000","last_clean":"2024-03-14T09:03:11.358150+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:11.358150+0000","last_undegraded":"2024-03-14T09:03:11.358150+0000","last_fullsized":"2024-03-14T09:03:11.358150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357282+0000","last_change":"2024-03-14T09:03:12.357282+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357282+0000","last_undegraded":"2024-03-14T09:03:12.357282+0000","last_fullsized":"2024-03-14T09:03:12.357282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351242+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:11.351242+0000","last_peered":"2024-03-14T09:03:11.351242+0000","last_clean":"2024-03-14T09:03:11.351242+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:11.351242+0000","last_undegraded":"2024-03-14T09:03:11.351242+0000","last_fullsized":"2024-03-14T09:03:11.351242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364628+0000","last_change":"2024-03-14T09:03:12.364628+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364628+0000","last_undegraded":"2024-03-14T09:03:12.364628+0000","last_fullsized":"2024-03-14T09:03:12.364628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358083+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:11.358083+0000","last_peered":"2024-03-14T09:03:11.358083+0000","last_clean":"2024-03-14T09:03:11.358083+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:11.358083+0000","last_undegraded":"2024-03-14T09:03:11.358083+0000","last_fullsized":"2024-03-14T09:03:11.358083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362012+0000","last_change":"2024-03-14T09:03:12.362012+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362012+0000","last_undegraded":"2024-03-14T09:03:12.362012+0000","last_fullsized":"2024-03-14T09:03:12.362012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353228+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.353228+0000","last_peered":"2024-03-14T09:03:11.353228+0000","last_clean":"2024-03-14T09:03:11.353228+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:11.353228+0000","last_undegraded":"2024-03-14T09:03:11.353228+0000","last_fullsized":"2024-03-14T09:03:11.353228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352812+0000","last_change":"2024-03-14T09:03:12.352812+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352812+0000","last_undegraded":"2024-03-14T09:03:12.352812+0000","last_fullsized":"2024-03-14T09:03:12.352812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353434+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:11.353434+0000","last_peered":"2024-03-14T09:03:11.353434+0000","last_clean":"2024-03-14T09:03:11.353434+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:11.353434+0000","last_undegraded":"2024-03-14T09:03:11.353434+0000","last_fullsized":"2024-03-14T09:03:11.353434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361474+0000","last_change":"2024-03-14T09:03:12.361474+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361474+0000","last_undegraded":"2024-03-14T09:03:12.361474+0000","last_fullsized":"2024-03-14T09:03:12.361474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355459+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:11.355459+0000","last_peered":"2024-03-14T09:03:11.355459+0000","last_clean":"2024-03-14T09:03:11.355459+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:11.355459+0000","last_undegraded":"2024-03-14T09:03:11.355459+0000","last_fullsized":"2024-03-14T09:03:11.355459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361988+0000","last_change":"2024-03-14T09:03:12.361988+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361988+0000","last_undegraded":"2024-03-14T09:03:12.361988+0000","last_fullsized":"2024-03-14T09:03:12.361988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362109+0000","last_change":"2024-03-14T09:03:12.362109+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362109+0000","last_undegraded":"2024-03-14T09:03:12.362109+0000","last_fullsized":"2024-03-14T09:03:12.362109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353264+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:11.353264+0000","last_peered":"2024-03-14T09:03:11.353264+0000","last_clean":"2024-03-14T09:03:11.353264+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:11.353264+0000","last_undegraded":"2024-03-14T09:03:11.353264+0000","last_fullsized":"2024-03-14T09:03:11.353264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362252+0000","last_change":"2024-03-14T09:03:12.362252+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362252+0000","last_undegraded":"2024-03-14T09:03:12.362252+0000","last_fullsized":"2024-03-14T09:03:12.362252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351526+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:11.351526+0000","last_peered":"2024-03-14T09:03:11.351526+0000","last_clean":"2024-03-14T09:03:11.351526+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:11.351526+0000","last_undegraded":"2024-03-14T09:03:11.351526+0000","last_fullsized":"2024-03-14T09:03:11.351526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353125+0000","last_change":"2024-03-14T09:03:12.353125+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353125+0000","last_undegraded":"2024-03-14T09:03:12.353125+0000","last_fullsized":"2024-03-14T09:03:12.353125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358400+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:11.358400+0000","last_peered":"2024-03-14T09:03:11.358400+0000","last_clean":"2024-03-14T09:03:11.358400+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:11.358400+0000","last_undegraded":"2024-03-14T09:03:11.358400+0000","last_fullsized":"2024-03-14T09:03:11.358400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354173+0000","last_change":"2024-03-14T09:03:12.354173+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354173+0000","last_undegraded":"2024-03-14T09:03:12.354173+0000","last_fullsized":"2024-03-14T09:03:12.354173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.352840+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:11.352840+0000","last_peered":"2024-03-14T09:03:11.352840+0000","last_clean":"2024-03-14T09:03:11.352840+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:11.352840+0000","last_undegraded":"2024-03-14T09:03:11.352840+0000","last_fullsized":"2024-03-14T09:03:11.352840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361509+0000","last_change":"2024-03-14T09:03:12.361509+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361509+0000","last_undegraded":"2024-03-14T09:03:12.361509+0000","last_fullsized":"2024-03-14T09:03:12.361509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349002+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:11.349002+0000","last_peered":"2024-03-14T09:03:11.349002+0000","last_clean":"2024-03-14T09:03:11.349002+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:11.349002+0000","last_undegraded":"2024-03-14T09:03:11.349002+0000","last_fullsized":"2024-03-14T09:03:11.349002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354070+0000","last_change":"2024-03-14T09:03:12.354070+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354070+0000","last_undegraded":"2024-03-14T09:03:12.354070+0000","last_fullsized":"2024-03-14T09:03:12.354070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376057+0000","last_change":"2024-03-14T09:03:12.371272+0000","last_active":"2024-03-14T09:03:12.376057+0000","last_peered":"2024-03-14T09:03:12.376057+0000","last_clean":"2024-03-14T09:03:12.376057+0000","last_became_active":"2024-03-14T09:03:12.370996+0000","last_became_peered":"2024-03-14T09:03:12.370996+0000","last_unstale":"2024-03-14T09:03:12.376057+0000","last_undegraded":"2024-03-14T09:03:12.376057+0000","last_fullsized":"2024-03-14T09:03:12.376057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:44.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349268+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:11.349268+0000","last_peered":"2024-03-14T09:03:11.349268+0000","last_clean":"2024-03-14T09:03:11.349268+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:11.349268+0000","last_undegraded":"2024-03-14T09:03:11.349268+0000","last_fullsized":"2024-03-14T09:03:11.349268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349423+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:11.349423+0000","last_peered":"2024-03-14T09:03:11.349423+0000","last_clean":"2024-03-14T09:03:11.349423+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:11.349423+0000","last_undegraded":"2024-03-14T09:03:11.349423+0000","last_fullsized":"2024-03-14T09:03:11.349423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356924+0000","last_change":"2024-03-14T09:03:12.356924+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356924+0000","last_undegraded":"2024-03-14T09:03:12.356924+0000","last_fullsized":"2024-03-14T09:03:12.356924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378773+0000","last_change":"2024-03-14T09:03:12.374278+0000","last_active":"2024-03-14T09:03:12.378773+0000","last_peered":"2024-03-14T09:03:12.378773+0000","last_clean":"2024-03-14T09:03:12.378773+0000","last_became_active":"2024-03-14T09:03:12.373968+0000","last_became_peered":"2024-03-14T09:03:12.373968+0000","last_unstale":"2024-03-14T09:03:12.378773+0000","last_undegraded":"2024-03-14T09:03:12.378773+0000","last_fullsized":"2024-03-14T09:03:12.378773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:40.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358207+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:11.358207+0000","last_peered":"2024-03-14T09:03:11.358207+0000","last_clean":"2024-03-14T09:03:11.358207+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:11.358207+0000","last_undegraded":"2024-03-14T09:03:11.358207+0000","last_fullsized":"2024-03-14T09:03:11.358207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353512+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:11.353512+0000","last_peered":"2024-03-14T09:03:11.353512+0000","last_clean":"2024-03-14T09:03:11.353512+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:11.353512+0000","last_undegraded":"2024-03-14T09:03:11.353512+0000","last_fullsized":"2024-03-14T09:03:11.353512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377433+0000","last_change":"2024-03-14T09:03:12.373911+0000","last_active":"2024-03-14T09:03:12.377433+0000","last_peered":"2024-03-14T09:03:12.377433+0000","last_clean":"2024-03-14T09:03:12.377433+0000","last_became_active":"2024-03-14T09:03:12.373633+0000","last_became_peered":"2024-03-14T09:03:12.373633+0000","last_unstale":"2024-03-14T09:03:12.377433+0000","last_undegraded":"2024-03-14T09:03:12.377433+0000","last_fullsized":"2024-03-14T09:03:12.377433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349440+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:11.349440+0000","last_peered":"2024-03-14T09:03:11.349440+0000","last_clean":"2024-03-14T09:03:11.349440+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:11.349440+0000","last_undegraded":"2024-03-14T09:03:11.349440+0000","last_fullsized":"2024-03-14T09:03:11.349440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359921+0000","last_change":"2024-03-14T09:03:12.359921+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359921+0000","last_undegraded":"2024-03-14T09:03:12.359921+0000","last_fullsized":"2024-03-14T09:03:12.359921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371201+0000","last_change":"2024-03-14T09:03:12.370015+0000","last_active":"2024-03-14T09:03:12.371201+0000","last_peered":"2024-03-14T09:03:12.371201+0000","last_clean":"2024-03-14T09:03:12.371201+0000","last_became_active":"2024-03-14T09:03:12.369735+0000","last_became_peered":"2024-03-14T09:03:12.369735+0000","last_unstale":"2024-03-14T09:03:12.371201+0000","last_undegraded":"2024-03-14T09:03:12.371201+0000","last_fullsized":"2024-03-14T09:03:12.371201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349286+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.349286+0000","last_peered":"2024-03-14T09:03:11.349286+0000","last_clean":"2024-03-14T09:03:11.349286+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:11.349286+0000","last_undegraded":"2024-03-14T09:03:11.349286+0000","last_fullsized":"2024-03-14T09:03:11.349286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355554+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:11.355554+0000","last_peered":"2024-03-14T09:03:11.355554+0000","last_clean":"2024-03-14T09:03:11.355554+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:11.355554+0000","last_undegraded":"2024-03-14T09:03:11.355554+0000","last_fullsized":"2024-03-14T09:03:11.355554+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360660+0000","last_change":"2024-03-14T09:03:12.360660+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360660+0000","last_undegraded":"2024-03-14T09:03:12.360660+0000","last_fullsized":"2024-03-14T09:03:12.360660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375005+0000","last_change":"2024-03-14T09:03:12.372188+0000","last_active":"2024-03-14T09:03:12.375005+0000","last_peered":"2024-03-14T09:03:12.375005+0000","last_clean":"2024-03-14T09:03:12.375005+0000","last_became_active":"2024-03-14T09:03:12.371919+0000","last_became_peered":"2024-03-14T09:03:12.371919+0000","last_unstale":"2024-03-14T09:03:12.375005+0000","last_undegraded":"2024-03-14T09:03:12.375005+0000","last_fullsized":"2024-03-14T09:03:12.375005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351321+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:11.351321+0000","last_peered":"2024-03-14T09:03:11.351321+0000","last_clean":"2024-03-14T09:03:11.351321+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:11.351321+0000","last_undegraded":"2024-03-14T09:03:11.351321+0000","last_fullsized":"2024-03-14T09:03:11.351321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355947+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:11.355947+0000","last_peered":"2024-03-14T09:03:11.355947+0000","last_clean":"2024-03-14T09:03:11.355947+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:11.355947+0000","last_undegraded":"2024-03-14T09:03:11.355947+0000","last_fullsized":"2024-03-14T09:03:11.355947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355339+0000","last_change":"2024-03-14T09:03:12.355339+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355339+0000","last_undegraded":"2024-03-14T09:03:12.355339+0000","last_fullsized":"2024-03-14T09:03:12.355339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372079+0000","last_change":"2024-03-14T09:03:12.369918+0000","last_active":"2024-03-14T09:03:12.372079+0000","last_peered":"2024-03-14T09:03:12.372079+0000","last_clean":"2024-03-14T09:03:12.372079+0000","last_became_active":"2024-03-14T09:03:12.369599+0000","last_became_peered":"2024-03-14T09:03:12.369599+0000","last_unstale":"2024-03-14T09:03:12.372079+0000","last_undegraded":"2024-03-14T09:03:12.372079+0000","last_fullsized":"2024-03-14T09:03:12.372079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:52.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351535+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:11.351535+0000","last_peered":"2024-03-14T09:03:11.351535+0000","last_clean":"2024-03-14T09:03:11.351535+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:11.351535+0000","last_undegraded":"2024-03-14T09:03:11.351535+0000","last_fullsized":"2024-03-14T09:03:11.351535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349367+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:11.349367+0000","last_peered":"2024-03-14T09:03:11.349367+0000","last_clean":"2024-03-14T09:03:11.349367+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:11.349367+0000","last_undegraded":"2024-03-14T09:03:11.349367+0000","last_fullsized":"2024-03-14T09:03:11.349367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361025+0000","last_change":"2024-03-14T09:03:12.361025+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361025+0000","last_undegraded":"2024-03-14T09:03:12.361025+0000","last_fullsized":"2024-03-14T09:03:12.361025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374895+0000","last_change":"2024-03-14T09:03:12.373841+0000","last_active":"2024-03-14T09:03:12.374895+0000","last_peered":"2024-03-14T09:03:12.374895+0000","last_clean":"2024-03-14T09:03:12.374895+0000","last_became_active":"2024-03-14T09:03:12.373551+0000","last_became_peered":"2024-03-14T09:03:12.373551+0000","last_unstale":"2024-03-14T09:03:12.374895+0000","last_undegraded":"2024-03-14T09:03:12.374895+0000","last_fullsized":"2024-03-14T09:03:12.374895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.965540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353268+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:11.353268+0000","last_peered":"2024-03-14T09:03:11.353268+0000","last_clean":"2024-03-14T09:03:11.353268+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:11.353268+0000","last_undegraded":"2024-03-14T09:03:11.353268+0000","last_fullsized":"2024-03-14T09:03:11.353268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355710+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:11.355710+0000","last_peered":"2024-03-14T09:03:11.355710+0000","last_clean":"2024-03-14T09:03:11.355710+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:11.355710+0000","last_undegraded":"2024-03-14T09:03:11.355710+0000","last_fullsized":"2024-03-14T09:03:11.355710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361000+0000","last_change":"2024-03-14T09:03:12.361000+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361000+0000","last_undegraded":"2024-03-14T09:03:12.361000+0000","last_fullsized":"2024-03-14T09:03:12.361000+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368944+0000","last_change":"2024-03-14T09:03:12.366900+0000","last_active":"2024-03-14T09:03:12.368944+0000","last_peered":"2024-03-14T09:03:12.368944+0000","last_clean":"2024-03-14T09:03:12.368944+0000","last_became_active":"2024-03-14T09:03:12.366301+0000","last_became_peered":"2024-03-14T09:03:12.366301+0000","last_unstale":"2024-03-14T09:03:12.368944+0000","last_undegraded":"2024-03-14T09:03:12.368944+0000","last_fullsized":"2024-03-14T09:03:12.368944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:11.884116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354744+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:11.354744+0000","last_peered":"2024-03-14T09:03:11.354744+0000","last_clean":"2024-03-14T09:03:11.354744+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:11.354744+0000","last_undegraded":"2024-03-14T09:03:11.354744+0000","last_fullsized":"2024-03-14T09:03:11.354744+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353503+0000","last_change":"2024-03-14T09:03:12.353503+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353503+0000","last_undegraded":"2024-03-14T09:03:12.353503+0000","last_fullsized":"2024-03-14T09:03:12.353503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371091+0000","last_change":"2024-03-14T09:03:12.367305+0000","last_active":"2024-03-14T09:03:12.371091+0000","last_peered":"2024-03-14T09:03:12.371091+0000","last_clean":"2024-03-14T09:03:12.371091+0000","last_became_active":"2024-03-14T09:03:12.367020+0000","last_became_peered":"2024-03-14T09:03:12.367020+0000","last_unstale":"2024-03-14T09:03:12.371091+0000","last_undegraded":"2024-03-14T09:03:12.371091+0000","last_fullsized":"2024-03-14T09:03:12.371091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:39.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356736+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:11.356736+0000","last_peered":"2024-03-14T09:03:11.356736+0000","last_clean":"2024-03-14T09:03:11.356736+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:11.356736+0000","last_undegraded":"2024-03-14T09:03:11.356736+0000","last_fullsized":"2024-03-14T09:03:11.356736+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358626+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:11.358626+0000","last_peered":"2024-03-14T09:03:11.358626+0000","last_clean":"2024-03-14T09:03:11.358626+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:11.358626+0000","last_undegraded":"2024-03-14T09:03:11.358626+0000","last_fullsized":"2024-03-14T09:03:11.358626+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364515+0000","last_change":"2024-03-14T09:03:12.364515+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364515+0000","last_undegraded":"2024-03-14T09:03:12.364515+0000","last_fullsized":"2024-03-14T09:03:12.364515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373487+0000","last_change":"2024-03-14T09:03:12.370627+0000","last_active":"2024-03-14T09:03:12.373487+0000","last_peered":"2024-03-14T09:03:12.373487+0000","last_clean":"2024-03-14T09:03:12.373487+0000","last_became_active":"2024-03-14T09:03:12.370314+0000","last_became_peered":"2024-03-14T09:03:12.370314+0000","last_unstale":"2024-03-14T09:03:12.373487+0000","last_undegraded":"2024-03-14T09:03:12.373487+0000","last_fullsized":"2024-03-14T09:03:12.373487+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:09.613459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358558+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:11.358558+0000","last_peered":"2024-03-14T09:03:11.358558+0000","last_clean":"2024-03-14T09:03:11.358558+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:11.358558+0000","last_undegraded":"2024-03-14T09:03:11.358558+0000","last_fullsized":"2024-03-14T09:03:11.358558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353599+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:11.353599+0000","last_peered":"2024-03-14T09:03:11.353599+0000","last_clean":"2024-03-14T09:03:11.353599+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:11.353599+0000","last_undegraded":"2024-03-14T09:03:11.353599+0000","last_fullsized":"2024-03-14T09:03:11.353599+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359668+0000","last_change":"2024-03-14T09:03:12.359668+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359668+0000","last_undegraded":"2024-03-14T09:03:12.359668+0000","last_fullsized":"2024-03-14T09:03:12.359668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378802+0000","last_change":"2024-03-14T09:03:12.374028+0000","last_active":"2024-03-14T09:03:12.378802+0000","last_peered":"2024-03-14T09:03:12.378802+0000","last_clean":"2024-03-14T09:03:12.378802+0000","last_became_active":"2024-03-14T09:03:12.373770+0000","last_became_peered":"2024-03-14T09:03:12.373770+0000","last_unstale":"2024-03-14T09:03:12.378802+0000","last_undegraded":"2024-03-14T09:03:12.378802+0000","last_fullsized":"2024-03-14T09:03:12.378802+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:51.114973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357121+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:11.357121+0000","last_peered":"2024-03-14T09:03:11.357121+0000","last_clean":"2024-03-14T09:03:11.357121+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:11.357121+0000","last_undegraded":"2024-03-14T09:03:11.357121+0000","last_fullsized":"2024-03-14T09:03:11.357121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358587+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:11.358587+0000","last_peered":"2024-03-14T09:03:11.358587+0000","last_clean":"2024-03-14T09:03:11.358587+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:11.358587+0000","last_undegraded":"2024-03-14T09:03:11.358587+0000","last_fullsized":"2024-03-14T09:03:11.358587+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357946+0000","last_change":"2024-03-14T09:03:12.357946+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357946+0000","last_undegraded":"2024-03-14T09:03:12.357946+0000","last_fullsized":"2024-03-14T09:03:12.357946+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368918+0000","last_change":"2024-03-14T09:03:12.367742+0000","last_active":"2024-03-14T09:03:12.368918+0000","last_peered":"2024-03-14T09:03:12.368918+0000","last_clean":"2024-03-14T09:03:12.368918+0000","last_became_active":"2024-03-14T09:03:12.367471+0000","last_became_peered":"2024-03-14T09:03:12.367471+0000","last_unstale":"2024-03-14T09:03:12.368918+0000","last_undegraded":"2024-03-14T09:03:12.368918+0000","last_fullsized":"2024-03-14T09:03:12.368918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:34.678842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358346+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:11.358346+0000","last_peered":"2024-03-14T09:03:11.358346+0000","last_clean":"2024-03-14T09:03:11.358346+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:11.358346+0000","last_undegraded":"2024-03-14T09:03:11.358346+0000","last_fullsized":"2024-03-14T09:03:11.358346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356446+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:11.356446+0000","last_peered":"2024-03-14T09:03:11.356446+0000","last_clean":"2024-03-14T09:03:11.356446+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:11.356446+0000","last_undegraded":"2024-03-14T09:03:11.356446+0000","last_fullsized":"2024-03-14T09:03:11.356446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355535+0000","last_change":"2024-03-14T09:03:12.355535+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355535+0000","last_undegraded":"2024-03-14T09:03:12.355535+0000","last_fullsized":"2024-03-14T09:03:12.355535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371980+0000","last_change":"2024-03-14T09:03:12.366980+0000","last_active":"2024-03-14T09:03:12.371980+0000","last_peered":"2024-03-14T09:03:12.371980+0000","last_clean":"2024-03-14T09:03:12.371980+0000","last_became_active":"2024-03-14T09:03:12.366728+0000","last_became_peered":"2024-03-14T09:03:12.366728+0000","last_unstale":"2024-03-14T09:03:12.371980+0000","last_undegraded":"2024-03-14T09:03:12.371980+0000","last_fullsized":"2024-03-14T09:03:12.371980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:53.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355135+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:11.355135+0000","last_peered":"2024-03-14T09:03:11.355135+0000","last_clean":"2024-03-14T09:03:11.355135+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:11.355135+0000","last_undegraded":"2024-03-14T09:03:11.355135+0000","last_fullsized":"2024-03-14T09:03:11.355135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351630+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:11.351630+0000","last_peered":"2024-03-14T09:03:11.351630+0000","last_clean":"2024-03-14T09:03:11.351630+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:11.351630+0000","last_undegraded":"2024-03-14T09:03:11.351630+0000","last_fullsized":"2024-03-14T09:03:11.351630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358690+0000","last_change":"2024-03-14T09:03:12.358690+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358690+0000","last_undegraded":"2024-03-14T09:03:12.358690+0000","last_fullsized":"2024-03-14T09:03:12.358690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376094+0000","last_change":"2024-03-14T09:03:12.371369+0000","last_active":"2024-03-14T09:03:12.376094+0000","last_peered":"2024-03-14T09:03:12.376094+0000","last_clean":"2024-03-14T09:03:12.376094+0000","last_became_active":"2024-03-14T09:03:12.371087+0000","last_became_peered":"2024-03-14T09:03:12.371087+0000","last_unstale":"2024-03-14T09:03:12.376094+0000","last_undegraded":"2024-03-14T09:03:12.376094+0000","last_fullsized":"2024-03-14T09:03:12.376094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:38.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358447+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:11.358447+0000","last_peered":"2024-03-14T09:03:11.358447+0000","last_clean":"2024-03-14T09:03:11.358447+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:11.358447+0000","last_undegraded":"2024-03-14T09:03:11.358447+0000","last_fullsized":"2024-03-14T09:03:11.358447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354648+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:11.354648+0000","last_peered":"2024-03-14T09:03:11.354648+0000","last_clean":"2024-03-14T09:03:11.354648+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:11.354648+0000","last_undegraded":"2024-03-14T09:03:11.354648+0000","last_fullsized":"2024-03-14T09:03:11.354648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356321+0000","last_change":"2024-03-14T09:03:12.356321+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356321+0000","last_undegraded":"2024-03-14T09:03:12.356321+0000","last_fullsized":"2024-03-14T09:03:12.356321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373386+0000","last_change":"2024-03-14T09:03:12.368944+0000","last_active":"2024-03-14T09:03:12.373386+0000","last_peered":"2024-03-14T09:03:12.373386+0000","last_clean":"2024-03-14T09:03:12.373386+0000","last_became_active":"2024-03-14T09:03:12.368631+0000","last_became_peered":"2024-03-14T09:03:12.368631+0000","last_unstale":"2024-03-14T09:03:12.373386+0000","last_undegraded":"2024-03-14T09:03:12.373386+0000","last_fullsized":"2024-03-14T09:03:12.373386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354667+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:11.354667+0000","last_peered":"2024-03-14T09:03:11.354667+0000","last_clean":"2024-03-14T09:03:11.354667+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:11.354667+0000","last_undegraded":"2024-03-14T09:03:11.354667+0000","last_fullsized":"2024-03-14T09:03:11.354667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355806+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:11.355806+0000","last_peered":"2024-03-14T09:03:11.355806+0000","last_clean":"2024-03-14T09:03:11.355806+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:11.355806+0000","last_undegraded":"2024-03-14T09:03:11.355806+0000","last_fullsized":"2024-03-14T09:03:11.355806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360063+0000","last_change":"2024-03-14T09:03:12.360063+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360063+0000","last_undegraded":"2024-03-14T09:03:12.360063+0000","last_fullsized":"2024-03-14T09:03:12.360063+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371484+0000","last_change":"2024-03-14T09:03:12.370316+0000","last_active":"2024-03-14T09:03:12.371484+0000","last_peered":"2024-03-14T09:03:12.371484+0000","last_clean":"2024-03-14T09:03:12.371484+0000","last_became_active":"2024-03-14T09:03:12.370035+0000","last_became_peered":"2024-03-14T09:03:12.370035+0000","last_unstale":"2024-03-14T09:03:12.371484+0000","last_undegraded":"2024-03-14T09:03:12.371484+0000","last_fullsized":"2024-03-14T09:03:12.371484+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:07.312244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358188+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:11.358188+0000","last_peered":"2024-03-14T09:03:11.358188+0000","last_clean":"2024-03-14T09:03:11.358188+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:11.358188+0000","last_undegraded":"2024-03-14T09:03:11.358188+0000","last_fullsized":"2024-03-14T09:03:11.358188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358017+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:11.358017+0000","last_peered":"2024-03-14T09:03:11.358017+0000","last_clean":"2024-03-14T09:03:11.358017+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:11.358017+0000","last_undegraded":"2024-03-14T09:03:11.358017+0000","last_fullsized":"2024-03-14T09:03:11.358017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357489+0000","last_change":"2024-03-14T09:03:12.357489+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357489+0000","last_undegraded":"2024-03-14T09:03:12.357489+0000","last_fullsized":"2024-03-14T09:03:12.357489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377230+0000","last_change":"2024-03-14T09:03:12.373196+0000","last_active":"2024-03-14T09:03:12.377230+0000","last_peered":"2024-03-14T09:03:12.377230+0000","last_clean":"2024-03-14T09:03:12.377230+0000","last_became_active":"2024-03-14T09:03:12.372917+0000","last_became_peered":"2024-03-14T09:03:12.372917+0000","last_unstale":"2024-03-14T09:03:12.377230+0000","last_undegraded":"2024-03-14T09:03:12.377230+0000","last_fullsized":"2024-03-14T09:03:12.377230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:56.766593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358565+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:11.358565+0000","last_peered":"2024-03-14T09:03:11.358565+0000","last_clean":"2024-03-14T09:03:11.358565+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:11.358565+0000","last_undegraded":"2024-03-14T09:03:11.358565+0000","last_fullsized":"2024-03-14T09:03:11.358565+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353682+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:11.353682+0000","last_peered":"2024-03-14T09:03:11.353682+0000","last_clean":"2024-03-14T09:03:11.353682+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:11.353682+0000","last_undegraded":"2024-03-14T09:03:11.353682+0000","last_fullsized":"2024-03-14T09:03:11.353682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359244+0000","last_change":"2024-03-14T09:03:12.359244+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359244+0000","last_undegraded":"2024-03-14T09:03:12.359244+0000","last_fullsized":"2024-03-14T09:03:12.359244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377421+0000","last_change":"2024-03-14T09:03:12.373227+0000","last_active":"2024-03-14T09:03:12.377421+0000","last_peered":"2024-03-14T09:03:12.377421+0000","last_clean":"2024-03-14T09:03:12.377421+0000","last_became_active":"2024-03-14T09:03:12.372761+0000","last_became_peered":"2024-03-14T09:03:12.372761+0000","last_unstale":"2024-03-14T09:03:12.377421+0000","last_undegraded":"2024-03-14T09:03:12.377421+0000","last_fullsized":"2024-03-14T09:03:12.377421+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355667+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:11.355667+0000","last_peered":"2024-03-14T09:03:11.355667+0000","last_clean":"2024-03-14T09:03:11.355667+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:11.355667+0000","last_undegraded":"2024-03-14T09:03:11.355667+0000","last_fullsized":"2024-03-14T09:03:11.355667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349366+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:11.349366+0000","last_peered":"2024-03-14T09:03:11.349366+0000","last_clean":"2024-03-14T09:03:11.349366+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:11.349366+0000","last_undegraded":"2024-03-14T09:03:11.349366+0000","last_fullsized":"2024-03-14T09:03:11.349366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364238+0000","last_change":"2024-03-14T09:03:12.364238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364238+0000","last_undegraded":"2024-03-14T09:03:12.364238+0000","last_fullsized":"2024-03-14T09:03:12.364238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367869+0000","last_change":"2024-03-14T09:03:12.365684+0000","last_active":"2024-03-14T09:03:12.367869+0000","last_peered":"2024-03-14T09:03:12.367869+0000","last_clean":"2024-03-14T09:03:12.367869+0000","last_became_active":"2024-03-14T09:03:12.365263+0000","last_became_peered":"2024-03-14T09:03:12.365263+0000","last_unstale":"2024-03-14T09:03:12.367869+0000","last_undegraded":"2024-03-14T09:03:12.367869+0000","last_fullsized":"2024-03-14T09:03:12.367869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351721+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:11.351721+0000","last_peered":"2024-03-14T09:03:11.351721+0000","last_clean":"2024-03-14T09:03:11.351721+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:11.351721+0000","last_undegraded":"2024-03-14T09:03:11.351721+0000","last_fullsized":"2024-03-14T09:03:11.351721+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355737+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:11.355737+0000","last_peered":"2024-03-14T09:03:11.355737+0000","last_clean":"2024-03-14T09:03:11.355737+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:11.355737+0000","last_undegraded":"2024-03-14T09:03:11.355737+0000","last_fullsized":"2024-03-14T09:03:11.355737+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357463+0000","last_change":"2024-03-14T09:03:12.357463+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357463+0000","last_undegraded":"2024-03-14T09:03:12.357463+0000","last_fullsized":"2024-03-14T09:03:12.357463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376072+0000","last_change":"2024-03-14T09:03:12.371367+0000","last_active":"2024-03-14T09:03:12.376072+0000","last_peered":"2024-03-14T09:03:12.376072+0000","last_clean":"2024-03-14T09:03:12.376072+0000","last_became_active":"2024-03-14T09:03:12.371044+0000","last_became_peered":"2024-03-14T09:03:12.371044+0000","last_unstale":"2024-03-14T09:03:12.376072+0000","last_undegraded":"2024-03-14T09:03:12.376072+0000","last_fullsized":"2024-03-14T09:03:12.376072+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:20:22.371301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358462+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:11.358462+0000","last_peered":"2024-03-14T09:03:11.358462+0000","last_clean":"2024-03-14T09:03:11.358462+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:11.358462+0000","last_undegraded":"2024-03-14T09:03:11.358462+0000","last_fullsized":"2024-03-14T09:03:11.358462+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349280+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:11.349280+0000","last_peered":"2024-03-14T09:03:11.349280+0000","last_clean":"2024-03-14T09:03:11.349280+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:11.349280+0000","last_undegraded":"2024-03-14T09:03:11.349280+0000","last_fullsized":"2024-03-14T09:03:11.349280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352704+0000","last_change":"2024-03-14T09:03:12.352704+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352704+0000","last_undegraded":"2024-03-14T09:03:12.352704+0000","last_fullsized":"2024-03-14T09:03:12.352704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374596+0000","last_change":"2024-03-14T09:03:12.372117+0000","last_active":"2024-03-14T09:03:12.374596+0000","last_peered":"2024-03-14T09:03:12.374596+0000","last_clean":"2024-03-14T09:03:12.374596+0000","last_became_active":"2024-03-14T09:03:12.371774+0000","last_became_peered":"2024-03-14T09:03:12.371774+0000","last_unstale":"2024-03-14T09:03:12.374596+0000","last_undegraded":"2024-03-14T09:03:12.374596+0000","last_fullsized":"2024-03-14T09:03:12.374596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:04.409001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355185+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:11.355185+0000","last_peered":"2024-03-14T09:03:11.355185+0000","last_clean":"2024-03-14T09:03:11.355185+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:11.355185+0000","last_undegraded":"2024-03-14T09:03:11.355185+0000","last_fullsized":"2024-03-14T09:03:11.355185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351724+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:11.351724+0000","last_peered":"2024-03-14T09:03:11.351724+0000","last_clean":"2024-03-14T09:03:11.351724+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:11.351724+0000","last_undegraded":"2024-03-14T09:03:11.351724+0000","last_fullsized":"2024-03-14T09:03:11.351724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354163+0000","last_change":"2024-03-14T09:03:12.354163+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354163+0000","last_undegraded":"2024-03-14T09:03:12.354163+0000","last_fullsized":"2024-03-14T09:03:12.354163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372069+0000","last_change":"2024-03-14T09:03:12.370993+0000","last_active":"2024-03-14T09:03:12.372069+0000","last_peered":"2024-03-14T09:03:12.372069+0000","last_clean":"2024-03-14T09:03:12.372069+0000","last_became_active":"2024-03-14T09:03:12.370745+0000","last_became_peered":"2024-03-14T09:03:12.370745+0000","last_unstale":"2024-03-14T09:03:12.372069+0000","last_undegraded":"2024-03-14T09:03:12.372069+0000","last_fullsized":"2024-03-14T09:03:12.372069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349338+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:11.349338+0000","last_peered":"2024-03-14T09:03:11.349338+0000","last_clean":"2024-03-14T09:03:11.349338+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:11.349338+0000","last_undegraded":"2024-03-14T09:03:11.349338+0000","last_fullsized":"2024-03-14T09:03:11.349338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356757+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:11.356757+0000","last_peered":"2024-03-14T09:03:11.356757+0000","last_clean":"2024-03-14T09:03:11.356757+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:11.356757+0000","last_undegraded":"2024-03-14T09:03:11.356757+0000","last_fullsized":"2024-03-14T09:03:11.356757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353676+0000","last_change":"2024-03-14T09:03:12.353676+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353676+0000","last_undegraded":"2024-03-14T09:03:12.353676+0000","last_fullsized":"2024-03-14T09:03:12.353676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368922+0000","last_change":"2024-03-14T09:03:12.366770+0000","last_active":"2024-03-14T09:03:12.368922+0000","last_peered":"2024-03-14T09:03:12.368922+0000","last_clean":"2024-03-14T09:03:12.368922+0000","last_became_active":"2024-03-14T09:03:12.366022+0000","last_became_peered":"2024-03-14T09:03:12.366022+0000","last_unstale":"2024-03-14T09:03:12.368922+0000","last_undegraded":"2024-03-14T09:03:12.368922+0000","last_fullsized":"2024-03-14T09:03:12.368922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349099+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:11.349099+0000","last_peered":"2024-03-14T09:03:11.349099+0000","last_clean":"2024-03-14T09:03:11.349099+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:11.349099+0000","last_undegraded":"2024-03-14T09:03:11.349099+0000","last_fullsized":"2024-03-14T09:03:11.349099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354967+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:11.354967+0000","last_peered":"2024-03-14T09:03:11.354967+0000","last_clean":"2024-03-14T09:03:11.354967+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:11.354967+0000","last_undegraded":"2024-03-14T09:03:11.354967+0000","last_fullsized":"2024-03-14T09:03:11.354967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356783+0000","last_change":"2024-03-14T09:03:12.356783+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356783+0000","last_undegraded":"2024-03-14T09:03:12.356783+0000","last_fullsized":"2024-03-14T09:03:12.356783+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375064+0000","last_change":"2024-03-14T09:03:12.373601+0000","last_active":"2024-03-14T09:03:12.375064+0000","last_peered":"2024-03-14T09:03:12.375064+0000","last_clean":"2024-03-14T09:03:12.375064+0000","last_became_active":"2024-03-14T09:03:12.373332+0000","last_became_peered":"2024-03-14T09:03:12.373332+0000","last_unstale":"2024-03-14T09:03:12.375064+0000","last_undegraded":"2024-03-14T09:03:12.375064+0000","last_fullsized":"2024-03-14T09:03:12.375064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:38:05.459173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355546+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:11.355546+0000","last_peered":"2024-03-14T09:03:11.355546+0000","last_clean":"2024-03-14T09:03:11.355546+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:11.355546+0000","last_undegraded":"2024-03-14T09:03:11.355546+0000","last_fullsized":"2024-03-14T09:03:11.355546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358290+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:11.358290+0000","last_peered":"2024-03-14T09:03:11.358290+0000","last_clean":"2024-03-14T09:03:11.358290+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:11.358290+0000","last_undegraded":"2024-03-14T09:03:11.358290+0000","last_fullsized":"2024-03-14T09:03:11.358290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357797+0000","last_change":"2024-03-14T09:03:12.357797+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357797+0000","last_undegraded":"2024-03-14T09:03:12.357797+0000","last_fullsized":"2024-03-14T09:03:12.357797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375031+0000","last_change":"2024-03-14T09:03:12.373610+0000","last_active":"2024-03-14T09:03:12.375031+0000","last_peered":"2024-03-14T09:03:12.375031+0000","last_clean":"2024-03-14T09:03:12.375031+0000","last_became_active":"2024-03-14T09:03:12.373360+0000","last_became_peered":"2024-03-14T09:03:12.373360+0000","last_unstale":"2024-03-14T09:03:12.375031+0000","last_undegraded":"2024-03-14T09:03:12.375031+0000","last_fullsized":"2024-03-14T09:03:12.375031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:32.334273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355435+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:11.355435+0000","last_peered":"2024-03-14T09:03:11.355435+0000","last_clean":"2024-03-14T09:03:11.355435+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:11.355435+0000","last_undegraded":"2024-03-14T09:03:11.355435+0000","last_fullsized":"2024-03-14T09:03:11.355435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"51'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349459+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:11.349459+0000","last_peered":"2024-03-14T09:03:11.349459+0000","last_clean":"2024-03-14T09:03:11.349459+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:11.349459+0000","last_undegraded":"2024-03-14T09:03:11.349459+0000","last_fullsized":"2024-03-14T09:03:11.349459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356751+0000","last_change":"2024-03-14T09:03:12.356751+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356751+0000","last_undegraded":"2024-03-14T09:03:12.356751+0000","last_fullsized":"2024-03-14T09:03:12.356751+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367852+0000","last_change":"2024-03-14T09:03:12.365515+0000","last_active":"2024-03-14T09:03:12.367852+0000","last_peered":"2024-03-14T09:03:12.367852+0000","last_clean":"2024-03-14T09:03:12.367852+0000","last_became_active":"2024-03-14T09:03:12.364921+0000","last_became_peered":"2024-03-14T09:03:12.364921+0000","last_unstale":"2024-03-14T09:03:12.367852+0000","last_undegraded":"2024-03-14T09:03:12.367852+0000","last_fullsized":"2024-03-14T09:03:12.367852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357128+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:11.357128+0000","last_peered":"2024-03-14T09:03:11.357128+0000","last_clean":"2024-03-14T09:03:11.357128+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:11.357128+0000","last_undegraded":"2024-03-14T09:03:11.357128+0000","last_fullsized":"2024-03-14T09:03:11.357128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355195+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:11.355195+0000","last_peered":"2024-03-14T09:03:11.355195+0000","last_clean":"2024-03-14T09:03:11.355195+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:11.355195+0000","last_undegraded":"2024-03-14T09:03:11.355195+0000","last_fullsized":"2024-03-14T09:03:11.355195+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361654+0000","last_change":"2024-03-14T09:03:12.361654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361654+0000","last_undegraded":"2024-03-14T09:03:12.361654+0000","last_fullsized":"2024-03-14T09:03:12.361654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371874+0000","last_change":"2024-03-14T09:03:12.367975+0000","last_active":"2024-03-14T09:03:12.371874+0000","last_peered":"2024-03-14T09:03:12.371874+0000","last_clean":"2024-03-14T09:03:12.371874+0000","last_became_active":"2024-03-14T09:03:12.367702+0000","last_became_peered":"2024-03-14T09:03:12.367702+0000","last_unstale":"2024-03-14T09:03:12.371874+0000","last_undegraded":"2024-03-14T09:03:12.371874+0000","last_fullsized":"2024-03-14T09:03:12.371874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:12.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349479+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:11.349479+0000","last_peered":"2024-03-14T09:03:11.349479+0000","last_clean":"2024-03-14T09:03:11.349479+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:11.349479+0000","last_undegraded":"2024-03-14T09:03:11.349479+0000","last_fullsized":"2024-03-14T09:03:11.349479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355796+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:11.355796+0000","last_peered":"2024-03-14T09:03:11.355796+0000","last_clean":"2024-03-14T09:03:11.355796+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:11.355796+0000","last_undegraded":"2024-03-14T09:03:11.355796+0000","last_fullsized":"2024-03-14T09:03:11.355796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358131+0000","last_change":"2024-03-14T09:03:12.358131+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358131+0000","last_undegraded":"2024-03-14T09:03:12.358131+0000","last_fullsized":"2024-03-14T09:03:12.358131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376099+0000","last_change":"2024-03-14T09:03:12.371984+0000","last_active":"2024-03-14T09:03:12.376099+0000","last_peered":"2024-03-14T09:03:12.376099+0000","last_clean":"2024-03-14T09:03:12.376099+0000","last_became_active":"2024-03-14T09:03:12.371727+0000","last_became_peered":"2024-03-14T09:03:12.371727+0000","last_unstale":"2024-03-14T09:03:12.376099+0000","last_undegraded":"2024-03-14T09:03:12.376099+0000","last_fullsized":"2024-03-14T09:03:12.376099+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:23.845822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351395+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:11.351395+0000","last_peered":"2024-03-14T09:03:11.351395+0000","last_clean":"2024-03-14T09:03:11.351395+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:11.351395+0000","last_undegraded":"2024-03-14T09:03:11.351395+0000","last_fullsized":"2024-03-14T09:03:11.351395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356875+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:11.356875+0000","last_peered":"2024-03-14T09:03:11.356875+0000","last_clean":"2024-03-14T09:03:11.356875+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:11.356875+0000","last_undegraded":"2024-03-14T09:03:11.356875+0000","last_fullsized":"2024-03-14T09:03:11.356875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352756+0000","last_change":"2024-03-14T09:03:12.352756+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352756+0000","last_undegraded":"2024-03-14T09:03:12.352756+0000","last_fullsized":"2024-03-14T09:03:12.352756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.366806+0000","last_change":"2024-03-14T09:03:12.363729+0000","last_active":"2024-03-14T09:03:12.366806+0000","last_peered":"2024-03-14T09:03:12.366806+0000","last_clean":"2024-03-14T09:03:12.366806+0000","last_became_active":"2024-03-14T09:03:12.363440+0000","last_became_peered":"2024-03-14T09:03:12.363440+0000","last_unstale":"2024-03-14T09:03:12.366806+0000","last_undegraded":"2024-03-14T09:03:12.366806+0000","last_fullsized":"2024-03-14T09:03:12.366806+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:13.058703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358367+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:11.358367+0000","last_peered":"2024-03-14T09:03:11.358367+0000","last_clean":"2024-03-14T09:03:11.358367+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:11.358367+0000","last_undegraded":"2024-03-14T09:03:11.358367+0000","last_fullsized":"2024-03-14T09:03:11.358367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355086+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:11.355086+0000","last_peered":"2024-03-14T09:03:11.355086+0000","last_clean":"2024-03-14T09:03:11.355086+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:11.355086+0000","last_undegraded":"2024-03-14T09:03:11.355086+0000","last_fullsized":"2024-03-14T09:03:11.355086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359840+0000","last_change":"2024-03-14T09:03:12.359840+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359840+0000","last_undegraded":"2024-03-14T09:03:12.359840+0000","last_fullsized":"2024-03-14T09:03:12.359840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372010+0000","last_change":"2024-03-14T09:03:12.370434+0000","last_active":"2024-03-14T09:03:12.372010+0000","last_peered":"2024-03-14T09:03:12.372010+0000","last_clean":"2024-03-14T09:03:12.372010+0000","last_became_active":"2024-03-14T09:03:12.370170+0000","last_became_peered":"2024-03-14T09:03:12.370170+0000","last_unstale":"2024-03-14T09:03:12.372010+0000","last_undegraded":"2024-03-14T09:03:12.372010+0000","last_fullsized":"2024-03-14T09:03:12.372010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:55.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356464+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:11.356464+0000","last_peered":"2024-03-14T09:03:11.356464+0000","last_clean":"2024-03-14T09:03:11.356464+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:11.356464+0000","last_undegraded":"2024-03-14T09:03:11.356464+0000","last_fullsized":"2024-03-14T09:03:11.356464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356577+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:11.356577+0000","last_peered":"2024-03-14T09:03:11.356577+0000","last_clean":"2024-03-14T09:03:11.356577+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:11.356577+0000","last_undegraded":"2024-03-14T09:03:11.356577+0000","last_fullsized":"2024-03-14T09:03:11.356577+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353449+0000","last_change":"2024-03-14T09:03:12.353449+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353449+0000","last_undegraded":"2024-03-14T09:03:12.353449+0000","last_fullsized":"2024-03-14T09:03:12.353449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375525+0000","last_change":"2024-03-14T09:03:12.373132+0000","last_active":"2024-03-14T09:03:12.375525+0000","last_peered":"2024-03-14T09:03:12.375525+0000","last_clean":"2024-03-14T09:03:12.375525+0000","last_became_active":"2024-03-14T09:03:12.372802+0000","last_became_peered":"2024-03-14T09:03:12.372802+0000","last_unstale":"2024-03-14T09:03:12.375525+0000","last_undegraded":"2024-03-14T09:03:12.375525+0000","last_fullsized":"2024-03-14T09:03:12.375525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351200+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:11.351200+0000","last_peered":"2024-03-14T09:03:11.351200+0000","last_clean":"2024-03-14T09:03:11.351200+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:11.351200+0000","last_undegraded":"2024-03-14T09:03:11.351200+0000","last_fullsized":"2024-03-14T09:03:11.351200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351201+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:11.351201+0000","last_peered":"2024-03-14T09:03:11.351201+0000","last_clean":"2024-03-14T09:03:11.351201+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:11.351201+0000","last_undegraded":"2024-03-14T09:03:11.351201+0000","last_fullsized":"2024-03-14T09:03:11.351201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353238+0000","last_change":"2024-03-14T09:03:12.353238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353238+0000","last_undegraded":"2024-03-14T09:03:12.353238+0000","last_fullsized":"2024-03-14T09:03:12.353238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367162+0000","last_change":"2024-03-14T09:03:12.364039+0000","last_active":"2024-03-14T09:03:12.367162+0000","last_peered":"2024-03-14T09:03:12.367162+0000","last_clean":"2024-03-14T09:03:12.367162+0000","last_became_active":"2024-03-14T09:03:12.363778+0000","last_became_peered":"2024-03-14T09:03:12.363778+0000","last_unstale":"2024-03-14T09:03:12.367162+0000","last_undegraded":"2024-03-14T09:03:12.367162+0000","last_fullsized":"2024-03-14T09:03:12.367162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:53.168005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353295+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:11.353295+0000","last_peered":"2024-03-14T09:03:11.353295+0000","last_clean":"2024-03-14T09:03:11.353295+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:11.353295+0000","last_undegraded":"2024-03-14T09:03:11.353295+0000","last_fullsized":"2024-03-14T09:03:11.353295+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358124+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:11.358124+0000","last_peered":"2024-03-14T09:03:11.358124+0000","last_clean":"2024-03-14T09:03:11.358124+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:11.358124+0000","last_undegraded":"2024-03-14T09:03:11.358124+0000","last_fullsized":"2024-03-14T09:03:11.358124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1236992,"data_stored":861723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.623}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":271195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":264155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.37}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.318}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.621}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":858090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.372},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.316}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.44}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.504},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":271251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.254},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.435}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":274270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.625},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:16.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:17.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.046+0000 7fb710129700 1 Processor -- start 2024-03-14T09:03:17.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.046+0000 7fb710129700 1 -- start start 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb710129700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb70806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb710129700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080ac8e0 con 0x7fb70806acb0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb710129700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70806de30 con 0x7fb708156500 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb710129700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70806b310 con 0x7fb70814eaf0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70d6c4700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb70806acb0 0x7fb708155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35216/0 (socket says 172.21.15.154:35216) 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70d6c4700 1 -- 172.21.15.154:0/638984239 learned_addr learned my addr 172.21.15.154:0/638984239 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70dec5700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb70806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347014717 0 0) 0x7fb7080ac8e0 con 0x7fb70806acb0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f40035b0 con 0x7fb70806acb0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258809925 0 0) 0x7fb6f40035b0 con 0x7fb70806acb0 2024-03-14T09:03:17.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 msgr2=0x7fb70806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb70806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 >> v1:172.21.15.191:6789/0 conn(0x7fb708156500 legacy=0x7fb70815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708156950 con 0x7fb70806acb0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fb6f8001670 con 0x7fb70806acb0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 >> v1:172.21.15.154:6789/0 conn(0x7fb70806acb0 legacy=0x7fb708155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb6f4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6f4003f70 0x7fb6f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f4004d90 0x7fb6f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70806b310 con 0x7fb6f4004d90 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f400bbd0 con 0x7fb6f4003f70 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.047+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f400bd90 con 0x7fb70814eaf0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f4004d90 0x7fb6f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70dec5700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb6f4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70d6c4700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6f4003f70 0x7fb6f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 -- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 msgr2=0x7fb6f4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb70814eaf0 0x7fb6f4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 -- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6f4003f70 msgr2=0x7fb6f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6f4003f70 0x7fb6f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 -- 172.21.15.154:0/638984239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40035b0 con 0x7fb6f4004d90 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f4004d90 0x7fb6f4005210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c240 tx=0x7fb6f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06831eb0fad1531 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700001e70 con 0x7fb6f4004d90 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000cde0 con 0x7fb6f4004d90 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.048+0000 7fb70cec3700 1 -- 172.21.15.154:0/638984239 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700012660 con 0x7fb6f4004d90 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 -- 172.21.15.154:0/638984239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f4004d90 msgr2=0x7fb6f4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 --2- 172.21.15.154:0/638984239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f4004d90 0x7fb6f4005210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c240 tx=0x7fb6f4005880 comp rx=0 tx=0).stop 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb70e6c6700 1 -- 172.21.15.154:0/638984239 reap_dead start 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 -- 172.21.15.154:0/638984239 shutdown_connections 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 -- 172.21.15.154:0/638984239 >> 172.21.15.154:0/638984239 conn(0x7fb7080c3880 msgr2=0x7fb7080c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 -- 172.21.15.154:0/638984239 shutdown_connections 2024-03-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.049+0000 7fb710129700 1 -- 172.21.15.154:0/638984239 wait complete. 2024-03-14T09:03:17.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.050+0000 7fb710129700 1 Processor -- start 2024-03-14T09:03:17.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.050+0000 7fb710129700 1 -- start start 2024-03-14T09:03:17.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7081ed7a0 0x7fb7081edb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb708143c60 0x7fb7081519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081dee10 con 0x7fb708151f10 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70814f650 con 0x7fb708143c60 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081ee630 con 0x7fb7081ed7a0 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56330/0 (socket says 172.21.15.154:56330) 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 -- 172.21.15.154:0/119480410 learned_addr learned my addr 172.21.15.154:0/119480410 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70dec5700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7081ed7a0 0x7fb7081edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70d6c4700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb708143c60 0x7fb7081519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 -- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7081ed7a0 msgr2=0x7fb7081edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7081ed7a0 0x7fb7081edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 -- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb708143c60 msgr2=0x7fb7081519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb708143c60 0x7fb7081519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 -- 172.21.15.154:0/119480410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7081ee0c0 con 0x7fb708151f10 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb70e6c6700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400bba0 tx=0x7fb700001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700003de0 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700020070 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700005300 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70814f320 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.051+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb708143730 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.052+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7081438c0 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.053+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb700022030 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.053+0000 7fb6fe7fc700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 0x7fb6ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.053+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb700089a50 con 0x7fb708151f10 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.053+0000 7fb70dec5700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 0x7fb6ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.054+0000 7fb70dec5700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 0x7fb6ec06ccf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb70814fc70 tx=0x7fb704009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.058+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb700054d80 con 0x7fb708151f10 2024-03-14T09:03:17.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.248+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb70814f4b0 con 0x7fb6ec06a830 2024-03-14T09:03:17.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.262+0000 7fb6fe7fc700 1 -- 172.21.15.154:0/119480410 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524924 (secure 0 0 0) 0x7fb704019070 con 0x7fb6ec06a830 2024-03-14T09:03:17.268 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 msgr2=0x7fb6ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 0x7fb6ec06ccf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb70814fc70 tx=0x7fb704009210 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 msgr2=0x7fb7080c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400bba0 tx=0x7fb700001490 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 shutdown_connections 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb6ec06a830 0x7fb6ec06ccf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb708151f10 0x7fb7080c1e30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7081ed7a0 0x7fb7081edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 --2- 172.21.15.154:0/119480410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb708143c60 0x7fb7081519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 >> 172.21.15.154:0/119480410 conn(0x7fb7080c3880 msgr2=0x7fb708071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:17.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 shutdown_connections 2024-03-14T09:03:17.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.267+0000 7fb710129700 1 -- 172.21.15.154:0/119480410 wait complete. 2024-03-14T09:03:17.274 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:17.279 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-03-14T09:03:15.544985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222312,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752408,"statfs":{"total":773094113280,"available":772866465792,"internally_reserved":0,"allocated":6283264,"data_stored":3329522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.196195"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353917+0000","last_change":"2024-03-14T09:03:12.353917+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353917+0000","last_undegraded":"2024-03-14T09:03:12.353917+0000","last_fullsized":"2024-03-14T09:03:12.353917+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353360+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:11.353360+0000","last_peered":"2024-03-14T09:03:11.353360+0000","last_clean":"2024-03-14T09:03:11.353360+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:11.353360+0000","last_undegraded":"2024-03-14T09:03:11.353360+0000","last_fullsized":"2024-03-14T09:03:11.353360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361638+0000","last_change":"2024-03-14T09:03:12.361638+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361638+0000","last_undegraded":"2024-03-14T09:03:12.361638+0000","last_fullsized":"2024-03-14T09:03:12.361638+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353404+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:11.353404+0000","last_peered":"2024-03-14T09:03:11.353404+0000","last_clean":"2024-03-14T09:03:11.353404+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:11.353404+0000","last_undegraded":"2024-03-14T09:03:11.353404+0000","last_fullsized":"2024-03-14T09:03:11.353404+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358323+0000","last_change":"2024-03-14T09:03:12.358323+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358323+0000","last_undegraded":"2024-03-14T09:03:12.358323+0000","last_fullsized":"2024-03-14T09:03:12.358323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358248+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:11.358248+0000","last_peered":"2024-03-14T09:03:11.358248+0000","last_clean":"2024-03-14T09:03:11.358248+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:11.358248+0000","last_undegraded":"2024-03-14T09:03:11.358248+0000","last_fullsized":"2024-03-14T09:03:11.358248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354698+0000","last_change":"2024-03-14T09:03:12.354698+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354698+0000","last_undegraded":"2024-03-14T09:03:12.354698+0000","last_fullsized":"2024-03-14T09:03:12.354698+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355314+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:11.355314+0000","last_peered":"2024-03-14T09:03:11.355314+0000","last_clean":"2024-03-14T09:03:11.355314+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:11.355314+0000","last_undegraded":"2024-03-14T09:03:11.355314+0000","last_fullsized":"2024-03-14T09:03:11.355314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362720+0000","last_change":"2024-03-14T09:03:12.362720+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362720+0000","last_undegraded":"2024-03-14T09:03:12.362720+0000","last_fullsized":"2024-03-14T09:03:12.362720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353370+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:11.353370+0000","last_peered":"2024-03-14T09:03:11.353370+0000","last_clean":"2024-03-14T09:03:11.353370+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:11.353370+0000","last_undegraded":"2024-03-14T09:03:11.353370+0000","last_fullsized":"2024-03-14T09:03:11.353370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353623+0000","last_change":"2024-03-14T09:03:12.353623+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353623+0000","last_undegraded":"2024-03-14T09:03:12.353623+0000","last_fullsized":"2024-03-14T09:03:12.353623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349110+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:11.349110+0000","last_peered":"2024-03-14T09:03:11.349110+0000","last_clean":"2024-03-14T09:03:11.349110+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:11.349110+0000","last_undegraded":"2024-03-14T09:03:11.349110+0000","last_fullsized":"2024-03-14T09:03:11.349110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360227+0000","last_change":"2024-03-14T09:03:12.360227+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360227+0000","last_undegraded":"2024-03-14T09:03:12.360227+0000","last_fullsized":"2024-03-14T09:03:12.360227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355387+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:11.355387+0000","last_peered":"2024-03-14T09:03:11.355387+0000","last_clean":"2024-03-14T09:03:11.355387+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:11.355387+0000","last_undegraded":"2024-03-14T09:03:11.355387+0000","last_fullsized":"2024-03-14T09:03:11.355387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353748+0000","last_change":"2024-03-14T09:03:12.353748+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353748+0000","last_undegraded":"2024-03-14T09:03:12.353748+0000","last_fullsized":"2024-03-14T09:03:12.353748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351097+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:11.351097+0000","last_peered":"2024-03-14T09:03:11.351097+0000","last_clean":"2024-03-14T09:03:11.351097+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:11.351097+0000","last_undegraded":"2024-03-14T09:03:11.351097+0000","last_fullsized":"2024-03-14T09:03:11.351097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357388+0000","last_change":"2024-03-14T09:03:12.357388+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357388+0000","last_undegraded":"2024-03-14T09:03:12.357388+0000","last_fullsized":"2024-03-14T09:03:12.357388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356978+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:11.356978+0000","last_peered":"2024-03-14T09:03:11.356978+0000","last_clean":"2024-03-14T09:03:11.356978+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:11.356978+0000","last_undegraded":"2024-03-14T09:03:11.356978+0000","last_fullsized":"2024-03-14T09:03:11.356978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362629+0000","last_change":"2024-03-14T09:03:12.362629+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362629+0000","last_undegraded":"2024-03-14T09:03:12.362629+0000","last_fullsized":"2024-03-14T09:03:12.362629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357377+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:11.357377+0000","last_peered":"2024-03-14T09:03:11.357377+0000","last_clean":"2024-03-14T09:03:11.357377+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:11.357377+0000","last_undegraded":"2024-03-14T09:03:11.357377+0000","last_fullsized":"2024-03-14T09:03:11.357377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359448+0000","last_change":"2024-03-14T09:03:12.359448+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359448+0000","last_undegraded":"2024-03-14T09:03:12.359448+0000","last_fullsized":"2024-03-14T09:03:12.359448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357336+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:11.357336+0000","last_peered":"2024-03-14T09:03:11.357336+0000","last_clean":"2024-03-14T09:03:11.357336+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:11.357336+0000","last_undegraded":"2024-03-14T09:03:11.357336+0000","last_fullsized":"2024-03-14T09:03:11.357336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364654+0000","last_change":"2024-03-14T09:03:12.364654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364654+0000","last_undegraded":"2024-03-14T09:03:12.364654+0000","last_fullsized":"2024-03-14T09:03:12.364654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355433+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:11.355433+0000","last_peered":"2024-03-14T09:03:11.355433+0000","last_clean":"2024-03-14T09:03:11.355433+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:11.355433+0000","last_undegraded":"2024-03-14T09:03:11.355433+0000","last_fullsized":"2024-03-14T09:03:11.355433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356123+0000","last_change":"2024-03-14T09:03:12.356123+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356123+0000","last_undegraded":"2024-03-14T09:03:12.356123+0000","last_fullsized":"2024-03-14T09:03:12.356123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353947+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:11.353947+0000","last_peered":"2024-03-14T09:03:11.353947+0000","last_clean":"2024-03-14T09:03:11.353947+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:11.353947+0000","last_undegraded":"2024-03-14T09:03:11.353947+0000","last_fullsized":"2024-03-14T09:03:11.353947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361949+0000","last_change":"2024-03-14T09:03:12.361949+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361949+0000","last_undegraded":"2024-03-14T09:03:12.361949+0000","last_fullsized":"2024-03-14T09:03:12.361949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355064+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:11.355064+0000","last_peered":"2024-03-14T09:03:11.355064+0000","last_clean":"2024-03-14T09:03:11.355064+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:11.355064+0000","last_undegraded":"2024-03-14T09:03:11.355064+0000","last_fullsized":"2024-03-14T09:03:11.355064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355497+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:11.355497+0000","last_peered":"2024-03-14T09:03:11.355497+0000","last_clean":"2024-03-14T09:03:11.355497+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:11.355497+0000","last_undegraded":"2024-03-14T09:03:11.355497+0000","last_fullsized":"2024-03-14T09:03:11.355497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364020+0000","last_change":"2024-03-14T09:03:12.364020+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364020+0000","last_undegraded":"2024-03-14T09:03:12.364020+0000","last_fullsized":"2024-03-14T09:03:12.364020+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.348935+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:11.348935+0000","last_peered":"2024-03-14T09:03:11.348935+0000","last_clean":"2024-03-14T09:03:11.348935+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:11.348935+0000","last_undegraded":"2024-03-14T09:03:11.348935+0000","last_fullsized":"2024-03-14T09:03:11.348935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353377+0000","last_change":"2024-03-14T09:03:12.353377+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353377+0000","last_undegraded":"2024-03-14T09:03:12.353377+0000","last_fullsized":"2024-03-14T09:03:12.353377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353394+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:11.353394+0000","last_peered":"2024-03-14T09:03:11.353394+0000","last_clean":"2024-03-14T09:03:11.353394+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:11.353394+0000","last_undegraded":"2024-03-14T09:03:11.353394+0000","last_fullsized":"2024-03-14T09:03:11.353394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356427+0000","last_change":"2024-03-14T09:03:12.356427+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356427+0000","last_undegraded":"2024-03-14T09:03:12.356427+0000","last_fullsized":"2024-03-14T09:03:12.356427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358442+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:11.358442+0000","last_peered":"2024-03-14T09:03:11.358442+0000","last_clean":"2024-03-14T09:03:11.358442+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:11.358442+0000","last_undegraded":"2024-03-14T09:03:11.358442+0000","last_fullsized":"2024-03-14T09:03:11.358442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360390+0000","last_change":"2024-03-14T09:03:12.360390+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360390+0000","last_undegraded":"2024-03-14T09:03:12.360390+0000","last_fullsized":"2024-03-14T09:03:12.360390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353260+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:11.353260+0000","last_peered":"2024-03-14T09:03:11.353260+0000","last_clean":"2024-03-14T09:03:11.353260+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:11.353260+0000","last_undegraded":"2024-03-14T09:03:11.353260+0000","last_fullsized":"2024-03-14T09:03:11.353260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359276+0000","last_change":"2024-03-14T09:03:12.359276+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359276+0000","last_undegraded":"2024-03-14T09:03:12.359276+0000","last_fullsized":"2024-03-14T09:03:12.359276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351650+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:11.351650+0000","last_peered":"2024-03-14T09:03:11.351650+0000","last_clean":"2024-03-14T09:03:11.351650+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:11.351650+0000","last_undegraded":"2024-03-14T09:03:11.351650+0000","last_fullsized":"2024-03-14T09:03:11.351650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356216+0000","last_change":"2024-03-14T09:03:12.356216+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356216+0000","last_undegraded":"2024-03-14T09:03:12.356216+0000","last_fullsized":"2024-03-14T09:03:12.356216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358150+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:11.358150+0000","last_peered":"2024-03-14T09:03:11.358150+0000","last_clean":"2024-03-14T09:03:11.358150+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:11.358150+0000","last_undegraded":"2024-03-14T09:03:11.358150+0000","last_fullsized":"2024-03-14T09:03:11.358150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357282+0000","last_change":"2024-03-14T09:03:12.357282+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357282+0000","last_undegraded":"2024-03-14T09:03:12.357282+0000","last_fullsized":"2024-03-14T09:03:12.357282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351242+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:11.351242+0000","last_peered":"2024-03-14T09:03:11.351242+0000","last_clean":"2024-03-14T09:03:11.351242+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:11.351242+0000","last_undegraded":"2024-03-14T09:03:11.351242+0000","last_fullsized":"2024-03-14T09:03:11.351242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364628+0000","last_change":"2024-03-14T09:03:12.364628+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364628+0000","last_undegraded":"2024-03-14T09:03:12.364628+0000","last_fullsized":"2024-03-14T09:03:12.364628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358083+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:11.358083+0000","last_peered":"2024-03-14T09:03:11.358083+0000","last_clean":"2024-03-14T09:03:11.358083+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:11.358083+0000","last_undegraded":"2024-03-14T09:03:11.358083+0000","last_fullsized":"2024-03-14T09:03:11.358083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362012+0000","last_change":"2024-03-14T09:03:12.362012+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362012+0000","last_undegraded":"2024-03-14T09:03:12.362012+0000","last_fullsized":"2024-03-14T09:03:12.362012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353228+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.353228+0000","last_peered":"2024-03-14T09:03:11.353228+0000","last_clean":"2024-03-14T09:03:11.353228+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:11.353228+0000","last_undegraded":"2024-03-14T09:03:11.353228+0000","last_fullsized":"2024-03-14T09:03:11.353228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352812+0000","last_change":"2024-03-14T09:03:12.352812+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352812+0000","last_undegraded":"2024-03-14T09:03:12.352812+0000","last_fullsized":"2024-03-14T09:03:12.352812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353434+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:11.353434+0000","last_peered":"2024-03-14T09:03:11.353434+0000","last_clean":"2024-03-14T09:03:11.353434+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:11.353434+0000","last_undegraded":"2024-03-14T09:03:11.353434+0000","last_fullsized":"2024-03-14T09:03:11.353434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361474+0000","last_change":"2024-03-14T09:03:12.361474+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361474+0000","last_undegraded":"2024-03-14T09:03:12.361474+0000","last_fullsized":"2024-03-14T09:03:12.361474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355459+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:11.355459+0000","last_peered":"2024-03-14T09:03:11.355459+0000","last_clean":"2024-03-14T09:03:11.355459+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:11.355459+0000","last_undegraded":"2024-03-14T09:03:11.355459+0000","last_fullsized":"2024-03-14T09:03:11.355459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361988+0000","last_change":"2024-03-14T09:03:12.361988+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361988+0000","last_undegraded":"2024-03-14T09:03:12.361988+0000","last_fullsized":"2024-03-14T09:03:12.361988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362109+0000","last_change":"2024-03-14T09:03:12.362109+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362109+0000","last_undegraded":"2024-03-14T09:03:12.362109+0000","last_fullsized":"2024-03-14T09:03:12.362109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353264+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:11.353264+0000","last_peered":"2024-03-14T09:03:11.353264+0000","last_clean":"2024-03-14T09:03:11.353264+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:11.353264+0000","last_undegraded":"2024-03-14T09:03:11.353264+0000","last_fullsized":"2024-03-14T09:03:11.353264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362252+0000","last_change":"2024-03-14T09:03:12.362252+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362252+0000","last_undegraded":"2024-03-14T09:03:12.362252+0000","last_fullsized":"2024-03-14T09:03:12.362252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351526+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:11.351526+0000","last_peered":"2024-03-14T09:03:11.351526+0000","last_clean":"2024-03-14T09:03:11.351526+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:11.351526+0000","last_undegraded":"2024-03-14T09:03:11.351526+0000","last_fullsized":"2024-03-14T09:03:11.351526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353125+0000","last_change":"2024-03-14T09:03:12.353125+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353125+0000","last_undegraded":"2024-03-14T09:03:12.353125+0000","last_fullsized":"2024-03-14T09:03:12.353125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358400+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:11.358400+0000","last_peered":"2024-03-14T09:03:11.358400+0000","last_clean":"2024-03-14T09:03:11.358400+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:11.358400+0000","last_undegraded":"2024-03-14T09:03:11.358400+0000","last_fullsized":"2024-03-14T09:03:11.358400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354173+0000","last_change":"2024-03-14T09:03:12.354173+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354173+0000","last_undegraded":"2024-03-14T09:03:12.354173+0000","last_fullsized":"2024-03-14T09:03:12.354173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.352840+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:11.352840+0000","last_peered":"2024-03-14T09:03:11.352840+0000","last_clean":"2024-03-14T09:03:11.352840+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:11.352840+0000","last_undegraded":"2024-03-14T09:03:11.352840+0000","last_fullsized":"2024-03-14T09:03:11.352840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361509+0000","last_change":"2024-03-14T09:03:12.361509+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361509+0000","last_undegraded":"2024-03-14T09:03:12.361509+0000","last_fullsized":"2024-03-14T09:03:12.361509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349002+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:11.349002+0000","last_peered":"2024-03-14T09:03:11.349002+0000","last_clean":"2024-03-14T09:03:11.349002+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:11.349002+0000","last_undegraded":"2024-03-14T09:03:11.349002+0000","last_fullsized":"2024-03-14T09:03:11.349002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354070+0000","last_change":"2024-03-14T09:03:12.354070+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354070+0000","last_undegraded":"2024-03-14T09:03:12.354070+0000","last_fullsized":"2024-03-14T09:03:12.354070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376057+0000","last_change":"2024-03-14T09:03:12.371272+0000","last_active":"2024-03-14T09:03:12.376057+0000","last_peered":"2024-03-14T09:03:12.376057+0000","last_clean":"2024-03-14T09:03:12.376057+0000","last_became_active":"2024-03-14T09:03:12.370996+0000","last_became_peered":"2024-03-14T09:03:12.370996+0000","last_unstale":"2024-03-14T09:03:12.376057+0000","last_undegraded":"2024-03-14T09:03:12.376057+0000","last_fullsized":"2024-03-14T09:03:12.376057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:44.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349268+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:11.349268+0000","last_peered":"2024-03-14T09:03:11.349268+0000","last_clean":"2024-03-14T09:03:11.349268+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:11.349268+0000","last_undegraded":"2024-03-14T09:03:11.349268+0000","last_fullsized":"2024-03-14T09:03:11.349268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349423+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:11.349423+0000","last_peered":"2024-03-14T09:03:11.349423+0000","last_clean":"2024-03-14T09:03:11.349423+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:11.349423+0000","last_undegraded":"2024-03-14T09:03:11.349423+0000","last_fullsized":"2024-03-14T09:03:11.349423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356924+0000","last_change":"2024-03-14T09:03:12.356924+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356924+0000","last_undegraded":"2024-03-14T09:03:12.356924+0000","last_fullsized":"2024-03-14T09:03:12.356924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378773+0000","last_change":"2024-03-14T09:03:12.374278+0000","last_active":"2024-03-14T09:03:12.378773+0000","last_peered":"2024-03-14T09:03:12.378773+0000","last_clean":"2024-03-14T09:03:12.378773+0000","last_became_active":"2024-03-14T09:03:12.373968+0000","last_became_peered":"2024-03-14T09:03:12.373968+0000","last_unstale":"2024-03-14T09:03:12.378773+0000","last_undegraded":"2024-03-14T09:03:12.378773+0000","last_fullsized":"2024-03-14T09:03:12.378773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:40.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358207+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:11.358207+0000","last_peered":"2024-03-14T09:03:11.358207+0000","last_clean":"2024-03-14T09:03:11.358207+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:11.358207+0000","last_undegraded":"2024-03-14T09:03:11.358207+0000","last_fullsized":"2024-03-14T09:03:11.358207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353512+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:11.353512+0000","last_peered":"2024-03-14T09:03:11.353512+0000","last_clean":"2024-03-14T09:03:11.353512+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:11.353512+0000","last_undegraded":"2024-03-14T09:03:11.353512+0000","last_fullsized":"2024-03-14T09:03:11.353512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377433+0000","last_change":"2024-03-14T09:03:12.373911+0000","last_active":"2024-03-14T09:03:12.377433+0000","last_peered":"2024-03-14T09:03:12.377433+0000","last_clean":"2024-03-14T09:03:12.377433+0000","last_became_active":"2024-03-14T09:03:12.373633+0000","last_became_peered":"2024-03-14T09:03:12.373633+0000","last_unstale":"2024-03-14T09:03:12.377433+0000","last_undegraded":"2024-03-14T09:03:12.377433+0000","last_fullsized":"2024-03-14T09:03:12.377433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349440+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:11.349440+0000","last_peered":"2024-03-14T09:03:11.349440+0000","last_clean":"2024-03-14T09:03:11.349440+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:11.349440+0000","last_undegraded":"2024-03-14T09:03:11.349440+0000","last_fullsized":"2024-03-14T09:03:11.349440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359921+0000","last_change":"2024-03-14T09:03:12.359921+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359921+0000","last_undegraded":"2024-03-14T09:03:12.359921+0000","last_fullsized":"2024-03-14T09:03:12.359921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371201+0000","last_change":"2024-03-14T09:03:12.370015+0000","last_active":"2024-03-14T09:03:12.371201+0000","last_peered":"2024-03-14T09:03:12.371201+0000","last_clean":"2024-03-14T09:03:12.371201+0000","last_became_active":"2024-03-14T09:03:12.369735+0000","last_became_peered":"2024-03-14T09:03:12.369735+0000","last_unstale":"2024-03-14T09:03:12.371201+0000","last_undegraded":"2024-03-14T09:03:12.371201+0000","last_fullsized":"2024-03-14T09:03:12.371201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349286+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.349286+0000","last_peered":"2024-03-14T09:03:11.349286+0000","last_clean":"2024-03-14T09:03:11.349286+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:11.349286+0000","last_undegraded":"2024-03-14T09:03:11.349286+0000","last_fullsized":"2024-03-14T09:03:11.349286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355554+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:11.355554+0000","last_peered":"2024-03-14T09:03:11.355554+0000","last_clean":"2024-03-14T09:03:11.355554+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:11.355554+0000","last_undegraded":"2024-03-14T09:03:11.355554+0000","last_fullsized":"2024-03-14T09:03:11.355554+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360660+0000","last_change":"2024-03-14T09:03:12.360660+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360660+0000","last_undegraded":"2024-03-14T09:03:12.360660+0000","last_fullsized":"2024-03-14T09:03:12.360660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375005+0000","last_change":"2024-03-14T09:03:12.372188+0000","last_active":"2024-03-14T09:03:12.375005+0000","last_peered":"2024-03-14T09:03:12.375005+0000","last_clean":"2024-03-14T09:03:12.375005+0000","last_became_active":"2024-03-14T09:03:12.371919+0000","last_became_peered":"2024-03-14T09:03:12.371919+0000","last_unstale":"2024-03-14T09:03:12.375005+0000","last_undegraded":"2024-03-14T09:03:12.375005+0000","last_fullsized":"2024-03-14T09:03:12.375005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351321+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:11.351321+0000","last_peered":"2024-03-14T09:03:11.351321+0000","last_clean":"2024-03-14T09:03:11.351321+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:11.351321+0000","last_undegraded":"2024-03-14T09:03:11.351321+0000","last_fullsized":"2024-03-14T09:03:11.351321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355947+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:11.355947+0000","last_peered":"2024-03-14T09:03:11.355947+0000","last_clean":"2024-03-14T09:03:11.355947+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:11.355947+0000","last_undegraded":"2024-03-14T09:03:11.355947+0000","last_fullsized":"2024-03-14T09:03:11.355947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355339+0000","last_change":"2024-03-14T09:03:12.355339+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355339+0000","last_undegraded":"2024-03-14T09:03:12.355339+0000","last_fullsized":"2024-03-14T09:03:12.355339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372079+0000","last_change":"2024-03-14T09:03:12.369918+0000","last_active":"2024-03-14T09:03:12.372079+0000","last_peered":"2024-03-14T09:03:12.372079+0000","last_clean":"2024-03-14T09:03:12.372079+0000","last_became_active":"2024-03-14T09:03:12.369599+0000","last_became_peered":"2024-03-14T09:03:12.369599+0000","last_unstale":"2024-03-14T09:03:12.372079+0000","last_undegraded":"2024-03-14T09:03:12.372079+0000","last_fullsized":"2024-03-14T09:03:12.372079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:52.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351535+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:11.351535+0000","last_peered":"2024-03-14T09:03:11.351535+0000","last_clean":"2024-03-14T09:03:11.351535+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:11.351535+0000","last_undegraded":"2024-03-14T09:03:11.351535+0000","last_fullsized":"2024-03-14T09:03:11.351535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349367+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:11.349367+0000","last_peered":"2024-03-14T09:03:11.349367+0000","last_clean":"2024-03-14T09:03:11.349367+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:11.349367+0000","last_undegraded":"2024-03-14T09:03:11.349367+0000","last_fullsized":"2024-03-14T09:03:11.349367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361025+0000","last_change":"2024-03-14T09:03:12.361025+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361025+0000","last_undegraded":"2024-03-14T09:03:12.361025+0000","last_fullsized":"2024-03-14T09:03:12.361025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374895+0000","last_change":"2024-03-14T09:03:12.373841+0000","last_active":"2024-03-14T09:03:12.374895+0000","last_peered":"2024-03-14T09:03:12.374895+0000","last_clean":"2024-03-14T09:03:12.374895+0000","last_became_active":"2024-03-14T09:03:12.373551+0000","last_became_peered":"2024-03-14T09:03:12.373551+0000","last_unstale":"2024-03-14T09:03:12.374895+0000","last_undegraded":"2024-03-14T09:03:12.374895+0000","last_fullsized":"2024-03-14T09:03:12.374895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.965540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353268+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:11.353268+0000","last_peered":"2024-03-14T09:03:11.353268+0000","last_clean":"2024-03-14T09:03:11.353268+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:11.353268+0000","last_undegraded":"2024-03-14T09:03:11.353268+0000","last_fullsized":"2024-03-14T09:03:11.353268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355710+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:11.355710+0000","last_peered":"2024-03-14T09:03:11.355710+0000","last_clean":"2024-03-14T09:03:11.355710+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:11.355710+0000","last_undegraded":"2024-03-14T09:03:11.355710+0000","last_fullsized":"2024-03-14T09:03:11.355710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361000+0000","last_change":"2024-03-14T09:03:12.361000+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361000+0000","last_undegraded":"2024-03-14T09:03:12.361000+0000","last_fullsized":"2024-03-14T09:03:12.361000+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368944+0000","last_change":"2024-03-14T09:03:12.366900+0000","last_active":"2024-03-14T09:03:12.368944+0000","last_peered":"2024-03-14T09:03:12.368944+0000","last_clean":"2024-03-14T09:03:12.368944+0000","last_became_active":"2024-03-14T09:03:12.366301+0000","last_became_peered":"2024-03-14T09:03:12.366301+0000","last_unstale":"2024-03-14T09:03:12.368944+0000","last_undegraded":"2024-03-14T09:03:12.368944+0000","last_fullsized":"2024-03-14T09:03:12.368944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:11.884116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354744+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:11.354744+0000","last_peered":"2024-03-14T09:03:11.354744+0000","last_clean":"2024-03-14T09:03:11.354744+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:11.354744+0000","last_undegraded":"2024-03-14T09:03:11.354744+0000","last_fullsized":"2024-03-14T09:03:11.354744+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353503+0000","last_change":"2024-03-14T09:03:12.353503+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353503+0000","last_undegraded":"2024-03-14T09:03:12.353503+0000","last_fullsized":"2024-03-14T09:03:12.353503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371091+0000","last_change":"2024-03-14T09:03:12.367305+0000","last_active":"2024-03-14T09:03:12.371091+0000","last_peered":"2024-03-14T09:03:12.371091+0000","last_clean":"2024-03-14T09:03:12.371091+0000","last_became_active":"2024-03-14T09:03:12.367020+0000","last_became_peered":"2024-03-14T09:03:12.367020+0000","last_unstale":"2024-03-14T09:03:12.371091+0000","last_undegraded":"2024-03-14T09:03:12.371091+0000","last_fullsized":"2024-03-14T09:03:12.371091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:39.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356736+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:11.356736+0000","last_peered":"2024-03-14T09:03:11.356736+0000","last_clean":"2024-03-14T09:03:11.356736+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:11.356736+0000","last_undegraded":"2024-03-14T09:03:11.356736+0000","last_fullsized":"2024-03-14T09:03:11.356736+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358626+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:11.358626+0000","last_peered":"2024-03-14T09:03:11.358626+0000","last_clean":"2024-03-14T09:03:11.358626+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:11.358626+0000","last_undegraded":"2024-03-14T09:03:11.358626+0000","last_fullsized":"2024-03-14T09:03:11.358626+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364515+0000","last_change":"2024-03-14T09:03:12.364515+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364515+0000","last_undegraded":"2024-03-14T09:03:12.364515+0000","last_fullsized":"2024-03-14T09:03:12.364515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373487+0000","last_change":"2024-03-14T09:03:12.370627+0000","last_active":"2024-03-14T09:03:12.373487+0000","last_peered":"2024-03-14T09:03:12.373487+0000","last_clean":"2024-03-14T09:03:12.373487+0000","last_became_active":"2024-03-14T09:03:12.370314+0000","last_became_peered":"2024-03-14T09:03:12.370314+0000","last_unstale":"2024-03-14T09:03:12.373487+0000","last_undegraded":"2024-03-14T09:03:12.373487+0000","last_fullsized":"2024-03-14T09:03:12.373487+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:09.613459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358558+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:11.358558+0000","last_peered":"2024-03-14T09:03:11.358558+0000","last_clean":"2024-03-14T09:03:11.358558+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:11.358558+0000","last_undegraded":"2024-03-14T09:03:11.358558+0000","last_fullsized":"2024-03-14T09:03:11.358558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353599+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:11.353599+0000","last_peered":"2024-03-14T09:03:11.353599+0000","last_clean":"2024-03-14T09:03:11.353599+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:11.353599+0000","last_undegraded":"2024-03-14T09:03:11.353599+0000","last_fullsized":"2024-03-14T09:03:11.353599+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359668+0000","last_change":"2024-03-14T09:03:12.359668+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359668+0000","last_undegraded":"2024-03-14T09:03:12.359668+0000","last_fullsized":"2024-03-14T09:03:12.359668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378802+0000","last_change":"2024-03-14T09:03:12.374028+0000","last_active":"2024-03-14T09:03:12.378802+0000","last_peered":"2024-03-14T09:03:12.378802+0000","last_clean":"2024-03-14T09:03:12.378802+0000","last_became_active":"2024-03-14T09:03:12.373770+0000","last_became_peered":"2024-03-14T09:03:12.373770+0000","last_unstale":"2024-03-14T09:03:12.378802+0000","last_undegraded":"2024-03-14T09:03:12.378802+0000","last_fullsized":"2024-03-14T09:03:12.378802+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:51.114973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357121+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:11.357121+0000","last_peered":"2024-03-14T09:03:11.357121+0000","last_clean":"2024-03-14T09:03:11.357121+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:11.357121+0000","last_undegraded":"2024-03-14T09:03:11.357121+0000","last_fullsized":"2024-03-14T09:03:11.357121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358587+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:11.358587+0000","last_peered":"2024-03-14T09:03:11.358587+0000","last_clean":"2024-03-14T09:03:11.358587+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:11.358587+0000","last_undegraded":"2024-03-14T09:03:11.358587+0000","last_fullsized":"2024-03-14T09:03:11.358587+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357946+0000","last_change":"2024-03-14T09:03:12.357946+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357946+0000","last_undegraded":"2024-03-14T09:03:12.357946+0000","last_fullsized":"2024-03-14T09:03:12.357946+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368918+0000","last_change":"2024-03-14T09:03:12.367742+0000","last_active":"2024-03-14T09:03:12.368918+0000","last_peered":"2024-03-14T09:03:12.368918+0000","last_clean":"2024-03-14T09:03:12.368918+0000","last_became_active":"2024-03-14T09:03:12.367471+0000","last_became_peered":"2024-03-14T09:03:12.367471+0000","last_unstale":"2024-03-14T09:03:12.368918+0000","last_undegraded":"2024-03-14T09:03:12.368918+0000","last_fullsized":"2024-03-14T09:03:12.368918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:34.678842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358346+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:11.358346+0000","last_peered":"2024-03-14T09:03:11.358346+0000","last_clean":"2024-03-14T09:03:11.358346+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:11.358346+0000","last_undegraded":"2024-03-14T09:03:11.358346+0000","last_fullsized":"2024-03-14T09:03:11.358346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356446+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:11.356446+0000","last_peered":"2024-03-14T09:03:11.356446+0000","last_clean":"2024-03-14T09:03:11.356446+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:11.356446+0000","last_undegraded":"2024-03-14T09:03:11.356446+0000","last_fullsized":"2024-03-14T09:03:11.356446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355535+0000","last_change":"2024-03-14T09:03:12.355535+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355535+0000","last_undegraded":"2024-03-14T09:03:12.355535+0000","last_fullsized":"2024-03-14T09:03:12.355535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371980+0000","last_change":"2024-03-14T09:03:12.366980+0000","last_active":"2024-03-14T09:03:12.371980+0000","last_peered":"2024-03-14T09:03:12.371980+0000","last_clean":"2024-03-14T09:03:12.371980+0000","last_became_active":"2024-03-14T09:03:12.366728+0000","last_became_peered":"2024-03-14T09:03:12.366728+0000","last_unstale":"2024-03-14T09:03:12.371980+0000","last_undegraded":"2024-03-14T09:03:12.371980+0000","last_fullsized":"2024-03-14T09:03:12.371980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:53.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355135+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:11.355135+0000","last_peered":"2024-03-14T09:03:11.355135+0000","last_clean":"2024-03-14T09:03:11.355135+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:11.355135+0000","last_undegraded":"2024-03-14T09:03:11.355135+0000","last_fullsized":"2024-03-14T09:03:11.355135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351630+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:11.351630+0000","last_peered":"2024-03-14T09:03:11.351630+0000","last_clean":"2024-03-14T09:03:11.351630+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:11.351630+0000","last_undegraded":"2024-03-14T09:03:11.351630+0000","last_fullsized":"2024-03-14T09:03:11.351630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358690+0000","last_change":"2024-03-14T09:03:12.358690+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358690+0000","last_undegraded":"2024-03-14T09:03:12.358690+0000","last_fullsized":"2024-03-14T09:03:12.358690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376094+0000","last_change":"2024-03-14T09:03:12.371369+0000","last_active":"2024-03-14T09:03:12.376094+0000","last_peered":"2024-03-14T09:03:12.376094+0000","last_clean":"2024-03-14T09:03:12.376094+0000","last_became_active":"2024-03-14T09:03:12.371087+0000","last_became_peered":"2024-03-14T09:03:12.371087+0000","last_unstale":"2024-03-14T09:03:12.376094+0000","last_undegraded":"2024-03-14T09:03:12.376094+0000","last_fullsized":"2024-03-14T09:03:12.376094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:38.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358447+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:11.358447+0000","last_peered":"2024-03-14T09:03:11.358447+0000","last_clean":"2024-03-14T09:03:11.358447+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:11.358447+0000","last_undegraded":"2024-03-14T09:03:11.358447+0000","last_fullsized":"2024-03-14T09:03:11.358447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354648+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:11.354648+0000","last_peered":"2024-03-14T09:03:11.354648+0000","last_clean":"2024-03-14T09:03:11.354648+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:11.354648+0000","last_undegraded":"2024-03-14T09:03:11.354648+0000","last_fullsized":"2024-03-14T09:03:11.354648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356321+0000","last_change":"2024-03-14T09:03:12.356321+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356321+0000","last_undegraded":"2024-03-14T09:03:12.356321+0000","last_fullsized":"2024-03-14T09:03:12.356321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373386+0000","last_change":"2024-03-14T09:03:12.368944+0000","last_active":"2024-03-14T09:03:12.373386+0000","last_peered":"2024-03-14T09:03:12.373386+0000","last_clean":"2024-03-14T09:03:12.373386+0000","last_became_active":"2024-03-14T09:03:12.368631+0000","last_became_peered":"2024-03-14T09:03:12.368631+0000","last_unstale":"2024-03-14T09:03:12.373386+0000","last_undegraded":"2024-03-14T09:03:12.373386+0000","last_fullsized":"2024-03-14T09:03:12.373386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354667+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:11.354667+0000","last_peered":"2024-03-14T09:03:11.354667+0000","last_clean":"2024-03-14T09:03:11.354667+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:11.354667+0000","last_undegraded":"2024-03-14T09:03:11.354667+0000","last_fullsized":"2024-03-14T09:03:11.354667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355806+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:11.355806+0000","last_peered":"2024-03-14T09:03:11.355806+0000","last_clean":"2024-03-14T09:03:11.355806+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:11.355806+0000","last_undegraded":"2024-03-14T09:03:11.355806+0000","last_fullsized":"2024-03-14T09:03:11.355806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360063+0000","last_change":"2024-03-14T09:03:12.360063+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360063+0000","last_undegraded":"2024-03-14T09:03:12.360063+0000","last_fullsized":"2024-03-14T09:03:12.360063+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371484+0000","last_change":"2024-03-14T09:03:12.370316+0000","last_active":"2024-03-14T09:03:12.371484+0000","last_peered":"2024-03-14T09:03:12.371484+0000","last_clean":"2024-03-14T09:03:12.371484+0000","last_became_active":"2024-03-14T09:03:12.370035+0000","last_became_peered":"2024-03-14T09:03:12.370035+0000","last_unstale":"2024-03-14T09:03:12.371484+0000","last_undegraded":"2024-03-14T09:03:12.371484+0000","last_fullsized":"2024-03-14T09:03:12.371484+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:07.312244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358188+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:11.358188+0000","last_peered":"2024-03-14T09:03:11.358188+0000","last_clean":"2024-03-14T09:03:11.358188+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:11.358188+0000","last_undegraded":"2024-03-14T09:03:11.358188+0000","last_fullsized":"2024-03-14T09:03:11.358188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358017+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:11.358017+0000","last_peered":"2024-03-14T09:03:11.358017+0000","last_clean":"2024-03-14T09:03:11.358017+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:11.358017+0000","last_undegraded":"2024-03-14T09:03:11.358017+0000","last_fullsized":"2024-03-14T09:03:11.358017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357489+0000","last_change":"2024-03-14T09:03:12.357489+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357489+0000","last_undegraded":"2024-03-14T09:03:12.357489+0000","last_fullsized":"2024-03-14T09:03:12.357489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377230+0000","last_change":"2024-03-14T09:03:12.373196+0000","last_active":"2024-03-14T09:03:12.377230+0000","last_peered":"2024-03-14T09:03:12.377230+0000","last_clean":"2024-03-14T09:03:12.377230+0000","last_became_active":"2024-03-14T09:03:12.372917+0000","last_became_peered":"2024-03-14T09:03:12.372917+0000","last_unstale":"2024-03-14T09:03:12.377230+0000","last_undegraded":"2024-03-14T09:03:12.377230+0000","last_fullsized":"2024-03-14T09:03:12.377230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:56.766593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358565+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:11.358565+0000","last_peered":"2024-03-14T09:03:11.358565+0000","last_clean":"2024-03-14T09:03:11.358565+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:11.358565+0000","last_undegraded":"2024-03-14T09:03:11.358565+0000","last_fullsized":"2024-03-14T09:03:11.358565+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353682+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:11.353682+0000","last_peered":"2024-03-14T09:03:11.353682+0000","last_clean":"2024-03-14T09:03:11.353682+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:11.353682+0000","last_undegraded":"2024-03-14T09:03:11.353682+0000","last_fullsized":"2024-03-14T09:03:11.353682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359244+0000","last_change":"2024-03-14T09:03:12.359244+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359244+0000","last_undegraded":"2024-03-14T09:03:12.359244+0000","last_fullsized":"2024-03-14T09:03:12.359244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377421+0000","last_change":"2024-03-14T09:03:12.373227+0000","last_active":"2024-03-14T09:03:12.377421+0000","last_peered":"2024-03-14T09:03:12.377421+0000","last_clean":"2024-03-14T09:03:12.377421+0000","last_became_active":"2024-03-14T09:03:12.372761+0000","last_became_peered":"2024-03-14T09:03:12.372761+0000","last_unstale":"2024-03-14T09:03:12.377421+0000","last_undegraded":"2024-03-14T09:03:12.377421+0000","last_fullsized":"2024-03-14T09:03:12.377421+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355667+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:11.355667+0000","last_peered":"2024-03-14T09:03:11.355667+0000","last_clean":"2024-03-14T09:03:11.355667+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:11.355667+0000","last_undegraded":"2024-03-14T09:03:11.355667+0000","last_fullsized":"2024-03-14T09:03:11.355667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349366+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:11.349366+0000","last_peered":"2024-03-14T09:03:11.349366+0000","last_clean":"2024-03-14T09:03:11.349366+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:11.349366+0000","last_undegraded":"2024-03-14T09:03:11.349366+0000","last_fullsized":"2024-03-14T09:03:11.349366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364238+0000","last_change":"2024-03-14T09:03:12.364238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364238+0000","last_undegraded":"2024-03-14T09:03:12.364238+0000","last_fullsized":"2024-03-14T09:03:12.364238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367869+0000","last_change":"2024-03-14T09:03:12.365684+0000","last_active":"2024-03-14T09:03:12.367869+0000","last_peered":"2024-03-14T09:03:12.367869+0000","last_clean":"2024-03-14T09:03:12.367869+0000","last_became_active":"2024-03-14T09:03:12.365263+0000","last_became_peered":"2024-03-14T09:03:12.365263+0000","last_unstale":"2024-03-14T09:03:12.367869+0000","last_undegraded":"2024-03-14T09:03:12.367869+0000","last_fullsized":"2024-03-14T09:03:12.367869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351721+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:11.351721+0000","last_peered":"2024-03-14T09:03:11.351721+0000","last_clean":"2024-03-14T09:03:11.351721+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:11.351721+0000","last_undegraded":"2024-03-14T09:03:11.351721+0000","last_fullsized":"2024-03-14T09:03:11.351721+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355737+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:11.355737+0000","last_peered":"2024-03-14T09:03:11.355737+0000","last_clean":"2024-03-14T09:03:11.355737+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:11.355737+0000","last_undegraded":"2024-03-14T09:03:11.355737+0000","last_fullsized":"2024-03-14T09:03:11.355737+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357463+0000","last_change":"2024-03-14T09:03:12.357463+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357463+0000","last_undegraded":"2024-03-14T09:03:12.357463+0000","last_fullsized":"2024-03-14T09:03:12.357463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376072+0000","last_change":"2024-03-14T09:03:12.371367+0000","last_active":"2024-03-14T09:03:12.376072+0000","last_peered":"2024-03-14T09:03:12.376072+0000","last_clean":"2024-03-14T09:03:12.376072+0000","last_became_active":"2024-03-14T09:03:12.371044+0000","last_became_peered":"2024-03-14T09:03:12.371044+0000","last_unstale":"2024-03-14T09:03:12.376072+0000","last_undegraded":"2024-03-14T09:03:12.376072+0000","last_fullsized":"2024-03-14T09:03:12.376072+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:20:22.371301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358462+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:11.358462+0000","last_peered":"2024-03-14T09:03:11.358462+0000","last_clean":"2024-03-14T09:03:11.358462+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:11.358462+0000","last_undegraded":"2024-03-14T09:03:11.358462+0000","last_fullsized":"2024-03-14T09:03:11.358462+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349280+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:11.349280+0000","last_peered":"2024-03-14T09:03:11.349280+0000","last_clean":"2024-03-14T09:03:11.349280+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:11.349280+0000","last_undegraded":"2024-03-14T09:03:11.349280+0000","last_fullsized":"2024-03-14T09:03:11.349280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352704+0000","last_change":"2024-03-14T09:03:12.352704+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352704+0000","last_undegraded":"2024-03-14T09:03:12.352704+0000","last_fullsized":"2024-03-14T09:03:12.352704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374596+0000","last_change":"2024-03-14T09:03:12.372117+0000","last_active":"2024-03-14T09:03:12.374596+0000","last_peered":"2024-03-14T09:03:12.374596+0000","last_clean":"2024-03-14T09:03:12.374596+0000","last_became_active":"2024-03-14T09:03:12.371774+0000","last_became_peered":"2024-03-14T09:03:12.371774+0000","last_unstale":"2024-03-14T09:03:12.374596+0000","last_undegraded":"2024-03-14T09:03:12.374596+0000","last_fullsized":"2024-03-14T09:03:12.374596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:04.409001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355185+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:11.355185+0000","last_peered":"2024-03-14T09:03:11.355185+0000","last_clean":"2024-03-14T09:03:11.355185+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:11.355185+0000","last_undegraded":"2024-03-14T09:03:11.355185+0000","last_fullsized":"2024-03-14T09:03:11.355185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351724+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:11.351724+0000","last_peered":"2024-03-14T09:03:11.351724+0000","last_clean":"2024-03-14T09:03:11.351724+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:11.351724+0000","last_undegraded":"2024-03-14T09:03:11.351724+0000","last_fullsized":"2024-03-14T09:03:11.351724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354163+0000","last_change":"2024-03-14T09:03:12.354163+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354163+0000","last_undegraded":"2024-03-14T09:03:12.354163+0000","last_fullsized":"2024-03-14T09:03:12.354163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372069+0000","last_change":"2024-03-14T09:03:12.370993+0000","last_active":"2024-03-14T09:03:12.372069+0000","last_peered":"2024-03-14T09:03:12.372069+0000","last_clean":"2024-03-14T09:03:12.372069+0000","last_became_active":"2024-03-14T09:03:12.370745+0000","last_became_peered":"2024-03-14T09:03:12.370745+0000","last_unstale":"2024-03-14T09:03:12.372069+0000","last_undegraded":"2024-03-14T09:03:12.372069+0000","last_fullsized":"2024-03-14T09:03:12.372069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349338+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:11.349338+0000","last_peered":"2024-03-14T09:03:11.349338+0000","last_clean":"2024-03-14T09:03:11.349338+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:11.349338+0000","last_undegraded":"2024-03-14T09:03:11.349338+0000","last_fullsized":"2024-03-14T09:03:11.349338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356757+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:11.356757+0000","last_peered":"2024-03-14T09:03:11.356757+0000","last_clean":"2024-03-14T09:03:11.356757+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:11.356757+0000","last_undegraded":"2024-03-14T09:03:11.356757+0000","last_fullsized":"2024-03-14T09:03:11.356757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353676+0000","last_change":"2024-03-14T09:03:12.353676+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353676+0000","last_undegraded":"2024-03-14T09:03:12.353676+0000","last_fullsized":"2024-03-14T09:03:12.353676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368922+0000","last_change":"2024-03-14T09:03:12.366770+0000","last_active":"2024-03-14T09:03:12.368922+0000","last_peered":"2024-03-14T09:03:12.368922+0000","last_clean":"2024-03-14T09:03:12.368922+0000","last_became_active":"2024-03-14T09:03:12.366022+0000","last_became_peered":"2024-03-14T09:03:12.366022+0000","last_unstale":"2024-03-14T09:03:12.368922+0000","last_undegraded":"2024-03-14T09:03:12.368922+0000","last_fullsized":"2024-03-14T09:03:12.368922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349099+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:11.349099+0000","last_peered":"2024-03-14T09:03:11.349099+0000","last_clean":"2024-03-14T09:03:11.349099+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:11.349099+0000","last_undegraded":"2024-03-14T09:03:11.349099+0000","last_fullsized":"2024-03-14T09:03:11.349099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354967+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:11.354967+0000","last_peered":"2024-03-14T09:03:11.354967+0000","last_clean":"2024-03-14T09:03:11.354967+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:11.354967+0000","last_undegraded":"2024-03-14T09:03:11.354967+0000","last_fullsized":"2024-03-14T09:03:11.354967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356783+0000","last_change":"2024-03-14T09:03:12.356783+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356783+0000","last_undegraded":"2024-03-14T09:03:12.356783+0000","last_fullsized":"2024-03-14T09:03:12.356783+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375064+0000","last_change":"2024-03-14T09:03:12.373601+0000","last_active":"2024-03-14T09:03:12.375064+0000","last_peered":"2024-03-14T09:03:12.375064+0000","last_clean":"2024-03-14T09:03:12.375064+0000","last_became_active":"2024-03-14T09:03:12.373332+0000","last_became_peered":"2024-03-14T09:03:12.373332+0000","last_unstale":"2024-03-14T09:03:12.375064+0000","last_undegraded":"2024-03-14T09:03:12.375064+0000","last_fullsized":"2024-03-14T09:03:12.375064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:38:05.459173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355546+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:11.355546+0000","last_peered":"2024-03-14T09:03:11.355546+0000","last_clean":"2024-03-14T09:03:11.355546+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:11.355546+0000","last_undegraded":"2024-03-14T09:03:11.355546+0000","last_fullsized":"2024-03-14T09:03:11.355546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358290+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:11.358290+0000","last_peered":"2024-03-14T09:03:11.358290+0000","last_clean":"2024-03-14T09:03:11.358290+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:11.358290+0000","last_undegraded":"2024-03-14T09:03:11.358290+0000","last_fullsized":"2024-03-14T09:03:11.358290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357797+0000","last_change":"2024-03-14T09:03:12.357797+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357797+0000","last_undegraded":"2024-03-14T09:03:12.357797+0000","last_fullsized":"2024-03-14T09:03:12.357797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375031+0000","last_change":"2024-03-14T09:03:12.373610+0000","last_active":"2024-03-14T09:03:12.375031+0000","last_peered":"2024-03-14T09:03:12.375031+0000","last_clean":"2024-03-14T09:03:12.375031+0000","last_became_active":"2024-03-14T09:03:12.373360+0000","last_became_peered":"2024-03-14T09:03:12.373360+0000","last_unstale":"2024-03-14T09:03:12.375031+0000","last_undegraded":"2024-03-14T09:03:12.375031+0000","last_fullsized":"2024-03-14T09:03:12.375031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:32.334273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355435+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:11.355435+0000","last_peered":"2024-03-14T09:03:11.355435+0000","last_clean":"2024-03-14T09:03:11.355435+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:11.355435+0000","last_undegraded":"2024-03-14T09:03:11.355435+0000","last_fullsized":"2024-03-14T09:03:11.355435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"51'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349459+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:11.349459+0000","last_peered":"2024-03-14T09:03:11.349459+0000","last_clean":"2024-03-14T09:03:11.349459+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:11.349459+0000","last_undegraded":"2024-03-14T09:03:11.349459+0000","last_fullsized":"2024-03-14T09:03:11.349459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356751+0000","last_change":"2024-03-14T09:03:12.356751+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356751+0000","last_undegraded":"2024-03-14T09:03:12.356751+0000","last_fullsized":"2024-03-14T09:03:12.356751+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367852+0000","last_change":"2024-03-14T09:03:12.365515+0000","last_active":"2024-03-14T09:03:12.367852+0000","last_peered":"2024-03-14T09:03:12.367852+0000","last_clean":"2024-03-14T09:03:12.367852+0000","last_became_active":"2024-03-14T09:03:12.364921+0000","last_became_peered":"2024-03-14T09:03:12.364921+0000","last_unstale":"2024-03-14T09:03:12.367852+0000","last_undegraded":"2024-03-14T09:03:12.367852+0000","last_fullsized":"2024-03-14T09:03:12.367852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357128+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:11.357128+0000","last_peered":"2024-03-14T09:03:11.357128+0000","last_clean":"2024-03-14T09:03:11.357128+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:11.357128+0000","last_undegraded":"2024-03-14T09:03:11.357128+0000","last_fullsized":"2024-03-14T09:03:11.357128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355195+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:11.355195+0000","last_peered":"2024-03-14T09:03:11.355195+0000","last_clean":"2024-03-14T09:03:11.355195+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:11.355195+0000","last_undegraded":"2024-03-14T09:03:11.355195+0000","last_fullsized":"2024-03-14T09:03:11.355195+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361654+0000","last_change":"2024-03-14T09:03:12.361654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361654+0000","last_undegraded":"2024-03-14T09:03:12.361654+0000","last_fullsized":"2024-03-14T09:03:12.361654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371874+0000","last_change":"2024-03-14T09:03:12.367975+0000","last_active":"2024-03-14T09:03:12.371874+0000","last_peered":"2024-03-14T09:03:12.371874+0000","last_clean":"2024-03-14T09:03:12.371874+0000","last_became_active":"2024-03-14T09:03:12.367702+0000","last_became_peered":"2024-03-14T09:03:12.367702+0000","last_unstale":"2024-03-14T09:03:12.371874+0000","last_undegraded":"2024-03-14T09:03:12.371874+0000","last_fullsized":"2024-03-14T09:03:12.371874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:12.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349479+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:11.349479+0000","last_peered":"2024-03-14T09:03:11.349479+0000","last_clean":"2024-03-14T09:03:11.349479+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:11.349479+0000","last_undegraded":"2024-03-14T09:03:11.349479+0000","last_fullsized":"2024-03-14T09:03:11.349479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355796+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:11.355796+0000","last_peered":"2024-03-14T09:03:11.355796+0000","last_clean":"2024-03-14T09:03:11.355796+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:11.355796+0000","last_undegraded":"2024-03-14T09:03:11.355796+0000","last_fullsized":"2024-03-14T09:03:11.355796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358131+0000","last_change":"2024-03-14T09:03:12.358131+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358131+0000","last_undegraded":"2024-03-14T09:03:12.358131+0000","last_fullsized":"2024-03-14T09:03:12.358131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376099+0000","last_change":"2024-03-14T09:03:12.371984+0000","last_active":"2024-03-14T09:03:12.376099+0000","last_peered":"2024-03-14T09:03:12.376099+0000","last_clean":"2024-03-14T09:03:12.376099+0000","last_became_active":"2024-03-14T09:03:12.371727+0000","last_became_peered":"2024-03-14T09:03:12.371727+0000","last_unstale":"2024-03-14T09:03:12.376099+0000","last_undegraded":"2024-03-14T09:03:12.376099+0000","last_fullsized":"2024-03-14T09:03:12.376099+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:23.845822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351395+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:11.351395+0000","last_peered":"2024-03-14T09:03:11.351395+0000","last_clean":"2024-03-14T09:03:11.351395+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:11.351395+0000","last_undegraded":"2024-03-14T09:03:11.351395+0000","last_fullsized":"2024-03-14T09:03:11.351395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356875+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:11.356875+0000","last_peered":"2024-03-14T09:03:11.356875+0000","last_clean":"2024-03-14T09:03:11.356875+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:11.356875+0000","last_undegraded":"2024-03-14T09:03:11.356875+0000","last_fullsized":"2024-03-14T09:03:11.356875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352756+0000","last_change":"2024-03-14T09:03:12.352756+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352756+0000","last_undegraded":"2024-03-14T09:03:12.352756+0000","last_fullsized":"2024-03-14T09:03:12.352756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.366806+0000","last_change":"2024-03-14T09:03:12.363729+0000","last_active":"2024-03-14T09:03:12.366806+0000","last_peered":"2024-03-14T09:03:12.366806+0000","last_clean":"2024-03-14T09:03:12.366806+0000","last_became_active":"2024-03-14T09:03:12.363440+0000","last_became_peered":"2024-03-14T09:03:12.363440+0000","last_unstale":"2024-03-14T09:03:12.366806+0000","last_undegraded":"2024-03-14T09:03:12.366806+0000","last_fullsized":"2024-03-14T09:03:12.366806+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:13.058703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358367+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:11.358367+0000","last_peered":"2024-03-14T09:03:11.358367+0000","last_clean":"2024-03-14T09:03:11.358367+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:11.358367+0000","last_undegraded":"2024-03-14T09:03:11.358367+0000","last_fullsized":"2024-03-14T09:03:11.358367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355086+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:11.355086+0000","last_peered":"2024-03-14T09:03:11.355086+0000","last_clean":"2024-03-14T09:03:11.355086+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:11.355086+0000","last_undegraded":"2024-03-14T09:03:11.355086+0000","last_fullsized":"2024-03-14T09:03:11.355086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359840+0000","last_change":"2024-03-14T09:03:12.359840+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359840+0000","last_undegraded":"2024-03-14T09:03:12.359840+0000","last_fullsized":"2024-03-14T09:03:12.359840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372010+0000","last_change":"2024-03-14T09:03:12.370434+0000","last_active":"2024-03-14T09:03:12.372010+0000","last_peered":"2024-03-14T09:03:12.372010+0000","last_clean":"2024-03-14T09:03:12.372010+0000","last_became_active":"2024-03-14T09:03:12.370170+0000","last_became_peered":"2024-03-14T09:03:12.370170+0000","last_unstale":"2024-03-14T09:03:12.372010+0000","last_undegraded":"2024-03-14T09:03:12.372010+0000","last_fullsized":"2024-03-14T09:03:12.372010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:55.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356464+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:11.356464+0000","last_peered":"2024-03-14T09:03:11.356464+0000","last_clean":"2024-03-14T09:03:11.356464+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:11.356464+0000","last_undegraded":"2024-03-14T09:03:11.356464+0000","last_fullsized":"2024-03-14T09:03:11.356464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356577+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:11.356577+0000","last_peered":"2024-03-14T09:03:11.356577+0000","last_clean":"2024-03-14T09:03:11.356577+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:11.356577+0000","last_undegraded":"2024-03-14T09:03:11.356577+0000","last_fullsized":"2024-03-14T09:03:11.356577+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353449+0000","last_change":"2024-03-14T09:03:12.353449+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353449+0000","last_undegraded":"2024-03-14T09:03:12.353449+0000","last_fullsized":"2024-03-14T09:03:12.353449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375525+0000","last_change":"2024-03-14T09:03:12.373132+0000","last_active":"2024-03-14T09:03:12.375525+0000","last_peered":"2024-03-14T09:03:12.375525+0000","last_clean":"2024-03-14T09:03:12.375525+0000","last_became_active":"2024-03-14T09:03:12.372802+0000","last_became_peered":"2024-03-14T09:03:12.372802+0000","last_unstale":"2024-03-14T09:03:12.375525+0000","last_undegraded":"2024-03-14T09:03:12.375525+0000","last_fullsized":"2024-03-14T09:03:12.375525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351200+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:11.351200+0000","last_peered":"2024-03-14T09:03:11.351200+0000","last_clean":"2024-03-14T09:03:11.351200+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:11.351200+0000","last_undegraded":"2024-03-14T09:03:11.351200+0000","last_fullsized":"2024-03-14T09:03:11.351200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351201+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:11.351201+0000","last_peered":"2024-03-14T09:03:11.351201+0000","last_clean":"2024-03-14T09:03:11.351201+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:11.351201+0000","last_undegraded":"2024-03-14T09:03:11.351201+0000","last_fullsized":"2024-03-14T09:03:11.351201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353238+0000","last_change":"2024-03-14T09:03:12.353238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353238+0000","last_undegraded":"2024-03-14T09:03:12.353238+0000","last_fullsized":"2024-03-14T09:03:12.353238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367162+0000","last_change":"2024-03-14T09:03:12.364039+0000","last_active":"2024-03-14T09:03:12.367162+0000","last_peered":"2024-03-14T09:03:12.367162+0000","last_clean":"2024-03-14T09:03:12.367162+0000","last_became_active":"2024-03-14T09:03:12.363778+0000","last_became_peered":"2024-03-14T09:03:12.363778+0000","last_unstale":"2024-03-14T09:03:12.367162+0000","last_undegraded":"2024-03-14T09:03:12.367162+0000","last_fullsized":"2024-03-14T09:03:12.367162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:53.168005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353295+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:11.353295+0000","last_peered":"2024-03-14T09:03:11.353295+0000","last_clean":"2024-03-14T09:03:11.353295+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:11.353295+0000","last_undegraded":"2024-03-14T09:03:11.353295+0000","last_fullsized":"2024-03-14T09:03:11.353295+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358124+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:11.358124+0000","last_peered":"2024-03-14T09:03:11.358124+0000","last_clean":"2024-03-14T09:03:11.358124+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:11.358124+0000","last_undegraded":"2024-03-14T09:03:11.358124+0000","last_fullsized":"2024-03-14T09:03:11.358124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1236992,"data_stored":861723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.623}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":271195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":264155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.37}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.318}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.621}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":858090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.372},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.316}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.44}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.504},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":271251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.254},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.435}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":274270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.625},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:17.287 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:17.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.441+0000 7fae398a0700 1 Processor -- start 2024-03-14T09:03:17.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.441+0000 7fae398a0700 1 -- start start 2024-03-14T09:03:17.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae398a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae3414e980 0x7fae341552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae398a0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3414ed60 con 0x7fae340c4a80 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae398a0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae34155820 con 0x7fae34158a80 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae398a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae340c6b10 con 0x7fae3414e980 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae327fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fae340c4a80 0x7fae34158370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35232/0 (socket says 172.21.15.154:35232) 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae327fc700 1 -- 172.21.15.154:0/2935372218 learned_addr learned my addr 172.21.15.154:0/2935372218 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae32ffd700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae3414e980 0x7fae341552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810095867 0 0) 0x7fae3414ed60 con 0x7fae340c4a80 2024-03-14T09:03:17.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae1c0035b0 con 0x7fae340c4a80 2024-03-14T09:03:17.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808920108 0 0) 0x7fae34155820 con 0x7fae34158a80 2024-03-14T09:03:17.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae3414ed60 con 0x7fae34158a80 2024-03-14T09:03:17.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3716593320 0 0) 0x7fae1c0035b0 con 0x7fae340c4a80 2024-03-14T09:03:17.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae3414e980 msgr2=0x7fae341552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae3414e980 0x7fae341552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 >> v1:172.21.15.191:6789/0 conn(0x7fae34158a80 legacy=0x7fae3415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340c6cf0 con 0x7fae340c4a80 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fae28001670 con 0x7fae340c4a80 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 >> v1:172.21.15.154:6789/0 conn(0x7fae340c4a80 legacy=0x7fae34158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae1c003bc0 0x7fae1c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae1c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae1c005210 0x7fae1c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340c6b10 con 0x7fae1c004850 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c00c0e0 con 0x7fae1c003bc0 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c00c2a0 con 0x7fae1c005210 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae32ffd700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae3414e980 0x7fae341552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.442+0000 7fae327fc700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae1c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae337fe700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae1c005210 0x7fae1c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae32ffd700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae1c003bc0 0x7fae1c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 -- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae1c005210 msgr2=0x7fae1c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae1c005210 0x7fae1c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 -- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae1c003bc0 msgr2=0x7fae1c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae1c003bc0 0x7fae1c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 -- 172.21.15.154:0/2935372218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0035b0 con 0x7fae1c004850 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae32ffd700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae1c003bc0 0x7fae1c003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae327fc700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae1c004cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c110 tx=0x7fae1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a8976f603c35af server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800ccb0 con 0x7fae1c004850 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2800ce10 con 0x7fae1c004850 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.443+0000 7fae31ffb700 1 -- 172.21.15.154:0/2935372218 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28005ba0 con 0x7fae1c004850 2024-03-14T09:03:17.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 -- 172.21.15.154:0/2935372218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 msgr2=0x7fae1c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 --2- 172.21.15.154:0/2935372218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae1c004cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c110 tx=0x7fae1c005880 comp rx=0 tx=0).stop 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae337fe700 1 -- 172.21.15.154:0/2935372218 reap_dead start 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 -- 172.21.15.154:0/2935372218 shutdown_connections 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 -- 172.21.15.154:0/2935372218 >> 172.21.15.154:0/2935372218 conn(0x7fae340abcf0 msgr2=0x7fae340ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 -- 172.21.15.154:0/2935372218 shutdown_connections 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.444+0000 7fae398a0700 1 -- 172.21.15.154:0/2935372218 wait complete. 2024-03-14T09:03:17.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.445+0000 7fae398a0700 1 Processor -- start 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.445+0000 7fae398a0700 1 -- start start 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae34152020 0x7fae341444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae34144a20 0x7fae341ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae341df620 con 0x7fae1c004850 2024-03-14T09:03:17.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae34147500 con 0x7fae34144a20 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae341e0d40 con 0x7fae34152020 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56354/0 (socket says 172.21.15.154:56354) 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 -- 172.21.15.154:0/3219929984 learned_addr learned my addr 172.21.15.154:0/3219929984 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae327fc700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae34152020 0x7fae341444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae337fe700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae34144a20 0x7fae341ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 -- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae34152020 msgr2=0x7fae341444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae34152020 0x7fae341444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 -- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae34144a20 msgr2=0x7fae341ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae34144a20 0x7fae341ddec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 -- 172.21.15.154:0/3219929984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34147050 con 0x7fae1c004850 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae32ffd700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fae240109e0 tx=0x7fae24010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400e870 con 0x7fae1c004850 2024-03-14T09:03:17.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400e9d0 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400ed40 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.446+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae341481f0 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.447+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae341479f0 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.447+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae340c5bc0 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.448+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fae24016460 con 0x7fae1c004850 2024-03-14T09:03:17.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.448+0000 7fae1bfff700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 0x7fae2006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.448+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fae24089820 con 0x7fae1c004850 2024-03-14T09:03:17.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.449+0000 7fae327fc700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 0x7fae2006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.449+0000 7fae327fc700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 0x7fae2006ccf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae28015040 tx=0x7fae28007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.453+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fae24054970 con 0x7fae1c004850 2024-03-14T09:03:17.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.644+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae340c7590 con 0x7fae2006a830 2024-03-14T09:03:17.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.659+0000 7fae1bfff700 1 -- 172.21.15.154:0/3219929984 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524924 (secure 0 0 0) 0x7fae28022070 con 0x7fae2006a830 2024-03-14T09:03:17.665 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 msgr2=0x7fae2006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 0x7fae2006ccf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae28015040 tx=0x7fae28007000 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 msgr2=0x7fae341e0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fae240109e0 tx=0x7fae24010da0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 shutdown_connections 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae34144a20 0x7fae341ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fae2006a830 0x7fae2006ccf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae1c004850 0x7fae341e0450 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 --2- 172.21.15.154:0/3219929984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae34152020 0x7fae341444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 >> 172.21.15.154:0/3219929984 conn(0x7fae340abcf0 msgr2=0x7fae3414ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.663+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 shutdown_connections 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.664+0000 7fae398a0700 1 -- 172.21.15.154:0/3219929984 wait complete. 2024-03-14T09:03:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:17.675 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-14T09:03:17.545379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222312,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752408,"statfs":{"total":773094113280,"available":772866465792,"internally_reserved":0,"allocated":6283264,"data_stored":3329522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001339"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353917+0000","last_change":"2024-03-14T09:03:12.353917+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353917+0000","last_undegraded":"2024-03-14T09:03:12.353917+0000","last_fullsized":"2024-03-14T09:03:12.353917+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353360+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:11.353360+0000","last_peered":"2024-03-14T09:03:11.353360+0000","last_clean":"2024-03-14T09:03:11.353360+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:11.353360+0000","last_undegraded":"2024-03-14T09:03:11.353360+0000","last_fullsized":"2024-03-14T09:03:11.353360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361638+0000","last_change":"2024-03-14T09:03:12.361638+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361638+0000","last_undegraded":"2024-03-14T09:03:12.361638+0000","last_fullsized":"2024-03-14T09:03:12.361638+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353404+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:11.353404+0000","last_peered":"2024-03-14T09:03:11.353404+0000","last_clean":"2024-03-14T09:03:11.353404+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:11.353404+0000","last_undegraded":"2024-03-14T09:03:11.353404+0000","last_fullsized":"2024-03-14T09:03:11.353404+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358323+0000","last_change":"2024-03-14T09:03:12.358323+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358323+0000","last_undegraded":"2024-03-14T09:03:12.358323+0000","last_fullsized":"2024-03-14T09:03:12.358323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358248+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:11.358248+0000","last_peered":"2024-03-14T09:03:11.358248+0000","last_clean":"2024-03-14T09:03:11.358248+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:11.358248+0000","last_undegraded":"2024-03-14T09:03:11.358248+0000","last_fullsized":"2024-03-14T09:03:11.358248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354698+0000","last_change":"2024-03-14T09:03:12.354698+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354698+0000","last_undegraded":"2024-03-14T09:03:12.354698+0000","last_fullsized":"2024-03-14T09:03:12.354698+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355314+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:11.355314+0000","last_peered":"2024-03-14T09:03:11.355314+0000","last_clean":"2024-03-14T09:03:11.355314+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:11.355314+0000","last_undegraded":"2024-03-14T09:03:11.355314+0000","last_fullsized":"2024-03-14T09:03:11.355314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362720+0000","last_change":"2024-03-14T09:03:12.362720+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362720+0000","last_undegraded":"2024-03-14T09:03:12.362720+0000","last_fullsized":"2024-03-14T09:03:12.362720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353370+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:11.353370+0000","last_peered":"2024-03-14T09:03:11.353370+0000","last_clean":"2024-03-14T09:03:11.353370+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:11.353370+0000","last_undegraded":"2024-03-14T09:03:11.353370+0000","last_fullsized":"2024-03-14T09:03:11.353370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353623+0000","last_change":"2024-03-14T09:03:12.353623+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353623+0000","last_undegraded":"2024-03-14T09:03:12.353623+0000","last_fullsized":"2024-03-14T09:03:12.353623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349110+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:11.349110+0000","last_peered":"2024-03-14T09:03:11.349110+0000","last_clean":"2024-03-14T09:03:11.349110+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:11.349110+0000","last_undegraded":"2024-03-14T09:03:11.349110+0000","last_fullsized":"2024-03-14T09:03:11.349110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360227+0000","last_change":"2024-03-14T09:03:12.360227+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360227+0000","last_undegraded":"2024-03-14T09:03:12.360227+0000","last_fullsized":"2024-03-14T09:03:12.360227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355387+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:11.355387+0000","last_peered":"2024-03-14T09:03:11.355387+0000","last_clean":"2024-03-14T09:03:11.355387+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:11.355387+0000","last_undegraded":"2024-03-14T09:03:11.355387+0000","last_fullsized":"2024-03-14T09:03:11.355387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353748+0000","last_change":"2024-03-14T09:03:12.353748+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353748+0000","last_undegraded":"2024-03-14T09:03:12.353748+0000","last_fullsized":"2024-03-14T09:03:12.353748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351097+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:11.351097+0000","last_peered":"2024-03-14T09:03:11.351097+0000","last_clean":"2024-03-14T09:03:11.351097+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:11.351097+0000","last_undegraded":"2024-03-14T09:03:11.351097+0000","last_fullsized":"2024-03-14T09:03:11.351097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357388+0000","last_change":"2024-03-14T09:03:12.357388+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357388+0000","last_undegraded":"2024-03-14T09:03:12.357388+0000","last_fullsized":"2024-03-14T09:03:12.357388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356978+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:11.356978+0000","last_peered":"2024-03-14T09:03:11.356978+0000","last_clean":"2024-03-14T09:03:11.356978+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:11.356978+0000","last_undegraded":"2024-03-14T09:03:11.356978+0000","last_fullsized":"2024-03-14T09:03:11.356978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362629+0000","last_change":"2024-03-14T09:03:12.362629+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362629+0000","last_undegraded":"2024-03-14T09:03:12.362629+0000","last_fullsized":"2024-03-14T09:03:12.362629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357377+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:11.357377+0000","last_peered":"2024-03-14T09:03:11.357377+0000","last_clean":"2024-03-14T09:03:11.357377+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:11.357377+0000","last_undegraded":"2024-03-14T09:03:11.357377+0000","last_fullsized":"2024-03-14T09:03:11.357377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359448+0000","last_change":"2024-03-14T09:03:12.359448+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359448+0000","last_undegraded":"2024-03-14T09:03:12.359448+0000","last_fullsized":"2024-03-14T09:03:12.359448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357336+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:11.357336+0000","last_peered":"2024-03-14T09:03:11.357336+0000","last_clean":"2024-03-14T09:03:11.357336+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:11.357336+0000","last_undegraded":"2024-03-14T09:03:11.357336+0000","last_fullsized":"2024-03-14T09:03:11.357336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364654+0000","last_change":"2024-03-14T09:03:12.364654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364654+0000","last_undegraded":"2024-03-14T09:03:12.364654+0000","last_fullsized":"2024-03-14T09:03:12.364654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355433+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:11.355433+0000","last_peered":"2024-03-14T09:03:11.355433+0000","last_clean":"2024-03-14T09:03:11.355433+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:11.355433+0000","last_undegraded":"2024-03-14T09:03:11.355433+0000","last_fullsized":"2024-03-14T09:03:11.355433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356123+0000","last_change":"2024-03-14T09:03:12.356123+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356123+0000","last_undegraded":"2024-03-14T09:03:12.356123+0000","last_fullsized":"2024-03-14T09:03:12.356123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353947+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:11.353947+0000","last_peered":"2024-03-14T09:03:11.353947+0000","last_clean":"2024-03-14T09:03:11.353947+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:11.353947+0000","last_undegraded":"2024-03-14T09:03:11.353947+0000","last_fullsized":"2024-03-14T09:03:11.353947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361949+0000","last_change":"2024-03-14T09:03:12.361949+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361949+0000","last_undegraded":"2024-03-14T09:03:12.361949+0000","last_fullsized":"2024-03-14T09:03:12.361949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355064+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:11.355064+0000","last_peered":"2024-03-14T09:03:11.355064+0000","last_clean":"2024-03-14T09:03:11.355064+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:11.355064+0000","last_undegraded":"2024-03-14T09:03:11.355064+0000","last_fullsized":"2024-03-14T09:03:11.355064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355497+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:11.355497+0000","last_peered":"2024-03-14T09:03:11.355497+0000","last_clean":"2024-03-14T09:03:11.355497+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:11.355497+0000","last_undegraded":"2024-03-14T09:03:11.355497+0000","last_fullsized":"2024-03-14T09:03:11.355497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364020+0000","last_change":"2024-03-14T09:03:12.364020+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364020+0000","last_undegraded":"2024-03-14T09:03:12.364020+0000","last_fullsized":"2024-03-14T09:03:12.364020+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.348935+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:11.348935+0000","last_peered":"2024-03-14T09:03:11.348935+0000","last_clean":"2024-03-14T09:03:11.348935+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:11.348935+0000","last_undegraded":"2024-03-14T09:03:11.348935+0000","last_fullsized":"2024-03-14T09:03:11.348935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353377+0000","last_change":"2024-03-14T09:03:12.353377+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353377+0000","last_undegraded":"2024-03-14T09:03:12.353377+0000","last_fullsized":"2024-03-14T09:03:12.353377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353394+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:11.353394+0000","last_peered":"2024-03-14T09:03:11.353394+0000","last_clean":"2024-03-14T09:03:11.353394+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:11.353394+0000","last_undegraded":"2024-03-14T09:03:11.353394+0000","last_fullsized":"2024-03-14T09:03:11.353394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356427+0000","last_change":"2024-03-14T09:03:12.356427+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356427+0000","last_undegraded":"2024-03-14T09:03:12.356427+0000","last_fullsized":"2024-03-14T09:03:12.356427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358442+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:11.358442+0000","last_peered":"2024-03-14T09:03:11.358442+0000","last_clean":"2024-03-14T09:03:11.358442+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:11.358442+0000","last_undegraded":"2024-03-14T09:03:11.358442+0000","last_fullsized":"2024-03-14T09:03:11.358442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360390+0000","last_change":"2024-03-14T09:03:12.360390+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360390+0000","last_undegraded":"2024-03-14T09:03:12.360390+0000","last_fullsized":"2024-03-14T09:03:12.360390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353260+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:11.353260+0000","last_peered":"2024-03-14T09:03:11.353260+0000","last_clean":"2024-03-14T09:03:11.353260+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:11.353260+0000","last_undegraded":"2024-03-14T09:03:11.353260+0000","last_fullsized":"2024-03-14T09:03:11.353260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359276+0000","last_change":"2024-03-14T09:03:12.359276+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359276+0000","last_undegraded":"2024-03-14T09:03:12.359276+0000","last_fullsized":"2024-03-14T09:03:12.359276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351650+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:11.351650+0000","last_peered":"2024-03-14T09:03:11.351650+0000","last_clean":"2024-03-14T09:03:11.351650+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:11.351650+0000","last_undegraded":"2024-03-14T09:03:11.351650+0000","last_fullsized":"2024-03-14T09:03:11.351650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356216+0000","last_change":"2024-03-14T09:03:12.356216+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356216+0000","last_undegraded":"2024-03-14T09:03:12.356216+0000","last_fullsized":"2024-03-14T09:03:12.356216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358150+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:11.358150+0000","last_peered":"2024-03-14T09:03:11.358150+0000","last_clean":"2024-03-14T09:03:11.358150+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:11.358150+0000","last_undegraded":"2024-03-14T09:03:11.358150+0000","last_fullsized":"2024-03-14T09:03:11.358150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357282+0000","last_change":"2024-03-14T09:03:12.357282+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357282+0000","last_undegraded":"2024-03-14T09:03:12.357282+0000","last_fullsized":"2024-03-14T09:03:12.357282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351242+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:11.351242+0000","last_peered":"2024-03-14T09:03:11.351242+0000","last_clean":"2024-03-14T09:03:11.351242+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:11.351242+0000","last_undegraded":"2024-03-14T09:03:11.351242+0000","last_fullsized":"2024-03-14T09:03:11.351242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364628+0000","last_change":"2024-03-14T09:03:12.364628+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364628+0000","last_undegraded":"2024-03-14T09:03:12.364628+0000","last_fullsized":"2024-03-14T09:03:12.364628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358083+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:11.358083+0000","last_peered":"2024-03-14T09:03:11.358083+0000","last_clean":"2024-03-14T09:03:11.358083+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:11.358083+0000","last_undegraded":"2024-03-14T09:03:11.358083+0000","last_fullsized":"2024-03-14T09:03:11.358083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362012+0000","last_change":"2024-03-14T09:03:12.362012+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362012+0000","last_undegraded":"2024-03-14T09:03:12.362012+0000","last_fullsized":"2024-03-14T09:03:12.362012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353228+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.353228+0000","last_peered":"2024-03-14T09:03:11.353228+0000","last_clean":"2024-03-14T09:03:11.353228+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:11.353228+0000","last_undegraded":"2024-03-14T09:03:11.353228+0000","last_fullsized":"2024-03-14T09:03:11.353228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352812+0000","last_change":"2024-03-14T09:03:12.352812+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352812+0000","last_undegraded":"2024-03-14T09:03:12.352812+0000","last_fullsized":"2024-03-14T09:03:12.352812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353434+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:11.353434+0000","last_peered":"2024-03-14T09:03:11.353434+0000","last_clean":"2024-03-14T09:03:11.353434+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:11.353434+0000","last_undegraded":"2024-03-14T09:03:11.353434+0000","last_fullsized":"2024-03-14T09:03:11.353434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361474+0000","last_change":"2024-03-14T09:03:12.361474+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361474+0000","last_undegraded":"2024-03-14T09:03:12.361474+0000","last_fullsized":"2024-03-14T09:03:12.361474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355459+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:11.355459+0000","last_peered":"2024-03-14T09:03:11.355459+0000","last_clean":"2024-03-14T09:03:11.355459+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:11.355459+0000","last_undegraded":"2024-03-14T09:03:11.355459+0000","last_fullsized":"2024-03-14T09:03:11.355459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361988+0000","last_change":"2024-03-14T09:03:12.361988+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361988+0000","last_undegraded":"2024-03-14T09:03:12.361988+0000","last_fullsized":"2024-03-14T09:03:12.361988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362109+0000","last_change":"2024-03-14T09:03:12.362109+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362109+0000","last_undegraded":"2024-03-14T09:03:12.362109+0000","last_fullsized":"2024-03-14T09:03:12.362109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353264+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:11.353264+0000","last_peered":"2024-03-14T09:03:11.353264+0000","last_clean":"2024-03-14T09:03:11.353264+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:11.353264+0000","last_undegraded":"2024-03-14T09:03:11.353264+0000","last_fullsized":"2024-03-14T09:03:11.353264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362252+0000","last_change":"2024-03-14T09:03:12.362252+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362252+0000","last_undegraded":"2024-03-14T09:03:12.362252+0000","last_fullsized":"2024-03-14T09:03:12.362252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351526+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:11.351526+0000","last_peered":"2024-03-14T09:03:11.351526+0000","last_clean":"2024-03-14T09:03:11.351526+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:11.351526+0000","last_undegraded":"2024-03-14T09:03:11.351526+0000","last_fullsized":"2024-03-14T09:03:11.351526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353125+0000","last_change":"2024-03-14T09:03:12.353125+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353125+0000","last_undegraded":"2024-03-14T09:03:12.353125+0000","last_fullsized":"2024-03-14T09:03:12.353125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358400+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:11.358400+0000","last_peered":"2024-03-14T09:03:11.358400+0000","last_clean":"2024-03-14T09:03:11.358400+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:11.358400+0000","last_undegraded":"2024-03-14T09:03:11.358400+0000","last_fullsized":"2024-03-14T09:03:11.358400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354173+0000","last_change":"2024-03-14T09:03:12.354173+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354173+0000","last_undegraded":"2024-03-14T09:03:12.354173+0000","last_fullsized":"2024-03-14T09:03:12.354173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.352840+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:11.352840+0000","last_peered":"2024-03-14T09:03:11.352840+0000","last_clean":"2024-03-14T09:03:11.352840+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:11.352840+0000","last_undegraded":"2024-03-14T09:03:11.352840+0000","last_fullsized":"2024-03-14T09:03:11.352840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361509+0000","last_change":"2024-03-14T09:03:12.361509+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361509+0000","last_undegraded":"2024-03-14T09:03:12.361509+0000","last_fullsized":"2024-03-14T09:03:12.361509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349002+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:11.349002+0000","last_peered":"2024-03-14T09:03:11.349002+0000","last_clean":"2024-03-14T09:03:11.349002+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:11.349002+0000","last_undegraded":"2024-03-14T09:03:11.349002+0000","last_fullsized":"2024-03-14T09:03:11.349002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354070+0000","last_change":"2024-03-14T09:03:12.354070+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354070+0000","last_undegraded":"2024-03-14T09:03:12.354070+0000","last_fullsized":"2024-03-14T09:03:12.354070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376057+0000","last_change":"2024-03-14T09:03:12.371272+0000","last_active":"2024-03-14T09:03:12.376057+0000","last_peered":"2024-03-14T09:03:12.376057+0000","last_clean":"2024-03-14T09:03:12.376057+0000","last_became_active":"2024-03-14T09:03:12.370996+0000","last_became_peered":"2024-03-14T09:03:12.370996+0000","last_unstale":"2024-03-14T09:03:12.376057+0000","last_undegraded":"2024-03-14T09:03:12.376057+0000","last_fullsized":"2024-03-14T09:03:12.376057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:44.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349268+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:11.349268+0000","last_peered":"2024-03-14T09:03:11.349268+0000","last_clean":"2024-03-14T09:03:11.349268+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:11.349268+0000","last_undegraded":"2024-03-14T09:03:11.349268+0000","last_fullsized":"2024-03-14T09:03:11.349268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349423+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:11.349423+0000","last_peered":"2024-03-14T09:03:11.349423+0000","last_clean":"2024-03-14T09:03:11.349423+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:11.349423+0000","last_undegraded":"2024-03-14T09:03:11.349423+0000","last_fullsized":"2024-03-14T09:03:11.349423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356924+0000","last_change":"2024-03-14T09:03:12.356924+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356924+0000","last_undegraded":"2024-03-14T09:03:12.356924+0000","last_fullsized":"2024-03-14T09:03:12.356924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378773+0000","last_change":"2024-03-14T09:03:12.374278+0000","last_active":"2024-03-14T09:03:12.378773+0000","last_peered":"2024-03-14T09:03:12.378773+0000","last_clean":"2024-03-14T09:03:12.378773+0000","last_became_active":"2024-03-14T09:03:12.373968+0000","last_became_peered":"2024-03-14T09:03:12.373968+0000","last_unstale":"2024-03-14T09:03:12.378773+0000","last_undegraded":"2024-03-14T09:03:12.378773+0000","last_fullsized":"2024-03-14T09:03:12.378773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:40.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358207+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:11.358207+0000","last_peered":"2024-03-14T09:03:11.358207+0000","last_clean":"2024-03-14T09:03:11.358207+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:11.358207+0000","last_undegraded":"2024-03-14T09:03:11.358207+0000","last_fullsized":"2024-03-14T09:03:11.358207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353512+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:11.353512+0000","last_peered":"2024-03-14T09:03:11.353512+0000","last_clean":"2024-03-14T09:03:11.353512+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:11.353512+0000","last_undegraded":"2024-03-14T09:03:11.353512+0000","last_fullsized":"2024-03-14T09:03:11.353512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377433+0000","last_change":"2024-03-14T09:03:12.373911+0000","last_active":"2024-03-14T09:03:12.377433+0000","last_peered":"2024-03-14T09:03:12.377433+0000","last_clean":"2024-03-14T09:03:12.377433+0000","last_became_active":"2024-03-14T09:03:12.373633+0000","last_became_peered":"2024-03-14T09:03:12.373633+0000","last_unstale":"2024-03-14T09:03:12.377433+0000","last_undegraded":"2024-03-14T09:03:12.377433+0000","last_fullsized":"2024-03-14T09:03:12.377433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349440+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:11.349440+0000","last_peered":"2024-03-14T09:03:11.349440+0000","last_clean":"2024-03-14T09:03:11.349440+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:11.349440+0000","last_undegraded":"2024-03-14T09:03:11.349440+0000","last_fullsized":"2024-03-14T09:03:11.349440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359921+0000","last_change":"2024-03-14T09:03:12.359921+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359921+0000","last_undegraded":"2024-03-14T09:03:12.359921+0000","last_fullsized":"2024-03-14T09:03:12.359921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371201+0000","last_change":"2024-03-14T09:03:12.370015+0000","last_active":"2024-03-14T09:03:12.371201+0000","last_peered":"2024-03-14T09:03:12.371201+0000","last_clean":"2024-03-14T09:03:12.371201+0000","last_became_active":"2024-03-14T09:03:12.369735+0000","last_became_peered":"2024-03-14T09:03:12.369735+0000","last_unstale":"2024-03-14T09:03:12.371201+0000","last_undegraded":"2024-03-14T09:03:12.371201+0000","last_fullsized":"2024-03-14T09:03:12.371201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349286+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.349286+0000","last_peered":"2024-03-14T09:03:11.349286+0000","last_clean":"2024-03-14T09:03:11.349286+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:11.349286+0000","last_undegraded":"2024-03-14T09:03:11.349286+0000","last_fullsized":"2024-03-14T09:03:11.349286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355554+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:11.355554+0000","last_peered":"2024-03-14T09:03:11.355554+0000","last_clean":"2024-03-14T09:03:11.355554+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:11.355554+0000","last_undegraded":"2024-03-14T09:03:11.355554+0000","last_fullsized":"2024-03-14T09:03:11.355554+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360660+0000","last_change":"2024-03-14T09:03:12.360660+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360660+0000","last_undegraded":"2024-03-14T09:03:12.360660+0000","last_fullsized":"2024-03-14T09:03:12.360660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375005+0000","last_change":"2024-03-14T09:03:12.372188+0000","last_active":"2024-03-14T09:03:12.375005+0000","last_peered":"2024-03-14T09:03:12.375005+0000","last_clean":"2024-03-14T09:03:12.375005+0000","last_became_active":"2024-03-14T09:03:12.371919+0000","last_became_peered":"2024-03-14T09:03:12.371919+0000","last_unstale":"2024-03-14T09:03:12.375005+0000","last_undegraded":"2024-03-14T09:03:12.375005+0000","last_fullsized":"2024-03-14T09:03:12.375005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351321+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:11.351321+0000","last_peered":"2024-03-14T09:03:11.351321+0000","last_clean":"2024-03-14T09:03:11.351321+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:11.351321+0000","last_undegraded":"2024-03-14T09:03:11.351321+0000","last_fullsized":"2024-03-14T09:03:11.351321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355947+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:11.355947+0000","last_peered":"2024-03-14T09:03:11.355947+0000","last_clean":"2024-03-14T09:03:11.355947+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:11.355947+0000","last_undegraded":"2024-03-14T09:03:11.355947+0000","last_fullsized":"2024-03-14T09:03:11.355947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355339+0000","last_change":"2024-03-14T09:03:12.355339+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355339+0000","last_undegraded":"2024-03-14T09:03:12.355339+0000","last_fullsized":"2024-03-14T09:03:12.355339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372079+0000","last_change":"2024-03-14T09:03:12.369918+0000","last_active":"2024-03-14T09:03:12.372079+0000","last_peered":"2024-03-14T09:03:12.372079+0000","last_clean":"2024-03-14T09:03:12.372079+0000","last_became_active":"2024-03-14T09:03:12.369599+0000","last_became_peered":"2024-03-14T09:03:12.369599+0000","last_unstale":"2024-03-14T09:03:12.372079+0000","last_undegraded":"2024-03-14T09:03:12.372079+0000","last_fullsized":"2024-03-14T09:03:12.372079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:52.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351535+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:11.351535+0000","last_peered":"2024-03-14T09:03:11.351535+0000","last_clean":"2024-03-14T09:03:11.351535+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:11.351535+0000","last_undegraded":"2024-03-14T09:03:11.351535+0000","last_fullsized":"2024-03-14T09:03:11.351535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349367+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:11.349367+0000","last_peered":"2024-03-14T09:03:11.349367+0000","last_clean":"2024-03-14T09:03:11.349367+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:11.349367+0000","last_undegraded":"2024-03-14T09:03:11.349367+0000","last_fullsized":"2024-03-14T09:03:11.349367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361025+0000","last_change":"2024-03-14T09:03:12.361025+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361025+0000","last_undegraded":"2024-03-14T09:03:12.361025+0000","last_fullsized":"2024-03-14T09:03:12.361025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374895+0000","last_change":"2024-03-14T09:03:12.373841+0000","last_active":"2024-03-14T09:03:12.374895+0000","last_peered":"2024-03-14T09:03:12.374895+0000","last_clean":"2024-03-14T09:03:12.374895+0000","last_became_active":"2024-03-14T09:03:12.373551+0000","last_became_peered":"2024-03-14T09:03:12.373551+0000","last_unstale":"2024-03-14T09:03:12.374895+0000","last_undegraded":"2024-03-14T09:03:12.374895+0000","last_fullsized":"2024-03-14T09:03:12.374895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.965540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353268+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:11.353268+0000","last_peered":"2024-03-14T09:03:11.353268+0000","last_clean":"2024-03-14T09:03:11.353268+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:11.353268+0000","last_undegraded":"2024-03-14T09:03:11.353268+0000","last_fullsized":"2024-03-14T09:03:11.353268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355710+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:11.355710+0000","last_peered":"2024-03-14T09:03:11.355710+0000","last_clean":"2024-03-14T09:03:11.355710+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:11.355710+0000","last_undegraded":"2024-03-14T09:03:11.355710+0000","last_fullsized":"2024-03-14T09:03:11.355710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361000+0000","last_change":"2024-03-14T09:03:12.361000+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361000+0000","last_undegraded":"2024-03-14T09:03:12.361000+0000","last_fullsized":"2024-03-14T09:03:12.361000+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368944+0000","last_change":"2024-03-14T09:03:12.366900+0000","last_active":"2024-03-14T09:03:12.368944+0000","last_peered":"2024-03-14T09:03:12.368944+0000","last_clean":"2024-03-14T09:03:12.368944+0000","last_became_active":"2024-03-14T09:03:12.366301+0000","last_became_peered":"2024-03-14T09:03:12.366301+0000","last_unstale":"2024-03-14T09:03:12.368944+0000","last_undegraded":"2024-03-14T09:03:12.368944+0000","last_fullsized":"2024-03-14T09:03:12.368944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:11.884116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354744+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:11.354744+0000","last_peered":"2024-03-14T09:03:11.354744+0000","last_clean":"2024-03-14T09:03:11.354744+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:11.354744+0000","last_undegraded":"2024-03-14T09:03:11.354744+0000","last_fullsized":"2024-03-14T09:03:11.354744+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353503+0000","last_change":"2024-03-14T09:03:12.353503+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353503+0000","last_undegraded":"2024-03-14T09:03:12.353503+0000","last_fullsized":"2024-03-14T09:03:12.353503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371091+0000","last_change":"2024-03-14T09:03:12.367305+0000","last_active":"2024-03-14T09:03:12.371091+0000","last_peered":"2024-03-14T09:03:12.371091+0000","last_clean":"2024-03-14T09:03:12.371091+0000","last_became_active":"2024-03-14T09:03:12.367020+0000","last_became_peered":"2024-03-14T09:03:12.367020+0000","last_unstale":"2024-03-14T09:03:12.371091+0000","last_undegraded":"2024-03-14T09:03:12.371091+0000","last_fullsized":"2024-03-14T09:03:12.371091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:39.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356736+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:11.356736+0000","last_peered":"2024-03-14T09:03:11.356736+0000","last_clean":"2024-03-14T09:03:11.356736+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:11.356736+0000","last_undegraded":"2024-03-14T09:03:11.356736+0000","last_fullsized":"2024-03-14T09:03:11.356736+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358626+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:11.358626+0000","last_peered":"2024-03-14T09:03:11.358626+0000","last_clean":"2024-03-14T09:03:11.358626+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:11.358626+0000","last_undegraded":"2024-03-14T09:03:11.358626+0000","last_fullsized":"2024-03-14T09:03:11.358626+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364515+0000","last_change":"2024-03-14T09:03:12.364515+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364515+0000","last_undegraded":"2024-03-14T09:03:12.364515+0000","last_fullsized":"2024-03-14T09:03:12.364515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373487+0000","last_change":"2024-03-14T09:03:12.370627+0000","last_active":"2024-03-14T09:03:12.373487+0000","last_peered":"2024-03-14T09:03:12.373487+0000","last_clean":"2024-03-14T09:03:12.373487+0000","last_became_active":"2024-03-14T09:03:12.370314+0000","last_became_peered":"2024-03-14T09:03:12.370314+0000","last_unstale":"2024-03-14T09:03:12.373487+0000","last_undegraded":"2024-03-14T09:03:12.373487+0000","last_fullsized":"2024-03-14T09:03:12.373487+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:09.613459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358558+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:11.358558+0000","last_peered":"2024-03-14T09:03:11.358558+0000","last_clean":"2024-03-14T09:03:11.358558+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:11.358558+0000","last_undegraded":"2024-03-14T09:03:11.358558+0000","last_fullsized":"2024-03-14T09:03:11.358558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353599+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:11.353599+0000","last_peered":"2024-03-14T09:03:11.353599+0000","last_clean":"2024-03-14T09:03:11.353599+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:11.353599+0000","last_undegraded":"2024-03-14T09:03:11.353599+0000","last_fullsized":"2024-03-14T09:03:11.353599+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359668+0000","last_change":"2024-03-14T09:03:12.359668+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359668+0000","last_undegraded":"2024-03-14T09:03:12.359668+0000","last_fullsized":"2024-03-14T09:03:12.359668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378802+0000","last_change":"2024-03-14T09:03:12.374028+0000","last_active":"2024-03-14T09:03:12.378802+0000","last_peered":"2024-03-14T09:03:12.378802+0000","last_clean":"2024-03-14T09:03:12.378802+0000","last_became_active":"2024-03-14T09:03:12.373770+0000","last_became_peered":"2024-03-14T09:03:12.373770+0000","last_unstale":"2024-03-14T09:03:12.378802+0000","last_undegraded":"2024-03-14T09:03:12.378802+0000","last_fullsized":"2024-03-14T09:03:12.378802+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:51.114973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357121+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:11.357121+0000","last_peered":"2024-03-14T09:03:11.357121+0000","last_clean":"2024-03-14T09:03:11.357121+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:11.357121+0000","last_undegraded":"2024-03-14T09:03:11.357121+0000","last_fullsized":"2024-03-14T09:03:11.357121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358587+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:11.358587+0000","last_peered":"2024-03-14T09:03:11.358587+0000","last_clean":"2024-03-14T09:03:11.358587+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:11.358587+0000","last_undegraded":"2024-03-14T09:03:11.358587+0000","last_fullsized":"2024-03-14T09:03:11.358587+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357946+0000","last_change":"2024-03-14T09:03:12.357946+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357946+0000","last_undegraded":"2024-03-14T09:03:12.357946+0000","last_fullsized":"2024-03-14T09:03:12.357946+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368918+0000","last_change":"2024-03-14T09:03:12.367742+0000","last_active":"2024-03-14T09:03:12.368918+0000","last_peered":"2024-03-14T09:03:12.368918+0000","last_clean":"2024-03-14T09:03:12.368918+0000","last_became_active":"2024-03-14T09:03:12.367471+0000","last_became_peered":"2024-03-14T09:03:12.367471+0000","last_unstale":"2024-03-14T09:03:12.368918+0000","last_undegraded":"2024-03-14T09:03:12.368918+0000","last_fullsized":"2024-03-14T09:03:12.368918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:34.678842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358346+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:11.358346+0000","last_peered":"2024-03-14T09:03:11.358346+0000","last_clean":"2024-03-14T09:03:11.358346+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:11.358346+0000","last_undegraded":"2024-03-14T09:03:11.358346+0000","last_fullsized":"2024-03-14T09:03:11.358346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356446+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:11.356446+0000","last_peered":"2024-03-14T09:03:11.356446+0000","last_clean":"2024-03-14T09:03:11.356446+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:11.356446+0000","last_undegraded":"2024-03-14T09:03:11.356446+0000","last_fullsized":"2024-03-14T09:03:11.356446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355535+0000","last_change":"2024-03-14T09:03:12.355535+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355535+0000","last_undegraded":"2024-03-14T09:03:12.355535+0000","last_fullsized":"2024-03-14T09:03:12.355535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371980+0000","last_change":"2024-03-14T09:03:12.366980+0000","last_active":"2024-03-14T09:03:12.371980+0000","last_peered":"2024-03-14T09:03:12.371980+0000","last_clean":"2024-03-14T09:03:12.371980+0000","last_became_active":"2024-03-14T09:03:12.366728+0000","last_became_peered":"2024-03-14T09:03:12.366728+0000","last_unstale":"2024-03-14T09:03:12.371980+0000","last_undegraded":"2024-03-14T09:03:12.371980+0000","last_fullsized":"2024-03-14T09:03:12.371980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:53.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355135+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:11.355135+0000","last_peered":"2024-03-14T09:03:11.355135+0000","last_clean":"2024-03-14T09:03:11.355135+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:11.355135+0000","last_undegraded":"2024-03-14T09:03:11.355135+0000","last_fullsized":"2024-03-14T09:03:11.355135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351630+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:11.351630+0000","last_peered":"2024-03-14T09:03:11.351630+0000","last_clean":"2024-03-14T09:03:11.351630+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:11.351630+0000","last_undegraded":"2024-03-14T09:03:11.351630+0000","last_fullsized":"2024-03-14T09:03:11.351630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358690+0000","last_change":"2024-03-14T09:03:12.358690+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358690+0000","last_undegraded":"2024-03-14T09:03:12.358690+0000","last_fullsized":"2024-03-14T09:03:12.358690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376094+0000","last_change":"2024-03-14T09:03:12.371369+0000","last_active":"2024-03-14T09:03:12.376094+0000","last_peered":"2024-03-14T09:03:12.376094+0000","last_clean":"2024-03-14T09:03:12.376094+0000","last_became_active":"2024-03-14T09:03:12.371087+0000","last_became_peered":"2024-03-14T09:03:12.371087+0000","last_unstale":"2024-03-14T09:03:12.376094+0000","last_undegraded":"2024-03-14T09:03:12.376094+0000","last_fullsized":"2024-03-14T09:03:12.376094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:38.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358447+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:11.358447+0000","last_peered":"2024-03-14T09:03:11.358447+0000","last_clean":"2024-03-14T09:03:11.358447+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:11.358447+0000","last_undegraded":"2024-03-14T09:03:11.358447+0000","last_fullsized":"2024-03-14T09:03:11.358447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354648+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:11.354648+0000","last_peered":"2024-03-14T09:03:11.354648+0000","last_clean":"2024-03-14T09:03:11.354648+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:11.354648+0000","last_undegraded":"2024-03-14T09:03:11.354648+0000","last_fullsized":"2024-03-14T09:03:11.354648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356321+0000","last_change":"2024-03-14T09:03:12.356321+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356321+0000","last_undegraded":"2024-03-14T09:03:12.356321+0000","last_fullsized":"2024-03-14T09:03:12.356321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373386+0000","last_change":"2024-03-14T09:03:12.368944+0000","last_active":"2024-03-14T09:03:12.373386+0000","last_peered":"2024-03-14T09:03:12.373386+0000","last_clean":"2024-03-14T09:03:12.373386+0000","last_became_active":"2024-03-14T09:03:12.368631+0000","last_became_peered":"2024-03-14T09:03:12.368631+0000","last_unstale":"2024-03-14T09:03:12.373386+0000","last_undegraded":"2024-03-14T09:03:12.373386+0000","last_fullsized":"2024-03-14T09:03:12.373386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354667+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:11.354667+0000","last_peered":"2024-03-14T09:03:11.354667+0000","last_clean":"2024-03-14T09:03:11.354667+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:11.354667+0000","last_undegraded":"2024-03-14T09:03:11.354667+0000","last_fullsized":"2024-03-14T09:03:11.354667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355806+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:11.355806+0000","last_peered":"2024-03-14T09:03:11.355806+0000","last_clean":"2024-03-14T09:03:11.355806+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:11.355806+0000","last_undegraded":"2024-03-14T09:03:11.355806+0000","last_fullsized":"2024-03-14T09:03:11.355806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360063+0000","last_change":"2024-03-14T09:03:12.360063+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360063+0000","last_undegraded":"2024-03-14T09:03:12.360063+0000","last_fullsized":"2024-03-14T09:03:12.360063+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371484+0000","last_change":"2024-03-14T09:03:12.370316+0000","last_active":"2024-03-14T09:03:12.371484+0000","last_peered":"2024-03-14T09:03:12.371484+0000","last_clean":"2024-03-14T09:03:12.371484+0000","last_became_active":"2024-03-14T09:03:12.370035+0000","last_became_peered":"2024-03-14T09:03:12.370035+0000","last_unstale":"2024-03-14T09:03:12.371484+0000","last_undegraded":"2024-03-14T09:03:12.371484+0000","last_fullsized":"2024-03-14T09:03:12.371484+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:07.312244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358188+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:11.358188+0000","last_peered":"2024-03-14T09:03:11.358188+0000","last_clean":"2024-03-14T09:03:11.358188+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:11.358188+0000","last_undegraded":"2024-03-14T09:03:11.358188+0000","last_fullsized":"2024-03-14T09:03:11.358188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358017+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:11.358017+0000","last_peered":"2024-03-14T09:03:11.358017+0000","last_clean":"2024-03-14T09:03:11.358017+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:11.358017+0000","last_undegraded":"2024-03-14T09:03:11.358017+0000","last_fullsized":"2024-03-14T09:03:11.358017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357489+0000","last_change":"2024-03-14T09:03:12.357489+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357489+0000","last_undegraded":"2024-03-14T09:03:12.357489+0000","last_fullsized":"2024-03-14T09:03:12.357489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377230+0000","last_change":"2024-03-14T09:03:12.373196+0000","last_active":"2024-03-14T09:03:12.377230+0000","last_peered":"2024-03-14T09:03:12.377230+0000","last_clean":"2024-03-14T09:03:12.377230+0000","last_became_active":"2024-03-14T09:03:12.372917+0000","last_became_peered":"2024-03-14T09:03:12.372917+0000","last_unstale":"2024-03-14T09:03:12.377230+0000","last_undegraded":"2024-03-14T09:03:12.377230+0000","last_fullsized":"2024-03-14T09:03:12.377230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:56.766593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358565+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:11.358565+0000","last_peered":"2024-03-14T09:03:11.358565+0000","last_clean":"2024-03-14T09:03:11.358565+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:11.358565+0000","last_undegraded":"2024-03-14T09:03:11.358565+0000","last_fullsized":"2024-03-14T09:03:11.358565+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353682+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:11.353682+0000","last_peered":"2024-03-14T09:03:11.353682+0000","last_clean":"2024-03-14T09:03:11.353682+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:11.353682+0000","last_undegraded":"2024-03-14T09:03:11.353682+0000","last_fullsized":"2024-03-14T09:03:11.353682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359244+0000","last_change":"2024-03-14T09:03:12.359244+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359244+0000","last_undegraded":"2024-03-14T09:03:12.359244+0000","last_fullsized":"2024-03-14T09:03:12.359244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377421+0000","last_change":"2024-03-14T09:03:12.373227+0000","last_active":"2024-03-14T09:03:12.377421+0000","last_peered":"2024-03-14T09:03:12.377421+0000","last_clean":"2024-03-14T09:03:12.377421+0000","last_became_active":"2024-03-14T09:03:12.372761+0000","last_became_peered":"2024-03-14T09:03:12.372761+0000","last_unstale":"2024-03-14T09:03:12.377421+0000","last_undegraded":"2024-03-14T09:03:12.377421+0000","last_fullsized":"2024-03-14T09:03:12.377421+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355667+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:11.355667+0000","last_peered":"2024-03-14T09:03:11.355667+0000","last_clean":"2024-03-14T09:03:11.355667+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:11.355667+0000","last_undegraded":"2024-03-14T09:03:11.355667+0000","last_fullsized":"2024-03-14T09:03:11.355667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349366+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:11.349366+0000","last_peered":"2024-03-14T09:03:11.349366+0000","last_clean":"2024-03-14T09:03:11.349366+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:11.349366+0000","last_undegraded":"2024-03-14T09:03:11.349366+0000","last_fullsized":"2024-03-14T09:03:11.349366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364238+0000","last_change":"2024-03-14T09:03:12.364238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364238+0000","last_undegraded":"2024-03-14T09:03:12.364238+0000","last_fullsized":"2024-03-14T09:03:12.364238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367869+0000","last_change":"2024-03-14T09:03:12.365684+0000","last_active":"2024-03-14T09:03:12.367869+0000","last_peered":"2024-03-14T09:03:12.367869+0000","last_clean":"2024-03-14T09:03:12.367869+0000","last_became_active":"2024-03-14T09:03:12.365263+0000","last_became_peered":"2024-03-14T09:03:12.365263+0000","last_unstale":"2024-03-14T09:03:12.367869+0000","last_undegraded":"2024-03-14T09:03:12.367869+0000","last_fullsized":"2024-03-14T09:03:12.367869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351721+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:11.351721+0000","last_peered":"2024-03-14T09:03:11.351721+0000","last_clean":"2024-03-14T09:03:11.351721+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:11.351721+0000","last_undegraded":"2024-03-14T09:03:11.351721+0000","last_fullsized":"2024-03-14T09:03:11.351721+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355737+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:11.355737+0000","last_peered":"2024-03-14T09:03:11.355737+0000","last_clean":"2024-03-14T09:03:11.355737+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:11.355737+0000","last_undegraded":"2024-03-14T09:03:11.355737+0000","last_fullsized":"2024-03-14T09:03:11.355737+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357463+0000","last_change":"2024-03-14T09:03:12.357463+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357463+0000","last_undegraded":"2024-03-14T09:03:12.357463+0000","last_fullsized":"2024-03-14T09:03:12.357463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376072+0000","last_change":"2024-03-14T09:03:12.371367+0000","last_active":"2024-03-14T09:03:12.376072+0000","last_peered":"2024-03-14T09:03:12.376072+0000","last_clean":"2024-03-14T09:03:12.376072+0000","last_became_active":"2024-03-14T09:03:12.371044+0000","last_became_peered":"2024-03-14T09:03:12.371044+0000","last_unstale":"2024-03-14T09:03:12.376072+0000","last_undegraded":"2024-03-14T09:03:12.376072+0000","last_fullsized":"2024-03-14T09:03:12.376072+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:20:22.371301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358462+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:11.358462+0000","last_peered":"2024-03-14T09:03:11.358462+0000","last_clean":"2024-03-14T09:03:11.358462+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:11.358462+0000","last_undegraded":"2024-03-14T09:03:11.358462+0000","last_fullsized":"2024-03-14T09:03:11.358462+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349280+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:11.349280+0000","last_peered":"2024-03-14T09:03:11.349280+0000","last_clean":"2024-03-14T09:03:11.349280+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:11.349280+0000","last_undegraded":"2024-03-14T09:03:11.349280+0000","last_fullsized":"2024-03-14T09:03:11.349280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352704+0000","last_change":"2024-03-14T09:03:12.352704+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352704+0000","last_undegraded":"2024-03-14T09:03:12.352704+0000","last_fullsized":"2024-03-14T09:03:12.352704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374596+0000","last_change":"2024-03-14T09:03:12.372117+0000","last_active":"2024-03-14T09:03:12.374596+0000","last_peered":"2024-03-14T09:03:12.374596+0000","last_clean":"2024-03-14T09:03:12.374596+0000","last_became_active":"2024-03-14T09:03:12.371774+0000","last_became_peered":"2024-03-14T09:03:12.371774+0000","last_unstale":"2024-03-14T09:03:12.374596+0000","last_undegraded":"2024-03-14T09:03:12.374596+0000","last_fullsized":"2024-03-14T09:03:12.374596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:04.409001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355185+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:11.355185+0000","last_peered":"2024-03-14T09:03:11.355185+0000","last_clean":"2024-03-14T09:03:11.355185+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:11.355185+0000","last_undegraded":"2024-03-14T09:03:11.355185+0000","last_fullsized":"2024-03-14T09:03:11.355185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351724+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:11.351724+0000","last_peered":"2024-03-14T09:03:11.351724+0000","last_clean":"2024-03-14T09:03:11.351724+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:11.351724+0000","last_undegraded":"2024-03-14T09:03:11.351724+0000","last_fullsized":"2024-03-14T09:03:11.351724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354163+0000","last_change":"2024-03-14T09:03:12.354163+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354163+0000","last_undegraded":"2024-03-14T09:03:12.354163+0000","last_fullsized":"2024-03-14T09:03:12.354163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372069+0000","last_change":"2024-03-14T09:03:12.370993+0000","last_active":"2024-03-14T09:03:12.372069+0000","last_peered":"2024-03-14T09:03:12.372069+0000","last_clean":"2024-03-14T09:03:12.372069+0000","last_became_active":"2024-03-14T09:03:12.370745+0000","last_became_peered":"2024-03-14T09:03:12.370745+0000","last_unstale":"2024-03-14T09:03:12.372069+0000","last_undegraded":"2024-03-14T09:03:12.372069+0000","last_fullsized":"2024-03-14T09:03:12.372069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349338+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:11.349338+0000","last_peered":"2024-03-14T09:03:11.349338+0000","last_clean":"2024-03-14T09:03:11.349338+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:11.349338+0000","last_undegraded":"2024-03-14T09:03:11.349338+0000","last_fullsized":"2024-03-14T09:03:11.349338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356757+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:11.356757+0000","last_peered":"2024-03-14T09:03:11.356757+0000","last_clean":"2024-03-14T09:03:11.356757+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:11.356757+0000","last_undegraded":"2024-03-14T09:03:11.356757+0000","last_fullsized":"2024-03-14T09:03:11.356757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353676+0000","last_change":"2024-03-14T09:03:12.353676+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353676+0000","last_undegraded":"2024-03-14T09:03:12.353676+0000","last_fullsized":"2024-03-14T09:03:12.353676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368922+0000","last_change":"2024-03-14T09:03:12.366770+0000","last_active":"2024-03-14T09:03:12.368922+0000","last_peered":"2024-03-14T09:03:12.368922+0000","last_clean":"2024-03-14T09:03:12.368922+0000","last_became_active":"2024-03-14T09:03:12.366022+0000","last_became_peered":"2024-03-14T09:03:12.366022+0000","last_unstale":"2024-03-14T09:03:12.368922+0000","last_undegraded":"2024-03-14T09:03:12.368922+0000","last_fullsized":"2024-03-14T09:03:12.368922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349099+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:11.349099+0000","last_peered":"2024-03-14T09:03:11.349099+0000","last_clean":"2024-03-14T09:03:11.349099+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:11.349099+0000","last_undegraded":"2024-03-14T09:03:11.349099+0000","last_fullsized":"2024-03-14T09:03:11.349099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354967+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:11.354967+0000","last_peered":"2024-03-14T09:03:11.354967+0000","last_clean":"2024-03-14T09:03:11.354967+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:11.354967+0000","last_undegraded":"2024-03-14T09:03:11.354967+0000","last_fullsized":"2024-03-14T09:03:11.354967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356783+0000","last_change":"2024-03-14T09:03:12.356783+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356783+0000","last_undegraded":"2024-03-14T09:03:12.356783+0000","last_fullsized":"2024-03-14T09:03:12.356783+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375064+0000","last_change":"2024-03-14T09:03:12.373601+0000","last_active":"2024-03-14T09:03:12.375064+0000","last_peered":"2024-03-14T09:03:12.375064+0000","last_clean":"2024-03-14T09:03:12.375064+0000","last_became_active":"2024-03-14T09:03:12.373332+0000","last_became_peered":"2024-03-14T09:03:12.373332+0000","last_unstale":"2024-03-14T09:03:12.375064+0000","last_undegraded":"2024-03-14T09:03:12.375064+0000","last_fullsized":"2024-03-14T09:03:12.375064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:38:05.459173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355546+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:11.355546+0000","last_peered":"2024-03-14T09:03:11.355546+0000","last_clean":"2024-03-14T09:03:11.355546+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:11.355546+0000","last_undegraded":"2024-03-14T09:03:11.355546+0000","last_fullsized":"2024-03-14T09:03:11.355546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358290+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:11.358290+0000","last_peered":"2024-03-14T09:03:11.358290+0000","last_clean":"2024-03-14T09:03:11.358290+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:11.358290+0000","last_undegraded":"2024-03-14T09:03:11.358290+0000","last_fullsized":"2024-03-14T09:03:11.358290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357797+0000","last_change":"2024-03-14T09:03:12.357797+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357797+0000","last_undegraded":"2024-03-14T09:03:12.357797+0000","last_fullsized":"2024-03-14T09:03:12.357797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375031+0000","last_change":"2024-03-14T09:03:12.373610+0000","last_active":"2024-03-14T09:03:12.375031+0000","last_peered":"2024-03-14T09:03:12.375031+0000","last_clean":"2024-03-14T09:03:12.375031+0000","last_became_active":"2024-03-14T09:03:12.373360+0000","last_became_peered":"2024-03-14T09:03:12.373360+0000","last_unstale":"2024-03-14T09:03:12.375031+0000","last_undegraded":"2024-03-14T09:03:12.375031+0000","last_fullsized":"2024-03-14T09:03:12.375031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:32.334273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355435+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:11.355435+0000","last_peered":"2024-03-14T09:03:11.355435+0000","last_clean":"2024-03-14T09:03:11.355435+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:11.355435+0000","last_undegraded":"2024-03-14T09:03:11.355435+0000","last_fullsized":"2024-03-14T09:03:11.355435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"51'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349459+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:11.349459+0000","last_peered":"2024-03-14T09:03:11.349459+0000","last_clean":"2024-03-14T09:03:11.349459+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:11.349459+0000","last_undegraded":"2024-03-14T09:03:11.349459+0000","last_fullsized":"2024-03-14T09:03:11.349459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356751+0000","last_change":"2024-03-14T09:03:12.356751+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356751+0000","last_undegraded":"2024-03-14T09:03:12.356751+0000","last_fullsized":"2024-03-14T09:03:12.356751+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367852+0000","last_change":"2024-03-14T09:03:12.365515+0000","last_active":"2024-03-14T09:03:12.367852+0000","last_peered":"2024-03-14T09:03:12.367852+0000","last_clean":"2024-03-14T09:03:12.367852+0000","last_became_active":"2024-03-14T09:03:12.364921+0000","last_became_peered":"2024-03-14T09:03:12.364921+0000","last_unstale":"2024-03-14T09:03:12.367852+0000","last_undegraded":"2024-03-14T09:03:12.367852+0000","last_fullsized":"2024-03-14T09:03:12.367852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357128+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:11.357128+0000","last_peered":"2024-03-14T09:03:11.357128+0000","last_clean":"2024-03-14T09:03:11.357128+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:11.357128+0000","last_undegraded":"2024-03-14T09:03:11.357128+0000","last_fullsized":"2024-03-14T09:03:11.357128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355195+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:11.355195+0000","last_peered":"2024-03-14T09:03:11.355195+0000","last_clean":"2024-03-14T09:03:11.355195+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:11.355195+0000","last_undegraded":"2024-03-14T09:03:11.355195+0000","last_fullsized":"2024-03-14T09:03:11.355195+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361654+0000","last_change":"2024-03-14T09:03:12.361654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361654+0000","last_undegraded":"2024-03-14T09:03:12.361654+0000","last_fullsized":"2024-03-14T09:03:12.361654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371874+0000","last_change":"2024-03-14T09:03:12.367975+0000","last_active":"2024-03-14T09:03:12.371874+0000","last_peered":"2024-03-14T09:03:12.371874+0000","last_clean":"2024-03-14T09:03:12.371874+0000","last_became_active":"2024-03-14T09:03:12.367702+0000","last_became_peered":"2024-03-14T09:03:12.367702+0000","last_unstale":"2024-03-14T09:03:12.371874+0000","last_undegraded":"2024-03-14T09:03:12.371874+0000","last_fullsized":"2024-03-14T09:03:12.371874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:12.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349479+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:11.349479+0000","last_peered":"2024-03-14T09:03:11.349479+0000","last_clean":"2024-03-14T09:03:11.349479+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:11.349479+0000","last_undegraded":"2024-03-14T09:03:11.349479+0000","last_fullsized":"2024-03-14T09:03:11.349479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355796+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:11.355796+0000","last_peered":"2024-03-14T09:03:11.355796+0000","last_clean":"2024-03-14T09:03:11.355796+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:11.355796+0000","last_undegraded":"2024-03-14T09:03:11.355796+0000","last_fullsized":"2024-03-14T09:03:11.355796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358131+0000","last_change":"2024-03-14T09:03:12.358131+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358131+0000","last_undegraded":"2024-03-14T09:03:12.358131+0000","last_fullsized":"2024-03-14T09:03:12.358131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376099+0000","last_change":"2024-03-14T09:03:12.371984+0000","last_active":"2024-03-14T09:03:12.376099+0000","last_peered":"2024-03-14T09:03:12.376099+0000","last_clean":"2024-03-14T09:03:12.376099+0000","last_became_active":"2024-03-14T09:03:12.371727+0000","last_became_peered":"2024-03-14T09:03:12.371727+0000","last_unstale":"2024-03-14T09:03:12.376099+0000","last_undegraded":"2024-03-14T09:03:12.376099+0000","last_fullsized":"2024-03-14T09:03:12.376099+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:23.845822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351395+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:11.351395+0000","last_peered":"2024-03-14T09:03:11.351395+0000","last_clean":"2024-03-14T09:03:11.351395+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:11.351395+0000","last_undegraded":"2024-03-14T09:03:11.351395+0000","last_fullsized":"2024-03-14T09:03:11.351395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356875+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:11.356875+0000","last_peered":"2024-03-14T09:03:11.356875+0000","last_clean":"2024-03-14T09:03:11.356875+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:11.356875+0000","last_undegraded":"2024-03-14T09:03:11.356875+0000","last_fullsized":"2024-03-14T09:03:11.356875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352756+0000","last_change":"2024-03-14T09:03:12.352756+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352756+0000","last_undegraded":"2024-03-14T09:03:12.352756+0000","last_fullsized":"2024-03-14T09:03:12.352756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.366806+0000","last_change":"2024-03-14T09:03:12.363729+0000","last_active":"2024-03-14T09:03:12.366806+0000","last_peered":"2024-03-14T09:03:12.366806+0000","last_clean":"2024-03-14T09:03:12.366806+0000","last_became_active":"2024-03-14T09:03:12.363440+0000","last_became_peered":"2024-03-14T09:03:12.363440+0000","last_unstale":"2024-03-14T09:03:12.366806+0000","last_undegraded":"2024-03-14T09:03:12.366806+0000","last_fullsized":"2024-03-14T09:03:12.366806+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:13.058703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358367+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:11.358367+0000","last_peered":"2024-03-14T09:03:11.358367+0000","last_clean":"2024-03-14T09:03:11.358367+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:11.358367+0000","last_undegraded":"2024-03-14T09:03:11.358367+0000","last_fullsized":"2024-03-14T09:03:11.358367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355086+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:11.355086+0000","last_peered":"2024-03-14T09:03:11.355086+0000","last_clean":"2024-03-14T09:03:11.355086+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:11.355086+0000","last_undegraded":"2024-03-14T09:03:11.355086+0000","last_fullsized":"2024-03-14T09:03:11.355086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359840+0000","last_change":"2024-03-14T09:03:12.359840+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359840+0000","last_undegraded":"2024-03-14T09:03:12.359840+0000","last_fullsized":"2024-03-14T09:03:12.359840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372010+0000","last_change":"2024-03-14T09:03:12.370434+0000","last_active":"2024-03-14T09:03:12.372010+0000","last_peered":"2024-03-14T09:03:12.372010+0000","last_clean":"2024-03-14T09:03:12.372010+0000","last_became_active":"2024-03-14T09:03:12.370170+0000","last_became_peered":"2024-03-14T09:03:12.370170+0000","last_unstale":"2024-03-14T09:03:12.372010+0000","last_undegraded":"2024-03-14T09:03:12.372010+0000","last_fullsized":"2024-03-14T09:03:12.372010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:55.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356464+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:11.356464+0000","last_peered":"2024-03-14T09:03:11.356464+0000","last_clean":"2024-03-14T09:03:11.356464+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:11.356464+0000","last_undegraded":"2024-03-14T09:03:11.356464+0000","last_fullsized":"2024-03-14T09:03:11.356464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356577+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:11.356577+0000","last_peered":"2024-03-14T09:03:11.356577+0000","last_clean":"2024-03-14T09:03:11.356577+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:11.356577+0000","last_undegraded":"2024-03-14T09:03:11.356577+0000","last_fullsized":"2024-03-14T09:03:11.356577+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353449+0000","last_change":"2024-03-14T09:03:12.353449+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353449+0000","last_undegraded":"2024-03-14T09:03:12.353449+0000","last_fullsized":"2024-03-14T09:03:12.353449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375525+0000","last_change":"2024-03-14T09:03:12.373132+0000","last_active":"2024-03-14T09:03:12.375525+0000","last_peered":"2024-03-14T09:03:12.375525+0000","last_clean":"2024-03-14T09:03:12.375525+0000","last_became_active":"2024-03-14T09:03:12.372802+0000","last_became_peered":"2024-03-14T09:03:12.372802+0000","last_unstale":"2024-03-14T09:03:12.375525+0000","last_undegraded":"2024-03-14T09:03:12.375525+0000","last_fullsized":"2024-03-14T09:03:12.375525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351200+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:11.351200+0000","last_peered":"2024-03-14T09:03:11.351200+0000","last_clean":"2024-03-14T09:03:11.351200+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:11.351200+0000","last_undegraded":"2024-03-14T09:03:11.351200+0000","last_fullsized":"2024-03-14T09:03:11.351200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351201+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:11.351201+0000","last_peered":"2024-03-14T09:03:11.351201+0000","last_clean":"2024-03-14T09:03:11.351201+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:11.351201+0000","last_undegraded":"2024-03-14T09:03:11.351201+0000","last_fullsized":"2024-03-14T09:03:11.351201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353238+0000","last_change":"2024-03-14T09:03:12.353238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353238+0000","last_undegraded":"2024-03-14T09:03:12.353238+0000","last_fullsized":"2024-03-14T09:03:12.353238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367162+0000","last_change":"2024-03-14T09:03:12.364039+0000","last_active":"2024-03-14T09:03:12.367162+0000","last_peered":"2024-03-14T09:03:12.367162+0000","last_clean":"2024-03-14T09:03:12.367162+0000","last_became_active":"2024-03-14T09:03:12.363778+0000","last_became_peered":"2024-03-14T09:03:12.363778+0000","last_unstale":"2024-03-14T09:03:12.367162+0000","last_undegraded":"2024-03-14T09:03:12.367162+0000","last_fullsized":"2024-03-14T09:03:12.367162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:53.168005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353295+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:11.353295+0000","last_peered":"2024-03-14T09:03:11.353295+0000","last_clean":"2024-03-14T09:03:11.353295+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:11.353295+0000","last_undegraded":"2024-03-14T09:03:11.353295+0000","last_fullsized":"2024-03-14T09:03:11.353295+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358124+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:11.358124+0000","last_peered":"2024-03-14T09:03:11.358124+0000","last_clean":"2024-03-14T09:03:11.358124+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:11.358124+0000","last_undegraded":"2024-03-14T09:03:11.358124+0000","last_fullsized":"2024-03-14T09:03:11.358124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1236992,"data_stored":861723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.623}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":271195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":264155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.37}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.318}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.621}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":858090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.372},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.316}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.44}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.504},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":271251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.254},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.435}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":274270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.625},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:17.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-03-14T09:03:17.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353917+0000', 'last_change': '2024-03-14T09:03:12.353917+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353917+0000', 'last_undegraded': '2024-03-14T09:03:12.353917+0000', 'last_fullsized': '2024-03-14T09:03:12.353917+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361638+0000', 'last_change': '2024-03-14T09:03:12.361638+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361638+0000', 'last_undegraded': '2024-03-14T09:03:12.361638+0000', 'last_fullsized': '2024-03-14T09:03:12.361638+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.358323+0000', 'last_change': '2024-03-14T09:03:12.358323+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.358323+0000', 'last_undegraded': '2024-03-14T09:03:12.358323+0000', 'last_fullsized': '2024-03-14T09:03:12.358323+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.354698+0000', 'last_change': '2024-03-14T09:03:12.354698+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.354698+0000', 'last_undegraded': '2024-03-14T09:03:12.354698+0000', 'last_fullsized': '2024-03-14T09:03:12.354698+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.362720+0000', 'last_change': '2024-03-14T09:03:12.362720+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.362720+0000', 'last_undegraded': '2024-03-14T09:03:12.362720+0000', 'last_fullsized': '2024-03-14T09:03:12.362720+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-03-14T09:03:17.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353623+0000', 'last_change': '2024-03-14T09:03:12.353623+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353623+0000', 'last_undegraded': '2024-03-14T09:03:12.353623+0000', 'last_fullsized': '2024-03-14T09:03:12.353623+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.360227+0000', 'last_change': '2024-03-14T09:03:12.360227+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.360227+0000', 'last_undegraded': '2024-03-14T09:03:12.360227+0000', 'last_fullsized': '2024-03-14T09:03:12.360227+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353748+0000', 'last_change': '2024-03-14T09:03:12.353748+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353748+0000', 'last_undegraded': '2024-03-14T09:03:12.353748+0000', 'last_fullsized': '2024-03-14T09:03:12.353748+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357388+0000', 'last_change': '2024-03-14T09:03:12.357388+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357388+0000', 'last_undegraded': '2024-03-14T09:03:12.357388+0000', 'last_fullsized': '2024-03-14T09:03:12.357388+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.362629+0000', 'last_change': '2024-03-14T09:03:12.362629+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.362629+0000', 'last_undegraded': '2024-03-14T09:03:12.362629+0000', 'last_fullsized': '2024-03-14T09:03:12.362629+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359448+0000', 'last_change': '2024-03-14T09:03:12.359448+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359448+0000', 'last_undegraded': '2024-03-14T09:03:12.359448+0000', 'last_fullsized': '2024-03-14T09:03:12.359448+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.364654+0000', 'last_change': '2024-03-14T09:03:12.364654+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.364654+0000', 'last_undegraded': '2024-03-14T09:03:12.364654+0000', 'last_fullsized': '2024-03-14T09:03:12.364654+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356123+0000', 'last_change': '2024-03-14T09:03:12.356123+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356123+0000', 'last_undegraded': '2024-03-14T09:03:12.356123+0000', 'last_fullsized': '2024-03-14T09:03:12.356123+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361949+0000', 'last_change': '2024-03-14T09:03:12.361949+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361949+0000', 'last_undegraded': '2024-03-14T09:03:12.361949+0000', 'last_fullsized': '2024-03-14T09:03:12.361949+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356192+0000', 'last_change': '2024-03-14T09:03:12.356192+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356192+0000', 'last_undegraded': '2024-03-14T09:03:12.356192+0000', 'last_fullsized': '2024-03-14T09:03:12.356192+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.364020+0000', 'last_change': '2024-03-14T09:03:12.364020+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.364020+0000', 'last_undegraded': '2024-03-14T09:03:12.364020+0000', 'last_fullsized': '2024-03-14T09:03:12.364020+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353377+0000', 'last_change': '2024-03-14T09:03:12.353377+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353377+0000', 'last_undegraded': '2024-03-14T09:03:12.353377+0000', 'last_fullsized': '2024-03-14T09:03:12.353377+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356427+0000', 'last_change': '2024-03-14T09:03:12.356427+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356427+0000', 'last_undegraded': '2024-03-14T09:03:12.356427+0000', 'last_fullsized': '2024-03-14T09:03:12.356427+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.360390+0000', 'last_change': '2024-03-14T09:03:12.360390+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.360390+0000', 'last_undegraded': '2024-03-14T09:03:12.360390+0000', 'last_fullsized': '2024-03-14T09:03:12.360390+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359276+0000', 'last_change': '2024-03-14T09:03:12.359276+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359276+0000', 'last_undegraded': '2024-03-14T09:03:12.359276+0000', 'last_fullsized': '2024-03-14T09:03:12.359276+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-03-14T09:03:17.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356216+0000', 'last_change': '2024-03-14T09:03:12.356216+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356216+0000', 'last_undegraded': '2024-03-14T09:03:12.356216+0000', 'last_fullsized': '2024-03-14T09:03:12.356216+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357282+0000', 'last_change': '2024-03-14T09:03:12.357282+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357282+0000', 'last_undegraded': '2024-03-14T09:03:12.357282+0000', 'last_fullsized': '2024-03-14T09:03:12.357282+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.364628+0000', 'last_change': '2024-03-14T09:03:12.364628+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.364628+0000', 'last_undegraded': '2024-03-14T09:03:12.364628+0000', 'last_fullsized': '2024-03-14T09:03:12.364628+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.362012+0000', 'last_change': '2024-03-14T09:03:12.362012+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.362012+0000', 'last_undegraded': '2024-03-14T09:03:12.362012+0000', 'last_fullsized': '2024-03-14T09:03:12.362012+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.352812+0000', 'last_change': '2024-03-14T09:03:12.352812+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.352812+0000', 'last_undegraded': '2024-03-14T09:03:12.352812+0000', 'last_fullsized': '2024-03-14T09:03:12.352812+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361474+0000', 'last_change': '2024-03-14T09:03:12.361474+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361474+0000', 'last_undegraded': '2024-03-14T09:03:12.361474+0000', 'last_fullsized': '2024-03-14T09:03:12.361474+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-03-14T09:03:17.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361988+0000', 'last_change': '2024-03-14T09:03:12.361988+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361988+0000', 'last_undegraded': '2024-03-14T09:03:12.361988+0000', 'last_fullsized': '2024-03-14T09:03:12.361988+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.362109+0000', 'last_change': '2024-03-14T09:03:12.362109+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.362109+0000', 'last_undegraded': '2024-03-14T09:03:12.362109+0000', 'last_fullsized': '2024-03-14T09:03:12.362109+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.362252+0000', 'last_change': '2024-03-14T09:03:12.362252+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.362252+0000', 'last_undegraded': '2024-03-14T09:03:12.362252+0000', 'last_fullsized': '2024-03-14T09:03:12.362252+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353125+0000', 'last_change': '2024-03-14T09:03:12.353125+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353125+0000', 'last_undegraded': '2024-03-14T09:03:12.353125+0000', 'last_fullsized': '2024-03-14T09:03:12.353125+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.354173+0000', 'last_change': '2024-03-14T09:03:12.354173+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.354173+0000', 'last_undegraded': '2024-03-14T09:03:12.354173+0000', 'last_fullsized': '2024-03-14T09:03:12.354173+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361509+0000', 'last_change': '2024-03-14T09:03:12.361509+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361509+0000', 'last_undegraded': '2024-03-14T09:03:12.361509+0000', 'last_fullsized': '2024-03-14T09:03:12.361509+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.354070+0000', 'last_change': '2024-03-14T09:03:12.354070+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.354070+0000', 'last_undegraded': '2024-03-14T09:03:12.354070+0000', 'last_fullsized': '2024-03-14T09:03:12.354070+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356924+0000', 'last_change': '2024-03-14T09:03:12.356924+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356924+0000', 'last_undegraded': '2024-03-14T09:03:12.356924+0000', 'last_fullsized': '2024-03-14T09:03:12.356924+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356192+0000', 'last_change': '2024-03-14T09:03:12.356192+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356192+0000', 'last_undegraded': '2024-03-14T09:03:12.356192+0000', 'last_fullsized': '2024-03-14T09:03:12.356192+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359921+0000', 'last_change': '2024-03-14T09:03:12.359921+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359921+0000', 'last_undegraded': '2024-03-14T09:03:12.359921+0000', 'last_fullsized': '2024-03-14T09:03:12.359921+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-03-14T09:03:17.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.360660+0000', 'last_change': '2024-03-14T09:03:12.360660+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.360660+0000', 'last_undegraded': '2024-03-14T09:03:12.360660+0000', 'last_fullsized': '2024-03-14T09:03:12.360660+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.355339+0000', 'last_change': '2024-03-14T09:03:12.355339+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.355339+0000', 'last_undegraded': '2024-03-14T09:03:12.355339+0000', 'last_fullsized': '2024-03-14T09:03:12.355339+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361025+0000', 'last_change': '2024-03-14T09:03:12.361025+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361025+0000', 'last_undegraded': '2024-03-14T09:03:12.361025+0000', 'last_fullsized': '2024-03-14T09:03:12.361025+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361000+0000', 'last_change': '2024-03-14T09:03:12.361000+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361000+0000', 'last_undegraded': '2024-03-14T09:03:12.361000+0000', 'last_fullsized': '2024-03-14T09:03:12.361000+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353503+0000', 'last_change': '2024-03-14T09:03:12.353503+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353503+0000', 'last_undegraded': '2024-03-14T09:03:12.353503+0000', 'last_fullsized': '2024-03-14T09:03:12.353503+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.364515+0000', 'last_change': '2024-03-14T09:03:12.364515+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.364515+0000', 'last_undegraded': '2024-03-14T09:03:12.364515+0000', 'last_fullsized': '2024-03-14T09:03:12.364515+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-03-14T09:03:17.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359668+0000', 'last_change': '2024-03-14T09:03:12.359668+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359668+0000', 'last_undegraded': '2024-03-14T09:03:12.359668+0000', 'last_fullsized': '2024-03-14T09:03:12.359668+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357946+0000', 'last_change': '2024-03-14T09:03:12.357946+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357946+0000', 'last_undegraded': '2024-03-14T09:03:12.357946+0000', 'last_fullsized': '2024-03-14T09:03:12.357946+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.355535+0000', 'last_change': '2024-03-14T09:03:12.355535+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.355535+0000', 'last_undegraded': '2024-03-14T09:03:12.355535+0000', 'last_fullsized': '2024-03-14T09:03:12.355535+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.358690+0000', 'last_change': '2024-03-14T09:03:12.358690+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.358690+0000', 'last_undegraded': '2024-03-14T09:03:12.358690+0000', 'last_fullsized': '2024-03-14T09:03:12.358690+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356321+0000', 'last_change': '2024-03-14T09:03:12.356321+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356321+0000', 'last_undegraded': '2024-03-14T09:03:12.356321+0000', 'last_fullsized': '2024-03-14T09:03:12.356321+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-03-14T09:03:17.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.360063+0000', 'last_change': '2024-03-14T09:03:12.360063+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.360063+0000', 'last_undegraded': '2024-03-14T09:03:12.360063+0000', 'last_fullsized': '2024-03-14T09:03:12.360063+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357489+0000', 'last_change': '2024-03-14T09:03:12.357489+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357489+0000', 'last_undegraded': '2024-03-14T09:03:12.357489+0000', 'last_fullsized': '2024-03-14T09:03:12.357489+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359244+0000', 'last_change': '2024-03-14T09:03:12.359244+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359244+0000', 'last_undegraded': '2024-03-14T09:03:12.359244+0000', 'last_fullsized': '2024-03-14T09:03:12.359244+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.364238+0000', 'last_change': '2024-03-14T09:03:12.364238+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.364238+0000', 'last_undegraded': '2024-03-14T09:03:12.364238+0000', 'last_fullsized': '2024-03-14T09:03:12.364238+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-03-14T09:03:17.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357463+0000', 'last_change': '2024-03-14T09:03:12.357463+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357463+0000', 'last_undegraded': '2024-03-14T09:03:12.357463+0000', 'last_fullsized': '2024-03-14T09:03:12.357463+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.352704+0000', 'last_change': '2024-03-14T09:03:12.352704+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.352704+0000', 'last_undegraded': '2024-03-14T09:03:12.352704+0000', 'last_fullsized': '2024-03-14T09:03:12.352704+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.354163+0000', 'last_change': '2024-03-14T09:03:12.354163+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.354163+0000', 'last_undegraded': '2024-03-14T09:03:12.354163+0000', 'last_fullsized': '2024-03-14T09:03:12.354163+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353676+0000', 'last_change': '2024-03-14T09:03:12.353676+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353676+0000', 'last_undegraded': '2024-03-14T09:03:12.353676+0000', 'last_fullsized': '2024-03-14T09:03:12.353676+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356783+0000', 'last_change': '2024-03-14T09:03:12.356783+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356783+0000', 'last_undegraded': '2024-03-14T09:03:12.356783+0000', 'last_fullsized': '2024-03-14T09:03:12.356783+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-03-14T09:03:17.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.357797+0000', 'last_change': '2024-03-14T09:03:12.357797+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.357797+0000', 'last_undegraded': '2024-03-14T09:03:12.357797+0000', 'last_fullsized': '2024-03-14T09:03:12.357797+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.356751+0000', 'last_change': '2024-03-14T09:03:12.356751+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.356751+0000', 'last_undegraded': '2024-03-14T09:03:12.356751+0000', 'last_fullsized': '2024-03-14T09:03:12.356751+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.361654+0000', 'last_change': '2024-03-14T09:03:12.361654+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.361654+0000', 'last_undegraded': '2024-03-14T09:03:12.361654+0000', 'last_fullsized': '2024-03-14T09:03:12.361654+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.358131+0000', 'last_change': '2024-03-14T09:03:12.358131+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.358131+0000', 'last_undegraded': '2024-03-14T09:03:12.358131+0000', 'last_fullsized': '2024-03-14T09:03:12.358131+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-03-14T09:03:17.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.352756+0000', 'last_change': '2024-03-14T09:03:12.352756+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.352756+0000', 'last_undegraded': '2024-03-14T09:03:12.352756+0000', 'last_fullsized': '2024-03-14T09:03:12.352756+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.359840+0000', 'last_change': '2024-03-14T09:03:12.359840+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.359840+0000', 'last_undegraded': '2024-03-14T09:03:12.359840+0000', 'last_fullsized': '2024-03-14T09:03:12.359840+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353449+0000', 'last_change': '2024-03-14T09:03:12.353449+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353449+0000', 'last_undegraded': '2024-03-14T09:03:12.353449+0000', 'last_fullsized': '2024-03-14T09:03:12.353449+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-03-14T09:03:17.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-03-14T09:03:12.353238+0000', 'last_change': '2024-03-14T09:03:12.353238+0000', 'last_active': '2024-03-14T09:03:12.347114+0000', 'last_peered': '2024-03-14T09:03:12.347114+0000', 'last_clean': '2024-03-14T09:03:12.347114+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-14T09:03:12.353238+0000', 'last_undegraded': '2024-03-14T09:03:12.353238+0000', 'last_fullsized': '2024-03-14T09:03:12.353238+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'last_clean_scrub_stamp': '2024-03-14T09:03:12.347114+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-14T09:03:17.697 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-14T09:03:17.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.836+0000 7fd18dea6700 1 Processor -- start 2024-03-14T09:03:17.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.836+0000 7fd18dea6700 1 -- start start 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd18dea6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd1880702f0 0x7fd1880c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd18dea6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd1880c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd18dea6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880acd10 con 0x7fd1880c78c0 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd18dea6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880ac7b0 con 0x7fd1880702f0 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd18dea6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd188150050 con 0x7fd18814fbd0 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd187fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1880c78c0 0x7fd1880c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35244/0 (socket says 172.21.15.154:35244) 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd187fff700 1 -- 172.21.15.154:0/1106816590 learned_addr learned my addr 172.21.15.154:0/1106816590 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd1880c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.837+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304341115 0 0) 0x7fd1880acd10 con 0x7fd1880c78c0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1877fe700 1 --2- 172.21.15.154:0/1106816590 >> v2:172.21.15.191:3300/0 conn(0x7fd1880702f0 0x7fd1880c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1740035b0 con 0x7fd1880c78c0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 605030425 0 0) 0x7fd1740035b0 con 0x7fd1880c78c0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 msgr2=0x7fd1880c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd1880c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 >> v2:172.21.15.191:3300/0 conn(0x7fd1880702f0 msgr2=0x7fd1880c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 --2- 172.21.15.154:0/1106816590 >> v2:172.21.15.191:3300/0 conn(0x7fd1880702f0 0x7fd1880c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880ac9c0 con 0x7fd1880c78c0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fd178001670 con 0x7fd1880c78c0 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 >> v1:172.21.15.154:6789/0 conn(0x7fd1880c78c0 legacy=0x7fd1880c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd174003a00 0x7fd174003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd174004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd174005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880ac7b0 con 0x7fd174004d90 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188150050 con 0x7fd174003a00 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17400ba10 con 0x7fd18814fbd0 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd174005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd174004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd1877fe700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd174003a00 0x7fd174003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 -- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 msgr2=0x7fd174004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd18814fbd0 0x7fd174004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 -- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd174003a00 msgr2=0x7fd174003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd174003a00 0x7fd174003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 -- 172.21.15.154:0/1106816590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740035b0 con 0x7fd174004d90 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.838+0000 7fd187fff700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd174005210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c370 tx=0x7fd174005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feeb786a2eced7f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178007c60 con 0x7fd174004d90 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178005960 con 0x7fd174004d90 2024-03-14T09:03:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd1867fc700 1 -- 172.21.15.154:0/1106816590 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178005d10 con 0x7fd174004d90 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 -- 172.21.15.154:0/1106816590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 msgr2=0x7fd174005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1106816590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd174005210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c370 tx=0x7fd174005880 comp rx=0 tx=0).stop 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd187fff700 1 -- 172.21.15.154:0/1106816590 reap_dead start 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 -- 172.21.15.154:0/1106816590 shutdown_connections 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 -- 172.21.15.154:0/1106816590 >> 172.21.15.154:0/1106816590 conn(0x7fd1880a9bf0 msgr2=0x7fd1880a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 -- 172.21.15.154:0/1106816590 shutdown_connections 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.839+0000 7fd18dea6700 1 -- 172.21.15.154:0/1106816590 wait complete. 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.840+0000 7fd18dea6700 1 Processor -- start 2024-03-14T09:03:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 -- start start 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 0x7fd1881506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd188148610 0x7fd188148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881481f0 con 0x7fd174004d90 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188149000 con 0x7fd188152cc0 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd18dea6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd188147090 con 0x7fd188148610 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd1877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd1877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56366/0 (socket says 172.21.15.154:56366) 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd1877fe700 1 -- 172.21.15.154:0/1115152785 learned_addr learned my addr 172.21.15.154:0/1115152785 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 0x7fd1881506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.841+0000 7fd187fff700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd188148610 0x7fd188148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 -- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd188148610 msgr2=0x7fd188148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd188148610 0x7fd188148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 -- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 msgr2=0x7fd1881506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 0x7fd1881506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 -- 172.21.15.154:0/1115152785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188147b30 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 0x7fd1881506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd1877fe700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd180011e30 tx=0x7fd18000f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000e040 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000f040 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188147e10 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180009c00 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.842+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881549f0 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.843+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188148330 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.843+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd180015650 con 0x7fd174004d90 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.844+0000 7fd17ffff700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 0x7fd16806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.844+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd180089d00 con 0x7fd174004d90 2024-03-14T09:03:17.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.844+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 0x7fd16806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:17.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.845+0000 7fd186ffd700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 0x7fd16806cd20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1880c4280 tx=0x7fd170008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:17.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:17.848+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd188148330 con 0x7fd174004d90 2024-03-14T09:03:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.120+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd188148330 con 0x7fd174004d90 2024-03-14T09:03:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.121+0000 7fd17ffff700 1 -- 172.21.15.154:0/1115152785 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1619 (secure 0 0 0) 0x7fd188148330 con 0x7fd174004d90 2024-03-14T09:03:18.121 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:18.121 INFO:teuthology.orchestra.run.smithi154.stdout:{"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":136,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1710406872,"num_in_osds":8,"osd_in_since":1710406864,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129},{"state_name":"creating+peering","count":64}],"num_pgs":193,"num_pools":5,"num_objects":24,"data_bytes":594241,"bytes_used":227647488,"bytes_avail":772866465792,"bytes_total":773094113280,"inactive_pgs_ratio":0.33160620927810669,"write_bytes_sec":488,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":5715},{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":5727}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-14T09:02:41.537646+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 msgr2=0x7fd16806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 0x7fd16806cd20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd1880c4280 tx=0x7fd170008040 comp rx=0 tx=0).stop 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 msgr2=0x7fd188146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd180011e30 tx=0x7fd18000f490 comp rx=0 tx=0).stop 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 shutdown_connections 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd188148610 0x7fd188148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd16806a860 0x7fd16806cd20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd174004d90 0x7fd188146ab0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 --2- 172.21.15.154:0/1115152785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd188152cc0 0x7fd1881506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 >> 172.21.15.154:0/1115152785 conn(0x7fd1880a9bf0 msgr2=0x7fd1880c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 shutdown_connections 2024-03-14T09:03:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.123+0000 7fd18dea6700 1 -- 172.21.15.154:0/1115152785 wait complete. 2024-03-14T09:03:18.135 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-14T09:03:18.135 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.255+0000 7f6c4b966700 1 Processor -- start 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 -- start start 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6c440702f0 0x7f6c440c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c4414fbd0 0x7f6c440c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440acd10 con 0x7f6c440c78c0 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440ac7b0 con 0x7f6c4414fbd0 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c4b966700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c44150050 con 0x7f6c440702f0 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c48f01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c4414fbd0 0x7f6c440c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c49f03700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6c440c78c0 0x7f6c440c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35258/0 (socket says 172.21.15.154:35258) 2024-03-14T09:03:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.256+0000 7f6c49f03700 1 -- 172.21.15.154:0/774738883 learned_addr learned my addr 172.21.15.154:0/774738883 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c49702700 1 --2- 172.21.15.154:0/774738883 >> v2:172.21.15.191:3300/0 conn(0x7f6c440702f0 0x7f6c440c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c48f01700 1 -- 172.21.15.154:0/774738883 >> v2:172.21.15.191:3300/0 conn(0x7f6c440702f0 msgr2=0x7f6c440c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c48f01700 1 --2- 172.21.15.154:0/774738883 >> v2:172.21.15.191:3300/0 conn(0x7f6c440702f0 0x7f6c440c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c48f01700 1 -- 172.21.15.154:0/774738883 >> v1:172.21.15.154:6789/0 conn(0x7f6c440c78c0 legacy=0x7f6c440c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c48f01700 1 -- 172.21.15.154:0/774738883 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440ac9c0 con 0x7f6c4414fbd0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c37fff700 1 -- 172.21.15.154:0/774738883 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397794152 0 0) 0x7f6c440acd10 con 0x7f6c440c78c0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c48f01700 1 --2- 172.21.15.154:0/774738883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4414fbd0 0x7f6c440c7380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6c440c8c00 tx=0x7f6c40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c389d1320bfc755 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c37fff700 1 -- 172.21.15.154:0/774738883 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40011830 con 0x7f6c4414fbd0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c37fff700 1 -- 172.21.15.154:0/774738883 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40011990 con 0x7f6c4414fbd0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.257+0000 7f6c37fff700 1 -- 172.21.15.154:0/774738883 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40011d20 con 0x7f6c4414fbd0 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 -- 172.21.15.154:0/774738883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4414fbd0 msgr2=0x7f6c440c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 --2- 172.21.15.154:0/774738883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4414fbd0 0x7f6c440c7380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6c440c8c00 tx=0x7f6c40009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 -- 172.21.15.154:0/774738883 shutdown_connections 2024-03-14T09:03:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 --2- 172.21.15.154:0/774738883 >> v2:172.21.15.191:3300/0 conn(0x7f6c440702f0 0x7f6c440c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 --2- 172.21.15.154:0/774738883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c4414fbd0 0x7f6c440c7380 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 -- 172.21.15.154:0/774738883 >> 172.21.15.154:0/774738883 conn(0x7f6c440a9bf0 msgr2=0x7f6c440a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 -- 172.21.15.154:0/774738883 shutdown_connections 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.258+0000 7f6c4b966700 1 -- 172.21.15.154:0/774738883 wait complete. 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.259+0000 7f6c4b966700 1 Processor -- start 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.259+0000 7f6c4b966700 1 -- start start 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c440702f0 0x7f6c4406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c4414fbd0 0x7f6c44145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44150590 con 0x7f6c440c78c0 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c5420 con 0x7f6c440702f0 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44153880 con 0x7f6c4414fbd0 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56378/0 (socket says 172.21.15.154:56378) 2024-03-14T09:03:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 -- 172.21.15.154:0/743384365 learned_addr learned my addr 172.21.15.154:0/743384365 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c49f03700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c4414fbd0 0x7f6c44145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c49702700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c440702f0 0x7f6c4406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 -- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c4414fbd0 msgr2=0x7f6c44145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c4414fbd0 0x7f6c44145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 -- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c440702f0 msgr2=0x7f6c4406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c440702f0 0x7f6c4406f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 -- 172.21.15.154:0/743384365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400094a0 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c48f01700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6c40014040 tx=0x7f6c40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400038a0 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4001e070 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c441e8fd0 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.260+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003c20 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.261+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c441e92b0 con 0x7f6c440c78c0 2024-03-14T09:03:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.261+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c44150700 con 0x7f6c440c78c0 2024-03-14T09:03:18.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.262+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6c4002d450 con 0x7f6c440c78c0 2024-03-14T09:03:18.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.263+0000 7f6c35ffb700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 0x7f6c2806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:18.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.263+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f6c4008a0c0 con 0x7f6c440c78c0 2024-03-14T09:03:18.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.263+0000 7f6c49702700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 0x7f6c2806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:18.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.263+0000 7f6c49702700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 0x7f6c2806ccf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009930 tx=0x7f6c38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:18.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.267+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6c44150700 con 0x7f6c440c78c0 2024-03-14T09:03:18.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.459+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c441e8e50 con 0x7f6c2806a830 2024-03-14T09:03:18.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.473+0000 7f6c35ffb700 1 -- 172.21.15.154:0/743384365 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524924 (secure 0 0 0) 0x7f6c38009e00 con 0x7f6c2806a830 2024-03-14T09:03:18.479 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 msgr2=0x7f6c2806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 0x7f6c2806ccf0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009930 tx=0x7f6c38008040 comp rx=0 tx=0).stop 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 msgr2=0x7f6c44144f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6c40014040 tx=0x7f6c40004750 comp rx=0 tx=0).stop 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 shutdown_connections 2024-03-14T09:03:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c4414fbd0 0x7f6c44145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c440702f0 0x7f6c4406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c2806a830 0x7f6c2806ccf0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 --2- 172.21.15.154:0/743384365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c440c78c0 0x7f6c44144f40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.477+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 >> 172.21.15.154:0/743384365 conn(0x7f6c440a9bf0 msgr2=0x7f6c4414e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.478+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 shutdown_connections 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:18.478+0000 7f6c4b966700 1 -- 172.21.15.154:0/743384365 wait complete. 2024-03-14T09:03:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:18.490 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-14T09:03:17.545379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594241,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":199,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222312,"kb_used_data":6136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752408,"statfs":{"total":773094113280,"available":772866465792,"internally_reserved":0,"allocated":6283264,"data_stored":3329522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001339"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353917+0000","last_change":"2024-03-14T09:03:12.353917+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353917+0000","last_undegraded":"2024-03-14T09:03:12.353917+0000","last_fullsized":"2024-03-14T09:03:12.353917+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353360+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:11.353360+0000","last_peered":"2024-03-14T09:03:11.353360+0000","last_clean":"2024-03-14T09:03:11.353360+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:11.353360+0000","last_undegraded":"2024-03-14T09:03:11.353360+0000","last_fullsized":"2024-03-14T09:03:11.353360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361638+0000","last_change":"2024-03-14T09:03:12.361638+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361638+0000","last_undegraded":"2024-03-14T09:03:12.361638+0000","last_fullsized":"2024-03-14T09:03:12.361638+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353404+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:11.353404+0000","last_peered":"2024-03-14T09:03:11.353404+0000","last_clean":"2024-03-14T09:03:11.353404+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:11.353404+0000","last_undegraded":"2024-03-14T09:03:11.353404+0000","last_fullsized":"2024-03-14T09:03:11.353404+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358323+0000","last_change":"2024-03-14T09:03:12.358323+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358323+0000","last_undegraded":"2024-03-14T09:03:12.358323+0000","last_fullsized":"2024-03-14T09:03:12.358323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358248+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:11.358248+0000","last_peered":"2024-03-14T09:03:11.358248+0000","last_clean":"2024-03-14T09:03:11.358248+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:11.358248+0000","last_undegraded":"2024-03-14T09:03:11.358248+0000","last_fullsized":"2024-03-14T09:03:11.358248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354698+0000","last_change":"2024-03-14T09:03:12.354698+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354698+0000","last_undegraded":"2024-03-14T09:03:12.354698+0000","last_fullsized":"2024-03-14T09:03:12.354698+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355314+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:11.355314+0000","last_peered":"2024-03-14T09:03:11.355314+0000","last_clean":"2024-03-14T09:03:11.355314+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:11.355314+0000","last_undegraded":"2024-03-14T09:03:11.355314+0000","last_fullsized":"2024-03-14T09:03:11.355314+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362720+0000","last_change":"2024-03-14T09:03:12.362720+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362720+0000","last_undegraded":"2024-03-14T09:03:12.362720+0000","last_fullsized":"2024-03-14T09:03:12.362720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353370+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:11.353370+0000","last_peered":"2024-03-14T09:03:11.353370+0000","last_clean":"2024-03-14T09:03:11.353370+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:11.353370+0000","last_undegraded":"2024-03-14T09:03:11.353370+0000","last_fullsized":"2024-03-14T09:03:11.353370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353623+0000","last_change":"2024-03-14T09:03:12.353623+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353623+0000","last_undegraded":"2024-03-14T09:03:12.353623+0000","last_fullsized":"2024-03-14T09:03:12.353623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349110+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:11.349110+0000","last_peered":"2024-03-14T09:03:11.349110+0000","last_clean":"2024-03-14T09:03:11.349110+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:11.349110+0000","last_undegraded":"2024-03-14T09:03:11.349110+0000","last_fullsized":"2024-03-14T09:03:11.349110+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360227+0000","last_change":"2024-03-14T09:03:12.360227+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360227+0000","last_undegraded":"2024-03-14T09:03:12.360227+0000","last_fullsized":"2024-03-14T09:03:12.360227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355387+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:11.355387+0000","last_peered":"2024-03-14T09:03:11.355387+0000","last_clean":"2024-03-14T09:03:11.355387+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:11.355387+0000","last_undegraded":"2024-03-14T09:03:11.355387+0000","last_fullsized":"2024-03-14T09:03:11.355387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353748+0000","last_change":"2024-03-14T09:03:12.353748+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353748+0000","last_undegraded":"2024-03-14T09:03:12.353748+0000","last_fullsized":"2024-03-14T09:03:12.353748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351097+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:11.351097+0000","last_peered":"2024-03-14T09:03:11.351097+0000","last_clean":"2024-03-14T09:03:11.351097+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:11.351097+0000","last_undegraded":"2024-03-14T09:03:11.351097+0000","last_fullsized":"2024-03-14T09:03:11.351097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357388+0000","last_change":"2024-03-14T09:03:12.357388+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357388+0000","last_undegraded":"2024-03-14T09:03:12.357388+0000","last_fullsized":"2024-03-14T09:03:12.357388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356978+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:11.356978+0000","last_peered":"2024-03-14T09:03:11.356978+0000","last_clean":"2024-03-14T09:03:11.356978+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:11.356978+0000","last_undegraded":"2024-03-14T09:03:11.356978+0000","last_fullsized":"2024-03-14T09:03:11.356978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362629+0000","last_change":"2024-03-14T09:03:12.362629+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362629+0000","last_undegraded":"2024-03-14T09:03:12.362629+0000","last_fullsized":"2024-03-14T09:03:12.362629+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357377+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:11.357377+0000","last_peered":"2024-03-14T09:03:11.357377+0000","last_clean":"2024-03-14T09:03:11.357377+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:11.357377+0000","last_undegraded":"2024-03-14T09:03:11.357377+0000","last_fullsized":"2024-03-14T09:03:11.357377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359448+0000","last_change":"2024-03-14T09:03:12.359448+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359448+0000","last_undegraded":"2024-03-14T09:03:12.359448+0000","last_fullsized":"2024-03-14T09:03:12.359448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357336+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:11.357336+0000","last_peered":"2024-03-14T09:03:11.357336+0000","last_clean":"2024-03-14T09:03:11.357336+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:11.357336+0000","last_undegraded":"2024-03-14T09:03:11.357336+0000","last_fullsized":"2024-03-14T09:03:11.357336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364654+0000","last_change":"2024-03-14T09:03:12.364654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364654+0000","last_undegraded":"2024-03-14T09:03:12.364654+0000","last_fullsized":"2024-03-14T09:03:12.364654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355433+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:11.355433+0000","last_peered":"2024-03-14T09:03:11.355433+0000","last_clean":"2024-03-14T09:03:11.355433+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:11.355433+0000","last_undegraded":"2024-03-14T09:03:11.355433+0000","last_fullsized":"2024-03-14T09:03:11.355433+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356123+0000","last_change":"2024-03-14T09:03:12.356123+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356123+0000","last_undegraded":"2024-03-14T09:03:12.356123+0000","last_fullsized":"2024-03-14T09:03:12.356123+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353947+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:11.353947+0000","last_peered":"2024-03-14T09:03:11.353947+0000","last_clean":"2024-03-14T09:03:11.353947+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:11.353947+0000","last_undegraded":"2024-03-14T09:03:11.353947+0000","last_fullsized":"2024-03-14T09:03:11.353947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361949+0000","last_change":"2024-03-14T09:03:12.361949+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361949+0000","last_undegraded":"2024-03-14T09:03:12.361949+0000","last_fullsized":"2024-03-14T09:03:12.361949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355064+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:11.355064+0000","last_peered":"2024-03-14T09:03:11.355064+0000","last_clean":"2024-03-14T09:03:11.355064+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:11.355064+0000","last_undegraded":"2024-03-14T09:03:11.355064+0000","last_fullsized":"2024-03-14T09:03:11.355064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355497+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:11.355497+0000","last_peered":"2024-03-14T09:03:11.355497+0000","last_clean":"2024-03-14T09:03:11.355497+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:11.355497+0000","last_undegraded":"2024-03-14T09:03:11.355497+0000","last_fullsized":"2024-03-14T09:03:11.355497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364020+0000","last_change":"2024-03-14T09:03:12.364020+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364020+0000","last_undegraded":"2024-03-14T09:03:12.364020+0000","last_fullsized":"2024-03-14T09:03:12.364020+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.348935+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:11.348935+0000","last_peered":"2024-03-14T09:03:11.348935+0000","last_clean":"2024-03-14T09:03:11.348935+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:11.348935+0000","last_undegraded":"2024-03-14T09:03:11.348935+0000","last_fullsized":"2024-03-14T09:03:11.348935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353377+0000","last_change":"2024-03-14T09:03:12.353377+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353377+0000","last_undegraded":"2024-03-14T09:03:12.353377+0000","last_fullsized":"2024-03-14T09:03:12.353377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353394+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:11.353394+0000","last_peered":"2024-03-14T09:03:11.353394+0000","last_clean":"2024-03-14T09:03:11.353394+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:11.353394+0000","last_undegraded":"2024-03-14T09:03:11.353394+0000","last_fullsized":"2024-03-14T09:03:11.353394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356427+0000","last_change":"2024-03-14T09:03:12.356427+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356427+0000","last_undegraded":"2024-03-14T09:03:12.356427+0000","last_fullsized":"2024-03-14T09:03:12.356427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358442+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:11.358442+0000","last_peered":"2024-03-14T09:03:11.358442+0000","last_clean":"2024-03-14T09:03:11.358442+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:11.358442+0000","last_undegraded":"2024-03-14T09:03:11.358442+0000","last_fullsized":"2024-03-14T09:03:11.358442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360390+0000","last_change":"2024-03-14T09:03:12.360390+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360390+0000","last_undegraded":"2024-03-14T09:03:12.360390+0000","last_fullsized":"2024-03-14T09:03:12.360390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353260+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:11.353260+0000","last_peered":"2024-03-14T09:03:11.353260+0000","last_clean":"2024-03-14T09:03:11.353260+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:11.353260+0000","last_undegraded":"2024-03-14T09:03:11.353260+0000","last_fullsized":"2024-03-14T09:03:11.353260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359276+0000","last_change":"2024-03-14T09:03:12.359276+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359276+0000","last_undegraded":"2024-03-14T09:03:12.359276+0000","last_fullsized":"2024-03-14T09:03:12.359276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351650+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:11.351650+0000","last_peered":"2024-03-14T09:03:11.351650+0000","last_clean":"2024-03-14T09:03:11.351650+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:11.351650+0000","last_undegraded":"2024-03-14T09:03:11.351650+0000","last_fullsized":"2024-03-14T09:03:11.351650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356216+0000","last_change":"2024-03-14T09:03:12.356216+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356216+0000","last_undegraded":"2024-03-14T09:03:12.356216+0000","last_fullsized":"2024-03-14T09:03:12.356216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358150+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:11.358150+0000","last_peered":"2024-03-14T09:03:11.358150+0000","last_clean":"2024-03-14T09:03:11.358150+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:11.358150+0000","last_undegraded":"2024-03-14T09:03:11.358150+0000","last_fullsized":"2024-03-14T09:03:11.358150+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357282+0000","last_change":"2024-03-14T09:03:12.357282+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357282+0000","last_undegraded":"2024-03-14T09:03:12.357282+0000","last_fullsized":"2024-03-14T09:03:12.357282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351242+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:11.351242+0000","last_peered":"2024-03-14T09:03:11.351242+0000","last_clean":"2024-03-14T09:03:11.351242+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:11.351242+0000","last_undegraded":"2024-03-14T09:03:11.351242+0000","last_fullsized":"2024-03-14T09:03:11.351242+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364628+0000","last_change":"2024-03-14T09:03:12.364628+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364628+0000","last_undegraded":"2024-03-14T09:03:12.364628+0000","last_fullsized":"2024-03-14T09:03:12.364628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358083+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:11.358083+0000","last_peered":"2024-03-14T09:03:11.358083+0000","last_clean":"2024-03-14T09:03:11.358083+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:11.358083+0000","last_undegraded":"2024-03-14T09:03:11.358083+0000","last_fullsized":"2024-03-14T09:03:11.358083+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362012+0000","last_change":"2024-03-14T09:03:12.362012+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362012+0000","last_undegraded":"2024-03-14T09:03:12.362012+0000","last_fullsized":"2024-03-14T09:03:12.362012+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353228+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.353228+0000","last_peered":"2024-03-14T09:03:11.353228+0000","last_clean":"2024-03-14T09:03:11.353228+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:11.353228+0000","last_undegraded":"2024-03-14T09:03:11.353228+0000","last_fullsized":"2024-03-14T09:03:11.353228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352812+0000","last_change":"2024-03-14T09:03:12.352812+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352812+0000","last_undegraded":"2024-03-14T09:03:12.352812+0000","last_fullsized":"2024-03-14T09:03:12.352812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353434+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:11.353434+0000","last_peered":"2024-03-14T09:03:11.353434+0000","last_clean":"2024-03-14T09:03:11.353434+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:11.353434+0000","last_undegraded":"2024-03-14T09:03:11.353434+0000","last_fullsized":"2024-03-14T09:03:11.353434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361474+0000","last_change":"2024-03-14T09:03:12.361474+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361474+0000","last_undegraded":"2024-03-14T09:03:12.361474+0000","last_fullsized":"2024-03-14T09:03:12.361474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355459+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:11.355459+0000","last_peered":"2024-03-14T09:03:11.355459+0000","last_clean":"2024-03-14T09:03:11.355459+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:11.355459+0000","last_undegraded":"2024-03-14T09:03:11.355459+0000","last_fullsized":"2024-03-14T09:03:11.355459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361988+0000","last_change":"2024-03-14T09:03:12.361988+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361988+0000","last_undegraded":"2024-03-14T09:03:12.361988+0000","last_fullsized":"2024-03-14T09:03:12.361988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362109+0000","last_change":"2024-03-14T09:03:12.362109+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362109+0000","last_undegraded":"2024-03-14T09:03:12.362109+0000","last_fullsized":"2024-03-14T09:03:12.362109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353264+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:11.353264+0000","last_peered":"2024-03-14T09:03:11.353264+0000","last_clean":"2024-03-14T09:03:11.353264+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:11.353264+0000","last_undegraded":"2024-03-14T09:03:11.353264+0000","last_fullsized":"2024-03-14T09:03:11.353264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.362252+0000","last_change":"2024-03-14T09:03:12.362252+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.362252+0000","last_undegraded":"2024-03-14T09:03:12.362252+0000","last_fullsized":"2024-03-14T09:03:12.362252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351526+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:11.351526+0000","last_peered":"2024-03-14T09:03:11.351526+0000","last_clean":"2024-03-14T09:03:11.351526+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:11.351526+0000","last_undegraded":"2024-03-14T09:03:11.351526+0000","last_fullsized":"2024-03-14T09:03:11.351526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353125+0000","last_change":"2024-03-14T09:03:12.353125+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353125+0000","last_undegraded":"2024-03-14T09:03:12.353125+0000","last_fullsized":"2024-03-14T09:03:12.353125+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358400+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:11.358400+0000","last_peered":"2024-03-14T09:03:11.358400+0000","last_clean":"2024-03-14T09:03:11.358400+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:11.358400+0000","last_undegraded":"2024-03-14T09:03:11.358400+0000","last_fullsized":"2024-03-14T09:03:11.358400+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354173+0000","last_change":"2024-03-14T09:03:12.354173+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354173+0000","last_undegraded":"2024-03-14T09:03:12.354173+0000","last_fullsized":"2024-03-14T09:03:12.354173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.352840+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:11.352840+0000","last_peered":"2024-03-14T09:03:11.352840+0000","last_clean":"2024-03-14T09:03:11.352840+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:11.352840+0000","last_undegraded":"2024-03-14T09:03:11.352840+0000","last_fullsized":"2024-03-14T09:03:11.352840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361509+0000","last_change":"2024-03-14T09:03:12.361509+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361509+0000","last_undegraded":"2024-03-14T09:03:12.361509+0000","last_fullsized":"2024-03-14T09:03:12.361509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349002+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:11.349002+0000","last_peered":"2024-03-14T09:03:11.349002+0000","last_clean":"2024-03-14T09:03:11.349002+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:11.349002+0000","last_undegraded":"2024-03-14T09:03:11.349002+0000","last_fullsized":"2024-03-14T09:03:11.349002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354070+0000","last_change":"2024-03-14T09:03:12.354070+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354070+0000","last_undegraded":"2024-03-14T09:03:12.354070+0000","last_fullsized":"2024-03-14T09:03:12.354070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376057+0000","last_change":"2024-03-14T09:03:12.371272+0000","last_active":"2024-03-14T09:03:12.376057+0000","last_peered":"2024-03-14T09:03:12.376057+0000","last_clean":"2024-03-14T09:03:12.376057+0000","last_became_active":"2024-03-14T09:03:12.370996+0000","last_became_peered":"2024-03-14T09:03:12.370996+0000","last_unstale":"2024-03-14T09:03:12.376057+0000","last_undegraded":"2024-03-14T09:03:12.376057+0000","last_fullsized":"2024-03-14T09:03:12.376057+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:44.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349268+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:11.349268+0000","last_peered":"2024-03-14T09:03:11.349268+0000","last_clean":"2024-03-14T09:03:11.349268+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:11.349268+0000","last_undegraded":"2024-03-14T09:03:11.349268+0000","last_fullsized":"2024-03-14T09:03:11.349268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349423+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:11.349423+0000","last_peered":"2024-03-14T09:03:11.349423+0000","last_clean":"2024-03-14T09:03:11.349423+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:11.349423+0000","last_undegraded":"2024-03-14T09:03:11.349423+0000","last_fullsized":"2024-03-14T09:03:11.349423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356924+0000","last_change":"2024-03-14T09:03:12.356924+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356924+0000","last_undegraded":"2024-03-14T09:03:12.356924+0000","last_fullsized":"2024-03-14T09:03:12.356924+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378773+0000","last_change":"2024-03-14T09:03:12.374278+0000","last_active":"2024-03-14T09:03:12.378773+0000","last_peered":"2024-03-14T09:03:12.378773+0000","last_clean":"2024-03-14T09:03:12.378773+0000","last_became_active":"2024-03-14T09:03:12.373968+0000","last_became_peered":"2024-03-14T09:03:12.373968+0000","last_unstale":"2024-03-14T09:03:12.378773+0000","last_undegraded":"2024-03-14T09:03:12.378773+0000","last_fullsized":"2024-03-14T09:03:12.378773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:40.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358207+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:11.358207+0000","last_peered":"2024-03-14T09:03:11.358207+0000","last_clean":"2024-03-14T09:03:11.358207+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:11.358207+0000","last_undegraded":"2024-03-14T09:03:11.358207+0000","last_fullsized":"2024-03-14T09:03:11.358207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353512+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:11.353512+0000","last_peered":"2024-03-14T09:03:11.353512+0000","last_clean":"2024-03-14T09:03:11.353512+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:11.353512+0000","last_undegraded":"2024-03-14T09:03:11.353512+0000","last_fullsized":"2024-03-14T09:03:11.353512+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356192+0000","last_change":"2024-03-14T09:03:12.356192+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356192+0000","last_undegraded":"2024-03-14T09:03:12.356192+0000","last_fullsized":"2024-03-14T09:03:12.356192+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377433+0000","last_change":"2024-03-14T09:03:12.373911+0000","last_active":"2024-03-14T09:03:12.377433+0000","last_peered":"2024-03-14T09:03:12.377433+0000","last_clean":"2024-03-14T09:03:12.377433+0000","last_became_active":"2024-03-14T09:03:12.373633+0000","last_became_peered":"2024-03-14T09:03:12.373633+0000","last_unstale":"2024-03-14T09:03:12.377433+0000","last_undegraded":"2024-03-14T09:03:12.377433+0000","last_fullsized":"2024-03-14T09:03:12.377433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349196+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:11.349196+0000","last_peered":"2024-03-14T09:03:11.349196+0000","last_clean":"2024-03-14T09:03:11.349196+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:11.349196+0000","last_undegraded":"2024-03-14T09:03:11.349196+0000","last_fullsized":"2024-03-14T09:03:11.349196+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349440+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:11.349440+0000","last_peered":"2024-03-14T09:03:11.349440+0000","last_clean":"2024-03-14T09:03:11.349440+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:11.349440+0000","last_undegraded":"2024-03-14T09:03:11.349440+0000","last_fullsized":"2024-03-14T09:03:11.349440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359921+0000","last_change":"2024-03-14T09:03:12.359921+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359921+0000","last_undegraded":"2024-03-14T09:03:12.359921+0000","last_fullsized":"2024-03-14T09:03:12.359921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371201+0000","last_change":"2024-03-14T09:03:12.370015+0000","last_active":"2024-03-14T09:03:12.371201+0000","last_peered":"2024-03-14T09:03:12.371201+0000","last_clean":"2024-03-14T09:03:12.371201+0000","last_became_active":"2024-03-14T09:03:12.369735+0000","last_became_peered":"2024-03-14T09:03:12.369735+0000","last_unstale":"2024-03-14T09:03:12.371201+0000","last_undegraded":"2024-03-14T09:03:12.371201+0000","last_fullsized":"2024-03-14T09:03:12.371201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349286+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:11.349286+0000","last_peered":"2024-03-14T09:03:11.349286+0000","last_clean":"2024-03-14T09:03:11.349286+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:11.349286+0000","last_undegraded":"2024-03-14T09:03:11.349286+0000","last_fullsized":"2024-03-14T09:03:11.349286+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355554+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:11.355554+0000","last_peered":"2024-03-14T09:03:11.355554+0000","last_clean":"2024-03-14T09:03:11.355554+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:11.355554+0000","last_undegraded":"2024-03-14T09:03:11.355554+0000","last_fullsized":"2024-03-14T09:03:11.355554+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360660+0000","last_change":"2024-03-14T09:03:12.360660+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360660+0000","last_undegraded":"2024-03-14T09:03:12.360660+0000","last_fullsized":"2024-03-14T09:03:12.360660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375005+0000","last_change":"2024-03-14T09:03:12.372188+0000","last_active":"2024-03-14T09:03:12.375005+0000","last_peered":"2024-03-14T09:03:12.375005+0000","last_clean":"2024-03-14T09:03:12.375005+0000","last_became_active":"2024-03-14T09:03:12.371919+0000","last_became_peered":"2024-03-14T09:03:12.371919+0000","last_unstale":"2024-03-14T09:03:12.375005+0000","last_undegraded":"2024-03-14T09:03:12.375005+0000","last_fullsized":"2024-03-14T09:03:12.375005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351321+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:11.351321+0000","last_peered":"2024-03-14T09:03:11.351321+0000","last_clean":"2024-03-14T09:03:11.351321+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:11.351321+0000","last_undegraded":"2024-03-14T09:03:11.351321+0000","last_fullsized":"2024-03-14T09:03:11.351321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355947+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:11.355947+0000","last_peered":"2024-03-14T09:03:11.355947+0000","last_clean":"2024-03-14T09:03:11.355947+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:11.355947+0000","last_undegraded":"2024-03-14T09:03:11.355947+0000","last_fullsized":"2024-03-14T09:03:11.355947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355339+0000","last_change":"2024-03-14T09:03:12.355339+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355339+0000","last_undegraded":"2024-03-14T09:03:12.355339+0000","last_fullsized":"2024-03-14T09:03:12.355339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372079+0000","last_change":"2024-03-14T09:03:12.369918+0000","last_active":"2024-03-14T09:03:12.372079+0000","last_peered":"2024-03-14T09:03:12.372079+0000","last_clean":"2024-03-14T09:03:12.372079+0000","last_became_active":"2024-03-14T09:03:12.369599+0000","last_became_peered":"2024-03-14T09:03:12.369599+0000","last_unstale":"2024-03-14T09:03:12.372079+0000","last_undegraded":"2024-03-14T09:03:12.372079+0000","last_fullsized":"2024-03-14T09:03:12.372079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:52.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351535+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:11.351535+0000","last_peered":"2024-03-14T09:03:11.351535+0000","last_clean":"2024-03-14T09:03:11.351535+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:11.351535+0000","last_undegraded":"2024-03-14T09:03:11.351535+0000","last_fullsized":"2024-03-14T09:03:11.351535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349367+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:11.349367+0000","last_peered":"2024-03-14T09:03:11.349367+0000","last_clean":"2024-03-14T09:03:11.349367+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:11.349367+0000","last_undegraded":"2024-03-14T09:03:11.349367+0000","last_fullsized":"2024-03-14T09:03:11.349367+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361025+0000","last_change":"2024-03-14T09:03:12.361025+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361025+0000","last_undegraded":"2024-03-14T09:03:12.361025+0000","last_fullsized":"2024-03-14T09:03:12.361025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374895+0000","last_change":"2024-03-14T09:03:12.373841+0000","last_active":"2024-03-14T09:03:12.374895+0000","last_peered":"2024-03-14T09:03:12.374895+0000","last_clean":"2024-03-14T09:03:12.374895+0000","last_became_active":"2024-03-14T09:03:12.373551+0000","last_became_peered":"2024-03-14T09:03:12.373551+0000","last_unstale":"2024-03-14T09:03:12.374895+0000","last_undegraded":"2024-03-14T09:03:12.374895+0000","last_fullsized":"2024-03-14T09:03:12.374895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.965540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353268+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:11.353268+0000","last_peered":"2024-03-14T09:03:11.353268+0000","last_clean":"2024-03-14T09:03:11.353268+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:11.353268+0000","last_undegraded":"2024-03-14T09:03:11.353268+0000","last_fullsized":"2024-03-14T09:03:11.353268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355710+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:11.355710+0000","last_peered":"2024-03-14T09:03:11.355710+0000","last_clean":"2024-03-14T09:03:11.355710+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:11.355710+0000","last_undegraded":"2024-03-14T09:03:11.355710+0000","last_fullsized":"2024-03-14T09:03:11.355710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361000+0000","last_change":"2024-03-14T09:03:12.361000+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361000+0000","last_undegraded":"2024-03-14T09:03:12.361000+0000","last_fullsized":"2024-03-14T09:03:12.361000+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368944+0000","last_change":"2024-03-14T09:03:12.366900+0000","last_active":"2024-03-14T09:03:12.368944+0000","last_peered":"2024-03-14T09:03:12.368944+0000","last_clean":"2024-03-14T09:03:12.368944+0000","last_became_active":"2024-03-14T09:03:12.366301+0000","last_became_peered":"2024-03-14T09:03:12.366301+0000","last_unstale":"2024-03-14T09:03:12.368944+0000","last_undegraded":"2024-03-14T09:03:12.368944+0000","last_fullsized":"2024-03-14T09:03:12.368944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:11.884116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354744+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:11.354744+0000","last_peered":"2024-03-14T09:03:11.354744+0000","last_clean":"2024-03-14T09:03:11.354744+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:11.354744+0000","last_undegraded":"2024-03-14T09:03:11.354744+0000","last_fullsized":"2024-03-14T09:03:11.354744+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353503+0000","last_change":"2024-03-14T09:03:12.353503+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353503+0000","last_undegraded":"2024-03-14T09:03:12.353503+0000","last_fullsized":"2024-03-14T09:03:12.353503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371091+0000","last_change":"2024-03-14T09:03:12.367305+0000","last_active":"2024-03-14T09:03:12.371091+0000","last_peered":"2024-03-14T09:03:12.371091+0000","last_clean":"2024-03-14T09:03:12.371091+0000","last_became_active":"2024-03-14T09:03:12.367020+0000","last_became_peered":"2024-03-14T09:03:12.367020+0000","last_unstale":"2024-03-14T09:03:12.371091+0000","last_undegraded":"2024-03-14T09:03:12.371091+0000","last_fullsized":"2024-03-14T09:03:12.371091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:39.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356736+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:11.356736+0000","last_peered":"2024-03-14T09:03:11.356736+0000","last_clean":"2024-03-14T09:03:11.356736+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:11.356736+0000","last_undegraded":"2024-03-14T09:03:11.356736+0000","last_fullsized":"2024-03-14T09:03:11.356736+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358626+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:11.358626+0000","last_peered":"2024-03-14T09:03:11.358626+0000","last_clean":"2024-03-14T09:03:11.358626+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:11.358626+0000","last_undegraded":"2024-03-14T09:03:11.358626+0000","last_fullsized":"2024-03-14T09:03:11.358626+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364515+0000","last_change":"2024-03-14T09:03:12.364515+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364515+0000","last_undegraded":"2024-03-14T09:03:12.364515+0000","last_fullsized":"2024-03-14T09:03:12.364515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373487+0000","last_change":"2024-03-14T09:03:12.370627+0000","last_active":"2024-03-14T09:03:12.373487+0000","last_peered":"2024-03-14T09:03:12.373487+0000","last_clean":"2024-03-14T09:03:12.373487+0000","last_became_active":"2024-03-14T09:03:12.370314+0000","last_became_peered":"2024-03-14T09:03:12.370314+0000","last_unstale":"2024-03-14T09:03:12.373487+0000","last_undegraded":"2024-03-14T09:03:12.373487+0000","last_fullsized":"2024-03-14T09:03:12.373487+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:09.613459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358558+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:11.358558+0000","last_peered":"2024-03-14T09:03:11.358558+0000","last_clean":"2024-03-14T09:03:11.358558+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:11.358558+0000","last_undegraded":"2024-03-14T09:03:11.358558+0000","last_fullsized":"2024-03-14T09:03:11.358558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353599+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:11.353599+0000","last_peered":"2024-03-14T09:03:11.353599+0000","last_clean":"2024-03-14T09:03:11.353599+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:11.353599+0000","last_undegraded":"2024-03-14T09:03:11.353599+0000","last_fullsized":"2024-03-14T09:03:11.353599+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359668+0000","last_change":"2024-03-14T09:03:12.359668+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359668+0000","last_undegraded":"2024-03-14T09:03:12.359668+0000","last_fullsized":"2024-03-14T09:03:12.359668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.378802+0000","last_change":"2024-03-14T09:03:12.374028+0000","last_active":"2024-03-14T09:03:12.378802+0000","last_peered":"2024-03-14T09:03:12.378802+0000","last_clean":"2024-03-14T09:03:12.378802+0000","last_became_active":"2024-03-14T09:03:12.373770+0000","last_became_peered":"2024-03-14T09:03:12.373770+0000","last_unstale":"2024-03-14T09:03:12.378802+0000","last_undegraded":"2024-03-14T09:03:12.378802+0000","last_fullsized":"2024-03-14T09:03:12.378802+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:51.114973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357121+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:11.357121+0000","last_peered":"2024-03-14T09:03:11.357121+0000","last_clean":"2024-03-14T09:03:11.357121+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:11.357121+0000","last_undegraded":"2024-03-14T09:03:11.357121+0000","last_fullsized":"2024-03-14T09:03:11.357121+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358587+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:11.358587+0000","last_peered":"2024-03-14T09:03:11.358587+0000","last_clean":"2024-03-14T09:03:11.358587+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:11.358587+0000","last_undegraded":"2024-03-14T09:03:11.358587+0000","last_fullsized":"2024-03-14T09:03:11.358587+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357946+0000","last_change":"2024-03-14T09:03:12.357946+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357946+0000","last_undegraded":"2024-03-14T09:03:12.357946+0000","last_fullsized":"2024-03-14T09:03:12.357946+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368918+0000","last_change":"2024-03-14T09:03:12.367742+0000","last_active":"2024-03-14T09:03:12.368918+0000","last_peered":"2024-03-14T09:03:12.368918+0000","last_clean":"2024-03-14T09:03:12.368918+0000","last_became_active":"2024-03-14T09:03:12.367471+0000","last_became_peered":"2024-03-14T09:03:12.367471+0000","last_unstale":"2024-03-14T09:03:12.368918+0000","last_undegraded":"2024-03-14T09:03:12.368918+0000","last_fullsized":"2024-03-14T09:03:12.368918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:34.678842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358346+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:11.358346+0000","last_peered":"2024-03-14T09:03:11.358346+0000","last_clean":"2024-03-14T09:03:11.358346+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:11.358346+0000","last_undegraded":"2024-03-14T09:03:11.358346+0000","last_fullsized":"2024-03-14T09:03:11.358346+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356446+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:11.356446+0000","last_peered":"2024-03-14T09:03:11.356446+0000","last_clean":"2024-03-14T09:03:11.356446+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:11.356446+0000","last_undegraded":"2024-03-14T09:03:11.356446+0000","last_fullsized":"2024-03-14T09:03:11.356446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.355535+0000","last_change":"2024-03-14T09:03:12.355535+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.355535+0000","last_undegraded":"2024-03-14T09:03:12.355535+0000","last_fullsized":"2024-03-14T09:03:12.355535+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371980+0000","last_change":"2024-03-14T09:03:12.366980+0000","last_active":"2024-03-14T09:03:12.371980+0000","last_peered":"2024-03-14T09:03:12.371980+0000","last_clean":"2024-03-14T09:03:12.371980+0000","last_became_active":"2024-03-14T09:03:12.366728+0000","last_became_peered":"2024-03-14T09:03:12.366728+0000","last_unstale":"2024-03-14T09:03:12.371980+0000","last_undegraded":"2024-03-14T09:03:12.371980+0000","last_fullsized":"2024-03-14T09:03:12.371980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:53.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355135+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:11.355135+0000","last_peered":"2024-03-14T09:03:11.355135+0000","last_clean":"2024-03-14T09:03:11.355135+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:11.355135+0000","last_undegraded":"2024-03-14T09:03:11.355135+0000","last_fullsized":"2024-03-14T09:03:11.355135+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351630+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:11.351630+0000","last_peered":"2024-03-14T09:03:11.351630+0000","last_clean":"2024-03-14T09:03:11.351630+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:11.351630+0000","last_undegraded":"2024-03-14T09:03:11.351630+0000","last_fullsized":"2024-03-14T09:03:11.351630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358690+0000","last_change":"2024-03-14T09:03:12.358690+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358690+0000","last_undegraded":"2024-03-14T09:03:12.358690+0000","last_fullsized":"2024-03-14T09:03:12.358690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376094+0000","last_change":"2024-03-14T09:03:12.371369+0000","last_active":"2024-03-14T09:03:12.376094+0000","last_peered":"2024-03-14T09:03:12.376094+0000","last_clean":"2024-03-14T09:03:12.376094+0000","last_became_active":"2024-03-14T09:03:12.371087+0000","last_became_peered":"2024-03-14T09:03:12.371087+0000","last_unstale":"2024-03-14T09:03:12.376094+0000","last_undegraded":"2024-03-14T09:03:12.376094+0000","last_fullsized":"2024-03-14T09:03:12.376094+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:38.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358447+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:11.358447+0000","last_peered":"2024-03-14T09:03:11.358447+0000","last_clean":"2024-03-14T09:03:11.358447+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:11.358447+0000","last_undegraded":"2024-03-14T09:03:11.358447+0000","last_fullsized":"2024-03-14T09:03:11.358447+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354648+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:11.354648+0000","last_peered":"2024-03-14T09:03:11.354648+0000","last_clean":"2024-03-14T09:03:11.354648+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:11.354648+0000","last_undegraded":"2024-03-14T09:03:11.354648+0000","last_fullsized":"2024-03-14T09:03:11.354648+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356321+0000","last_change":"2024-03-14T09:03:12.356321+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356321+0000","last_undegraded":"2024-03-14T09:03:12.356321+0000","last_fullsized":"2024-03-14T09:03:12.356321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.373386+0000","last_change":"2024-03-14T09:03:12.368944+0000","last_active":"2024-03-14T09:03:12.373386+0000","last_peered":"2024-03-14T09:03:12.373386+0000","last_clean":"2024-03-14T09:03:12.373386+0000","last_became_active":"2024-03-14T09:03:12.368631+0000","last_became_peered":"2024-03-14T09:03:12.368631+0000","last_unstale":"2024-03-14T09:03:12.373386+0000","last_undegraded":"2024-03-14T09:03:12.373386+0000","last_fullsized":"2024-03-14T09:03:12.373386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354667+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:11.354667+0000","last_peered":"2024-03-14T09:03:11.354667+0000","last_clean":"2024-03-14T09:03:11.354667+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:11.354667+0000","last_undegraded":"2024-03-14T09:03:11.354667+0000","last_fullsized":"2024-03-14T09:03:11.354667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355806+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:11.355806+0000","last_peered":"2024-03-14T09:03:11.355806+0000","last_clean":"2024-03-14T09:03:11.355806+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:11.355806+0000","last_undegraded":"2024-03-14T09:03:11.355806+0000","last_fullsized":"2024-03-14T09:03:11.355806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.360063+0000","last_change":"2024-03-14T09:03:12.360063+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.360063+0000","last_undegraded":"2024-03-14T09:03:12.360063+0000","last_fullsized":"2024-03-14T09:03:12.360063+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371484+0000","last_change":"2024-03-14T09:03:12.370316+0000","last_active":"2024-03-14T09:03:12.371484+0000","last_peered":"2024-03-14T09:03:12.371484+0000","last_clean":"2024-03-14T09:03:12.371484+0000","last_became_active":"2024-03-14T09:03:12.370035+0000","last_became_peered":"2024-03-14T09:03:12.370035+0000","last_unstale":"2024-03-14T09:03:12.371484+0000","last_undegraded":"2024-03-14T09:03:12.371484+0000","last_fullsized":"2024-03-14T09:03:12.371484+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:07.312244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358188+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:11.358188+0000","last_peered":"2024-03-14T09:03:11.358188+0000","last_clean":"2024-03-14T09:03:11.358188+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:11.358188+0000","last_undegraded":"2024-03-14T09:03:11.358188+0000","last_fullsized":"2024-03-14T09:03:11.358188+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358017+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:11.358017+0000","last_peered":"2024-03-14T09:03:11.358017+0000","last_clean":"2024-03-14T09:03:11.358017+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:11.358017+0000","last_undegraded":"2024-03-14T09:03:11.358017+0000","last_fullsized":"2024-03-14T09:03:11.358017+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357489+0000","last_change":"2024-03-14T09:03:12.357489+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357489+0000","last_undegraded":"2024-03-14T09:03:12.357489+0000","last_fullsized":"2024-03-14T09:03:12.357489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377230+0000","last_change":"2024-03-14T09:03:12.373196+0000","last_active":"2024-03-14T09:03:12.377230+0000","last_peered":"2024-03-14T09:03:12.377230+0000","last_clean":"2024-03-14T09:03:12.377230+0000","last_became_active":"2024-03-14T09:03:12.372917+0000","last_became_peered":"2024-03-14T09:03:12.372917+0000","last_unstale":"2024-03-14T09:03:12.377230+0000","last_undegraded":"2024-03-14T09:03:12.377230+0000","last_fullsized":"2024-03-14T09:03:12.377230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:56.766593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358565+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:11.358565+0000","last_peered":"2024-03-14T09:03:11.358565+0000","last_clean":"2024-03-14T09:03:11.358565+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:11.358565+0000","last_undegraded":"2024-03-14T09:03:11.358565+0000","last_fullsized":"2024-03-14T09:03:11.358565+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353682+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:11.353682+0000","last_peered":"2024-03-14T09:03:11.353682+0000","last_clean":"2024-03-14T09:03:11.353682+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:11.353682+0000","last_undegraded":"2024-03-14T09:03:11.353682+0000","last_fullsized":"2024-03-14T09:03:11.353682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359244+0000","last_change":"2024-03-14T09:03:12.359244+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359244+0000","last_undegraded":"2024-03-14T09:03:12.359244+0000","last_fullsized":"2024-03-14T09:03:12.359244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.377421+0000","last_change":"2024-03-14T09:03:12.373227+0000","last_active":"2024-03-14T09:03:12.377421+0000","last_peered":"2024-03-14T09:03:12.377421+0000","last_clean":"2024-03-14T09:03:12.377421+0000","last_became_active":"2024-03-14T09:03:12.372761+0000","last_became_peered":"2024-03-14T09:03:12.372761+0000","last_unstale":"2024-03-14T09:03:12.377421+0000","last_undegraded":"2024-03-14T09:03:12.377421+0000","last_fullsized":"2024-03-14T09:03:12.377421+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355667+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:11.355667+0000","last_peered":"2024-03-14T09:03:11.355667+0000","last_clean":"2024-03-14T09:03:11.355667+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:11.355667+0000","last_undegraded":"2024-03-14T09:03:11.355667+0000","last_fullsized":"2024-03-14T09:03:11.355667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349366+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:11.349366+0000","last_peered":"2024-03-14T09:03:11.349366+0000","last_clean":"2024-03-14T09:03:11.349366+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:11.349366+0000","last_undegraded":"2024-03-14T09:03:11.349366+0000","last_fullsized":"2024-03-14T09:03:11.349366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.364238+0000","last_change":"2024-03-14T09:03:12.364238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.364238+0000","last_undegraded":"2024-03-14T09:03:12.364238+0000","last_fullsized":"2024-03-14T09:03:12.364238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367869+0000","last_change":"2024-03-14T09:03:12.365684+0000","last_active":"2024-03-14T09:03:12.367869+0000","last_peered":"2024-03-14T09:03:12.367869+0000","last_clean":"2024-03-14T09:03:12.367869+0000","last_became_active":"2024-03-14T09:03:12.365263+0000","last_became_peered":"2024-03-14T09:03:12.365263+0000","last_unstale":"2024-03-14T09:03:12.367869+0000","last_undegraded":"2024-03-14T09:03:12.367869+0000","last_fullsized":"2024-03-14T09:03:12.367869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351721+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:11.351721+0000","last_peered":"2024-03-14T09:03:11.351721+0000","last_clean":"2024-03-14T09:03:11.351721+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:11.351721+0000","last_undegraded":"2024-03-14T09:03:11.351721+0000","last_fullsized":"2024-03-14T09:03:11.351721+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355737+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:11.355737+0000","last_peered":"2024-03-14T09:03:11.355737+0000","last_clean":"2024-03-14T09:03:11.355737+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:11.355737+0000","last_undegraded":"2024-03-14T09:03:11.355737+0000","last_fullsized":"2024-03-14T09:03:11.355737+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357463+0000","last_change":"2024-03-14T09:03:12.357463+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357463+0000","last_undegraded":"2024-03-14T09:03:12.357463+0000","last_fullsized":"2024-03-14T09:03:12.357463+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376072+0000","last_change":"2024-03-14T09:03:12.371367+0000","last_active":"2024-03-14T09:03:12.376072+0000","last_peered":"2024-03-14T09:03:12.376072+0000","last_clean":"2024-03-14T09:03:12.376072+0000","last_became_active":"2024-03-14T09:03:12.371044+0000","last_became_peered":"2024-03-14T09:03:12.371044+0000","last_unstale":"2024-03-14T09:03:12.376072+0000","last_undegraded":"2024-03-14T09:03:12.376072+0000","last_fullsized":"2024-03-14T09:03:12.376072+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:20:22.371301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358462+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:11.358462+0000","last_peered":"2024-03-14T09:03:11.358462+0000","last_clean":"2024-03-14T09:03:11.358462+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:11.358462+0000","last_undegraded":"2024-03-14T09:03:11.358462+0000","last_fullsized":"2024-03-14T09:03:11.358462+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349280+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:11.349280+0000","last_peered":"2024-03-14T09:03:11.349280+0000","last_clean":"2024-03-14T09:03:11.349280+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:11.349280+0000","last_undegraded":"2024-03-14T09:03:11.349280+0000","last_fullsized":"2024-03-14T09:03:11.349280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352704+0000","last_change":"2024-03-14T09:03:12.352704+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352704+0000","last_undegraded":"2024-03-14T09:03:12.352704+0000","last_fullsized":"2024-03-14T09:03:12.352704+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.374596+0000","last_change":"2024-03-14T09:03:12.372117+0000","last_active":"2024-03-14T09:03:12.374596+0000","last_peered":"2024-03-14T09:03:12.374596+0000","last_clean":"2024-03-14T09:03:12.374596+0000","last_became_active":"2024-03-14T09:03:12.371774+0000","last_became_peered":"2024-03-14T09:03:12.371774+0000","last_unstale":"2024-03-14T09:03:12.374596+0000","last_undegraded":"2024-03-14T09:03:12.374596+0000","last_fullsized":"2024-03-14T09:03:12.374596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:04.409001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355185+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:11.355185+0000","last_peered":"2024-03-14T09:03:11.355185+0000","last_clean":"2024-03-14T09:03:11.355185+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:11.355185+0000","last_undegraded":"2024-03-14T09:03:11.355185+0000","last_fullsized":"2024-03-14T09:03:11.355185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351724+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:11.351724+0000","last_peered":"2024-03-14T09:03:11.351724+0000","last_clean":"2024-03-14T09:03:11.351724+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:11.351724+0000","last_undegraded":"2024-03-14T09:03:11.351724+0000","last_fullsized":"2024-03-14T09:03:11.351724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.354163+0000","last_change":"2024-03-14T09:03:12.354163+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.354163+0000","last_undegraded":"2024-03-14T09:03:12.354163+0000","last_fullsized":"2024-03-14T09:03:12.354163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372069+0000","last_change":"2024-03-14T09:03:12.370993+0000","last_active":"2024-03-14T09:03:12.372069+0000","last_peered":"2024-03-14T09:03:12.372069+0000","last_clean":"2024-03-14T09:03:12.372069+0000","last_became_active":"2024-03-14T09:03:12.370745+0000","last_became_peered":"2024-03-14T09:03:12.370745+0000","last_unstale":"2024-03-14T09:03:12.372069+0000","last_undegraded":"2024-03-14T09:03:12.372069+0000","last_fullsized":"2024-03-14T09:03:12.372069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349338+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:11.349338+0000","last_peered":"2024-03-14T09:03:11.349338+0000","last_clean":"2024-03-14T09:03:11.349338+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:11.349338+0000","last_undegraded":"2024-03-14T09:03:11.349338+0000","last_fullsized":"2024-03-14T09:03:11.349338+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356757+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:11.356757+0000","last_peered":"2024-03-14T09:03:11.356757+0000","last_clean":"2024-03-14T09:03:11.356757+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:11.356757+0000","last_undegraded":"2024-03-14T09:03:11.356757+0000","last_fullsized":"2024-03-14T09:03:11.356757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353676+0000","last_change":"2024-03-14T09:03:12.353676+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353676+0000","last_undegraded":"2024-03-14T09:03:12.353676+0000","last_fullsized":"2024-03-14T09:03:12.353676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.368922+0000","last_change":"2024-03-14T09:03:12.366770+0000","last_active":"2024-03-14T09:03:12.368922+0000","last_peered":"2024-03-14T09:03:12.368922+0000","last_clean":"2024-03-14T09:03:12.368922+0000","last_became_active":"2024-03-14T09:03:12.366022+0000","last_became_peered":"2024-03-14T09:03:12.366022+0000","last_unstale":"2024-03-14T09:03:12.368922+0000","last_undegraded":"2024-03-14T09:03:12.368922+0000","last_fullsized":"2024-03-14T09:03:12.368922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349099+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:11.349099+0000","last_peered":"2024-03-14T09:03:11.349099+0000","last_clean":"2024-03-14T09:03:11.349099+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:11.349099+0000","last_undegraded":"2024-03-14T09:03:11.349099+0000","last_fullsized":"2024-03-14T09:03:11.349099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.354967+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:11.354967+0000","last_peered":"2024-03-14T09:03:11.354967+0000","last_clean":"2024-03-14T09:03:11.354967+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:11.354967+0000","last_undegraded":"2024-03-14T09:03:11.354967+0000","last_fullsized":"2024-03-14T09:03:11.354967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356783+0000","last_change":"2024-03-14T09:03:12.356783+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356783+0000","last_undegraded":"2024-03-14T09:03:12.356783+0000","last_fullsized":"2024-03-14T09:03:12.356783+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375064+0000","last_change":"2024-03-14T09:03:12.373601+0000","last_active":"2024-03-14T09:03:12.375064+0000","last_peered":"2024-03-14T09:03:12.375064+0000","last_clean":"2024-03-14T09:03:12.375064+0000","last_became_active":"2024-03-14T09:03:12.373332+0000","last_became_peered":"2024-03-14T09:03:12.373332+0000","last_unstale":"2024-03-14T09:03:12.375064+0000","last_undegraded":"2024-03-14T09:03:12.375064+0000","last_fullsized":"2024-03-14T09:03:12.375064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:38:05.459173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355546+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:11.355546+0000","last_peered":"2024-03-14T09:03:11.355546+0000","last_clean":"2024-03-14T09:03:11.355546+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:11.355546+0000","last_undegraded":"2024-03-14T09:03:11.355546+0000","last_fullsized":"2024-03-14T09:03:11.355546+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358290+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:11.358290+0000","last_peered":"2024-03-14T09:03:11.358290+0000","last_clean":"2024-03-14T09:03:11.358290+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:11.358290+0000","last_undegraded":"2024-03-14T09:03:11.358290+0000","last_fullsized":"2024-03-14T09:03:11.358290+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.357797+0000","last_change":"2024-03-14T09:03:12.357797+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.357797+0000","last_undegraded":"2024-03-14T09:03:12.357797+0000","last_fullsized":"2024-03-14T09:03:12.357797+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375031+0000","last_change":"2024-03-14T09:03:12.373610+0000","last_active":"2024-03-14T09:03:12.375031+0000","last_peered":"2024-03-14T09:03:12.375031+0000","last_clean":"2024-03-14T09:03:12.375031+0000","last_became_active":"2024-03-14T09:03:12.373360+0000","last_became_peered":"2024-03-14T09:03:12.373360+0000","last_unstale":"2024-03-14T09:03:12.375031+0000","last_undegraded":"2024-03-14T09:03:12.375031+0000","last_fullsized":"2024-03-14T09:03:12.375031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:32.334273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355435+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:11.355435+0000","last_peered":"2024-03-14T09:03:11.355435+0000","last_clean":"2024-03-14T09:03:11.355435+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:11.355435+0000","last_undegraded":"2024-03-14T09:03:11.355435+0000","last_fullsized":"2024-03-14T09:03:11.355435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"51'1","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349459+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:11.349459+0000","last_peered":"2024-03-14T09:03:11.349459+0000","last_clean":"2024-03-14T09:03:11.349459+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:11.349459+0000","last_undegraded":"2024-03-14T09:03:11.349459+0000","last_fullsized":"2024-03-14T09:03:11.349459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.356751+0000","last_change":"2024-03-14T09:03:12.356751+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.356751+0000","last_undegraded":"2024-03-14T09:03:12.356751+0000","last_fullsized":"2024-03-14T09:03:12.356751+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367852+0000","last_change":"2024-03-14T09:03:12.365515+0000","last_active":"2024-03-14T09:03:12.367852+0000","last_peered":"2024-03-14T09:03:12.367852+0000","last_clean":"2024-03-14T09:03:12.367852+0000","last_became_active":"2024-03-14T09:03:12.364921+0000","last_became_peered":"2024-03-14T09:03:12.364921+0000","last_unstale":"2024-03-14T09:03:12.367852+0000","last_undegraded":"2024-03-14T09:03:12.367852+0000","last_fullsized":"2024-03-14T09:03:12.367852+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.357128+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:11.357128+0000","last_peered":"2024-03-14T09:03:11.357128+0000","last_clean":"2024-03-14T09:03:11.357128+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:11.357128+0000","last_undegraded":"2024-03-14T09:03:11.357128+0000","last_fullsized":"2024-03-14T09:03:11.357128+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355195+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:11.355195+0000","last_peered":"2024-03-14T09:03:11.355195+0000","last_clean":"2024-03-14T09:03:11.355195+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:11.355195+0000","last_undegraded":"2024-03-14T09:03:11.355195+0000","last_fullsized":"2024-03-14T09:03:11.355195+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.361654+0000","last_change":"2024-03-14T09:03:12.361654+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.361654+0000","last_undegraded":"2024-03-14T09:03:12.361654+0000","last_fullsized":"2024-03-14T09:03:12.361654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.371874+0000","last_change":"2024-03-14T09:03:12.367975+0000","last_active":"2024-03-14T09:03:12.371874+0000","last_peered":"2024-03-14T09:03:12.371874+0000","last_clean":"2024-03-14T09:03:12.371874+0000","last_became_active":"2024-03-14T09:03:12.367702+0000","last_became_peered":"2024-03-14T09:03:12.367702+0000","last_unstale":"2024-03-14T09:03:12.371874+0000","last_undegraded":"2024-03-14T09:03:12.371874+0000","last_fullsized":"2024-03-14T09:03:12.371874+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:12.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.349479+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:11.349479+0000","last_peered":"2024-03-14T09:03:11.349479+0000","last_clean":"2024-03-14T09:03:11.349479+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:11.349479+0000","last_undegraded":"2024-03-14T09:03:11.349479+0000","last_fullsized":"2024-03-14T09:03:11.349479+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355796+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:11.355796+0000","last_peered":"2024-03-14T09:03:11.355796+0000","last_clean":"2024-03-14T09:03:11.355796+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:11.355796+0000","last_undegraded":"2024-03-14T09:03:11.355796+0000","last_fullsized":"2024-03-14T09:03:11.355796+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.358131+0000","last_change":"2024-03-14T09:03:12.358131+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.358131+0000","last_undegraded":"2024-03-14T09:03:12.358131+0000","last_fullsized":"2024-03-14T09:03:12.358131+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.376099+0000","last_change":"2024-03-14T09:03:12.371984+0000","last_active":"2024-03-14T09:03:12.376099+0000","last_peered":"2024-03-14T09:03:12.376099+0000","last_clean":"2024-03-14T09:03:12.376099+0000","last_became_active":"2024-03-14T09:03:12.371727+0000","last_became_peered":"2024-03-14T09:03:12.371727+0000","last_unstale":"2024-03-14T09:03:12.376099+0000","last_undegraded":"2024-03-14T09:03:12.376099+0000","last_fullsized":"2024-03-14T09:03:12.376099+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:23.845822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351395+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:11.351395+0000","last_peered":"2024-03-14T09:03:11.351395+0000","last_clean":"2024-03-14T09:03:11.351395+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:11.351395+0000","last_undegraded":"2024-03-14T09:03:11.351395+0000","last_fullsized":"2024-03-14T09:03:11.351395+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356875+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:11.356875+0000","last_peered":"2024-03-14T09:03:11.356875+0000","last_clean":"2024-03-14T09:03:11.356875+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:11.356875+0000","last_undegraded":"2024-03-14T09:03:11.356875+0000","last_fullsized":"2024-03-14T09:03:11.356875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.352756+0000","last_change":"2024-03-14T09:03:12.352756+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.352756+0000","last_undegraded":"2024-03-14T09:03:12.352756+0000","last_fullsized":"2024-03-14T09:03:12.352756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.366806+0000","last_change":"2024-03-14T09:03:12.363729+0000","last_active":"2024-03-14T09:03:12.366806+0000","last_peered":"2024-03-14T09:03:12.366806+0000","last_clean":"2024-03-14T09:03:12.366806+0000","last_became_active":"2024-03-14T09:03:12.363440+0000","last_became_peered":"2024-03-14T09:03:12.363440+0000","last_unstale":"2024-03-14T09:03:12.366806+0000","last_undegraded":"2024-03-14T09:03:12.366806+0000","last_fullsized":"2024-03-14T09:03:12.366806+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:13.058703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358367+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:11.358367+0000","last_peered":"2024-03-14T09:03:11.358367+0000","last_clean":"2024-03-14T09:03:11.358367+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:11.358367+0000","last_undegraded":"2024-03-14T09:03:11.358367+0000","last_fullsized":"2024-03-14T09:03:11.358367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.355086+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:11.355086+0000","last_peered":"2024-03-14T09:03:11.355086+0000","last_clean":"2024-03-14T09:03:11.355086+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:11.355086+0000","last_undegraded":"2024-03-14T09:03:11.355086+0000","last_fullsized":"2024-03-14T09:03:11.355086+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.359840+0000","last_change":"2024-03-14T09:03:12.359840+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.359840+0000","last_undegraded":"2024-03-14T09:03:12.359840+0000","last_fullsized":"2024-03-14T09:03:12.359840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.372010+0000","last_change":"2024-03-14T09:03:12.370434+0000","last_active":"2024-03-14T09:03:12.372010+0000","last_peered":"2024-03-14T09:03:12.372010+0000","last_clean":"2024-03-14T09:03:12.372010+0000","last_became_active":"2024-03-14T09:03:12.370170+0000","last_became_peered":"2024-03-14T09:03:12.370170+0000","last_unstale":"2024-03-14T09:03:12.372010+0000","last_undegraded":"2024-03-14T09:03:12.372010+0000","last_fullsized":"2024-03-14T09:03:12.372010+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:55.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356464+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:11.356464+0000","last_peered":"2024-03-14T09:03:11.356464+0000","last_clean":"2024-03-14T09:03:11.356464+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:11.356464+0000","last_undegraded":"2024-03-14T09:03:11.356464+0000","last_fullsized":"2024-03-14T09:03:11.356464+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.356577+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:11.356577+0000","last_peered":"2024-03-14T09:03:11.356577+0000","last_clean":"2024-03-14T09:03:11.356577+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:11.356577+0000","last_undegraded":"2024-03-14T09:03:11.356577+0000","last_fullsized":"2024-03-14T09:03:11.356577+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353449+0000","last_change":"2024-03-14T09:03:12.353449+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353449+0000","last_undegraded":"2024-03-14T09:03:12.353449+0000","last_fullsized":"2024-03-14T09:03:12.353449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.375525+0000","last_change":"2024-03-14T09:03:12.373132+0000","last_active":"2024-03-14T09:03:12.375525+0000","last_peered":"2024-03-14T09:03:12.375525+0000","last_clean":"2024-03-14T09:03:12.375525+0000","last_became_active":"2024-03-14T09:03:12.372802+0000","last_became_peered":"2024-03-14T09:03:12.372802+0000","last_unstale":"2024-03-14T09:03:12.375525+0000","last_undegraded":"2024-03-14T09:03:12.375525+0000","last_fullsized":"2024-03-14T09:03:12.375525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351200+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:11.351200+0000","last_peered":"2024-03-14T09:03:11.351200+0000","last_clean":"2024-03-14T09:03:11.351200+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:11.351200+0000","last_undegraded":"2024-03-14T09:03:11.351200+0000","last_fullsized":"2024-03-14T09:03:11.351200+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351201+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:11.351201+0000","last_peered":"2024-03-14T09:03:11.351201+0000","last_clean":"2024-03-14T09:03:11.351201+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:11.351201+0000","last_undegraded":"2024-03-14T09:03:11.351201+0000","last_fullsized":"2024-03-14T09:03:11.351201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-03-14T09:03:12.353238+0000","last_change":"2024-03-14T09:03:12.353238+0000","last_active":"2024-03-14T09:03:12.347114+0000","last_peered":"2024-03-14T09:03:12.347114+0000","last_clean":"2024-03-14T09:03:12.347114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-14T09:03:12.353238+0000","last_undegraded":"2024-03-14T09:03:12.353238+0000","last_fullsized":"2024-03-14T09:03:12.353238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.367162+0000","last_change":"2024-03-14T09:03:12.364039+0000","last_active":"2024-03-14T09:03:12.367162+0000","last_peered":"2024-03-14T09:03:12.367162+0000","last_clean":"2024-03-14T09:03:12.367162+0000","last_became_active":"2024-03-14T09:03:12.363778+0000","last_became_peered":"2024-03-14T09:03:12.363778+0000","last_unstale":"2024-03-14T09:03:12.367162+0000","last_undegraded":"2024-03-14T09:03:12.367162+0000","last_fullsized":"2024-03-14T09:03:12.367162+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:53.168005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.351195+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:11.351195+0000","last_peered":"2024-03-14T09:03:11.351195+0000","last_clean":"2024-03-14T09:03:11.351195+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:11.351195+0000","last_undegraded":"2024-03-14T09:03:11.351195+0000","last_fullsized":"2024-03-14T09:03:11.351195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.353295+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:11.353295+0000","last_peered":"2024-03-14T09:03:11.353295+0000","last_clean":"2024-03-14T09:03:11.353295+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:11.353295+0000","last_undegraded":"2024-03-14T09:03:11.353295+0000","last_fullsized":"2024-03-14T09:03:11.353295+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-03-14T09:03:11.358124+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:11.358124+0000","last_peered":"2024-03-14T09:03:11.358124+0000","last_clean":"2024-03-14T09:03:11.358124+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:11.358124+0000","last_undegraded":"2024-03-14T09:03:11.358124+0000","last_fullsized":"2024-03-14T09:03:11.358124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1236992,"data_stored":861723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.5,"5min":6.5,"15min":6.5},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.5},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.623}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.442}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":271195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":264155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.37}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.318}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.621}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":858090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.383,"5min":7.383,"15min":7.383},"last":0.372},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.316}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.9100000000000001,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.944,"5min":7.944,"15min":7.944},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":7.3410000000000002,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":7.5140000000000002,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.44}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":7.4669999999999996,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40000000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.504},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":643072,"data_stored":271251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.254},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":7.5,"5min":7.5,"15min":7.5},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.4349999999999996,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.33600000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.24099999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.435}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":274270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.498},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.433}]},{"osd":2,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.625},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.702,"5min":7.702,"15min":7.702},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:02:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":264717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.443}]},{"osd":1,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.5}]},{"osd":3,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:02:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:21.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:03:21.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.633+0000 7f1f04b96700 1 Processor -- start 2024-03-14T09:03:21.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.633+0000 7f1f04b96700 1 -- start start 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1f04b96700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f000a8540 0x7f1f0013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1f04b96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f00154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1f04b96700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0013ade0 con 0x7f1f001550b0 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1f04b96700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0006fb30 con 0x7f1f000a8540 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1f04b96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0006ec30 con 0x7f1f000c3250 2024-03-14T09:03:21.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1efe59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f000a8540 0x7f1f0013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1efe59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f000a8540 0x7f1f0013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35308/0 (socket says 172.21.15.154:35308) 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.634+0000 7f1efe59c700 1 -- 172.21.15.154:0/3668124204 learned_addr learned my addr 172.21.15.154:0/3668124204 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efdd9b700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f00154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efe59c700 1 -- 172.21.15.154:0/3668124204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 msgr2=0x7f1f00154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efe59c700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f00154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efe59c700 1 -- 172.21.15.154:0/3668124204 >> v1:172.21.15.191:6789/0 conn(0x7f1f001550b0 legacy=0x7f1f000c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efe59c700 1 -- 172.21.15.154:0/3668124204 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0006ef10 con 0x7f1f000a8540 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efe59c700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f000a8540 0x7f1f0013a8a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40098f0 tx=0x7f1ef4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5677335f7dc01765 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efd59a700 1 -- 172.21.15.154:0/3668124204 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011830 con 0x7f1f000a8540 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efd59a700 1 -- 172.21.15.154:0/3668124204 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4011990 con 0x7f1f000a8540 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.635+0000 7f1efd59a700 1 -- 172.21.15.154:0/3668124204 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011d20 con 0x7f1f000a8540 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 -- 172.21.15.154:0/3668124204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f000a8540 msgr2=0x7f1f0013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f000a8540 0x7f1f0013a8a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40098f0 tx=0x7f1ef4009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 -- 172.21.15.154:0/3668124204 shutdown_connections 2024-03-14T09:03:21.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f000a8540 0x7f1f0013a8a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3668124204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f00154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 -- 172.21.15.154:0/3668124204 >> 172.21.15.154:0/3668124204 conn(0x7f1f000c3880 msgr2=0x7f1f000c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:21.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 -- 172.21.15.154:0/3668124204 shutdown_connections 2024-03-14T09:03:21.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.636+0000 7f1f04b96700 1 -- 172.21.15.154:0/3668124204 wait complete. 2024-03-14T09:03:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.637+0000 7f1f04b96700 1 Processor -- start 2024-03-14T09:03:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.637+0000 7f1f04b96700 1 -- start start 2024-03-14T09:03:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f000a8540 0x7f1f000ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f001deff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000ac960 con 0x7f1f001550b0 2024-03-14T09:03:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000ac300 con 0x7f1f000a8540 2024-03-14T09:03:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001eac80 con 0x7f1f000c3250 2024-03-14T09:03:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35316/0 (socket says 172.21.15.154:35316) 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 -- 172.21.15.154:0/3609361577 learned_addr learned my addr 172.21.15.154:0/3609361577 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efdd9b700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f001deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efe59c700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f000a8540 0x7f1f000ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 -- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 msgr2=0x7f1f001deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f001deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 -- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f000a8540 msgr2=0x7f1f000ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f000a8540 0x7f1f000ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 -- 172.21.15.154:0/3609361577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4009550 con 0x7f1f001550b0 2024-03-14T09:03:21.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1efed9d700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800cb50 tx=0x7f1ef800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:21.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8003c00 con 0x7f1f001550b0 2024-03-14T09:03:21.646 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8003d60 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800aa20 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.638+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f001ea3b0 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.639+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001ea910 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.639+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00155600 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.640+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1ef800ab80 con 0x7f1f001550b0 2024-03-14T09:03:21.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.640+0000 7f1eeeffd700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 0x7f1ee006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.640+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1ef8088e60 con 0x7f1f001550b0 2024-03-14T09:03:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.641+0000 7f1efe59c700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 0x7f1ee006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.641+0000 7f1efe59c700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 0x7f1ee006ccf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009c40 tx=0x7f1ef4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.645+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1f00155600 con 0x7f1f001550b0 2024-03-14T09:03:21.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.833+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f00155900 con 0x7f1ee006a830 2024-03-14T09:03:21.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.848+0000 7f1eeeffd700 1 -- 172.21.15.154:0/3609361577 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528113 (secure 0 0 0) 0x7f1ef4011c00 con 0x7f1ee006a830 2024-03-14T09:03:21.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 msgr2=0x7f1ee006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 0x7f1ee006ccf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009c40 tx=0x7f1ef4008030 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 msgr2=0x7f1f00152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800cb50 tx=0x7f1ef800ce60 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 shutdown_connections 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1ee006a830 0x7f1ee006ccf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f001550b0 0x7f1f00152a70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f000a8540 0x7f1f000ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 --2- 172.21.15.154:0/3609361577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f000c3250 0x7f1f001deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 >> 172.21.15.154:0/3609361577 conn(0x7f1f000c3880 msgr2=0x7f1f0006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 shutdown_connections 2024-03-14T09:03:21.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:21.852+0000 7f1f04b96700 1 -- 172.21.15.154:0/3609361577 wait complete. 2024-03-14T09:03:21.860 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:03:21.864 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-14T09:03:21.546533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222600,"kb_used_data":6440,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752120,"statfs":{"total":773094113280,"available":772866170880,"internally_reserved":0,"allocated":6594560,"data_stored":3481086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.776661"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782354+0000","last_change":"2024-03-14T09:03:12.778763+0000","last_active":"2024-03-14T09:03:12.782354+0000","last_peered":"2024-03-14T09:03:12.782354+0000","last_clean":"2024-03-14T09:03:12.782354+0000","last_became_active":"2024-03-14T09:03:12.778514+0000","last_became_peered":"2024-03-14T09:03:12.778514+0000","last_unstale":"2024-03-14T09:03:12.782354+0000","last_undegraded":"2024-03-14T09:03:12.782354+0000","last_fullsized":"2024-03-14T09:03:12.782354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:34:04.791652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775201+0000","last_change":"2024-03-14T09:02:59.997701+0000","last_active":"2024-03-14T09:03:12.775201+0000","last_peered":"2024-03-14T09:03:12.775201+0000","last_clean":"2024-03-14T09:03:12.775201+0000","last_became_active":"2024-03-14T09:02:59.997441+0000","last_became_peered":"2024-03-14T09:02:59.997441+0000","last_unstale":"2024-03-14T09:03:12.775201+0000","last_undegraded":"2024-03-14T09:03:12.775201+0000","last_fullsized":"2024-03-14T09:03:12.775201+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:01.649027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.801444+0000","last_change":"2024-03-14T09:03:12.794266+0000","last_active":"2024-03-14T09:03:12.801444+0000","last_peered":"2024-03-14T09:03:12.801444+0000","last_clean":"2024-03-14T09:03:12.801444+0000","last_became_active":"2024-03-14T09:03:12.793979+0000","last_became_peered":"2024-03-14T09:03:12.793979+0000","last_unstale":"2024-03-14T09:03:12.801444+0000","last_undegraded":"2024-03-14T09:03:12.801444+0000","last_fullsized":"2024-03-14T09:03:12.801444+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:24.847556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773357+0000","last_change":"2024-03-14T09:02:59.988079+0000","last_active":"2024-03-14T09:03:12.773357+0000","last_peered":"2024-03-14T09:03:12.773357+0000","last_clean":"2024-03-14T09:03:12.773357+0000","last_became_active":"2024-03-14T09:02:59.987824+0000","last_became_peered":"2024-03-14T09:02:59.987824+0000","last_unstale":"2024-03-14T09:03:12.773357+0000","last_undegraded":"2024-03-14T09:03:12.773357+0000","last_fullsized":"2024-03-14T09:03:12.773357+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:20.545376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805352+0000","last_change":"2024-03-14T09:03:12.798284+0000","last_active":"2024-03-14T09:03:12.805352+0000","last_peered":"2024-03-14T09:03:12.805352+0000","last_clean":"2024-03-14T09:03:12.805352+0000","last_became_active":"2024-03-14T09:03:12.795626+0000","last_became_peered":"2024-03-14T09:03:12.795626+0000","last_unstale":"2024-03-14T09:03:12.805352+0000","last_undegraded":"2024-03-14T09:03:12.805352+0000","last_fullsized":"2024-03-14T09:03:12.805352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:40.525920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785189+0000","last_change":"2024-03-14T09:03:00.003712+0000","last_active":"2024-03-14T09:03:12.785189+0000","last_peered":"2024-03-14T09:03:12.785189+0000","last_clean":"2024-03-14T09:03:12.785189+0000","last_became_active":"2024-03-14T09:03:00.003166+0000","last_became_peered":"2024-03-14T09:03:00.003166+0000","last_unstale":"2024-03-14T09:03:12.785189+0000","last_undegraded":"2024-03-14T09:03:12.785189+0000","last_fullsized":"2024-03-14T09:03:12.785189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798877+0000","last_change":"2024-03-14T09:03:12.792222+0000","last_active":"2024-03-14T09:03:12.798877+0000","last_peered":"2024-03-14T09:03:12.798877+0000","last_clean":"2024-03-14T09:03:12.798877+0000","last_became_active":"2024-03-14T09:03:12.791890+0000","last_became_peered":"2024-03-14T09:03:12.791890+0000","last_unstale":"2024-03-14T09:03:12.798877+0000","last_undegraded":"2024-03-14T09:03:12.798877+0000","last_fullsized":"2024-03-14T09:03:12.798877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773484+0000","last_change":"2024-03-14T09:02:59.993841+0000","last_active":"2024-03-14T09:03:12.773484+0000","last_peered":"2024-03-14T09:03:12.773484+0000","last_clean":"2024-03-14T09:03:12.773484+0000","last_became_active":"2024-03-14T09:02:59.993563+0000","last_became_peered":"2024-03-14T09:02:59.993563+0000","last_unstale":"2024-03-14T09:03:12.773484+0000","last_undegraded":"2024-03-14T09:03:12.773484+0000","last_fullsized":"2024-03-14T09:03:12.773484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786990+0000","last_change":"2024-03-14T09:03:12.780749+0000","last_active":"2024-03-14T09:03:12.786990+0000","last_peered":"2024-03-14T09:03:12.786990+0000","last_clean":"2024-03-14T09:03:12.786990+0000","last_became_active":"2024-03-14T09:03:12.780423+0000","last_became_peered":"2024-03-14T09:03:12.780423+0000","last_unstale":"2024-03-14T09:03:12.786990+0000","last_undegraded":"2024-03-14T09:03:12.786990+0000","last_fullsized":"2024-03-14T09:03:12.786990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:30.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.770302+0000","last_change":"2024-03-14T09:02:59.992988+0000","last_active":"2024-03-14T09:03:12.770302+0000","last_peered":"2024-03-14T09:03:12.770302+0000","last_clean":"2024-03-14T09:03:12.770302+0000","last_became_active":"2024-03-14T09:02:59.992424+0000","last_became_peered":"2024-03-14T09:02:59.992424+0000","last_unstale":"2024-03-14T09:03:12.770302+0000","last_undegraded":"2024-03-14T09:03:12.770302+0000","last_fullsized":"2024-03-14T09:03:12.770302+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:01:01.283573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.799732+0000","last_change":"2024-03-14T09:03:12.794610+0000","last_active":"2024-03-14T09:03:12.799732+0000","last_peered":"2024-03-14T09:03:12.799732+0000","last_clean":"2024-03-14T09:03:12.799732+0000","last_became_active":"2024-03-14T09:03:12.793181+0000","last_became_peered":"2024-03-14T09:03:12.793181+0000","last_unstale":"2024-03-14T09:03:12.799732+0000","last_undegraded":"2024-03-14T09:03:12.799732+0000","last_fullsized":"2024-03-14T09:03:12.799732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764963+0000","last_change":"2024-03-14T09:02:59.989565+0000","last_active":"2024-03-14T09:03:12.764963+0000","last_peered":"2024-03-14T09:03:12.764963+0000","last_clean":"2024-03-14T09:03:12.764963+0000","last_became_active":"2024-03-14T09:02:59.989302+0000","last_became_peered":"2024-03-14T09:02:59.989302+0000","last_unstale":"2024-03-14T09:03:12.764963+0000","last_undegraded":"2024-03-14T09:03:12.764963+0000","last_fullsized":"2024-03-14T09:03:12.764963+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795634+0000","last_change":"2024-03-14T09:03:12.789387+0000","last_active":"2024-03-14T09:03:12.795634+0000","last_peered":"2024-03-14T09:03:12.795634+0000","last_clean":"2024-03-14T09:03:12.795634+0000","last_became_active":"2024-03-14T09:03:12.789123+0000","last_became_peered":"2024-03-14T09:03:12.789123+0000","last_unstale":"2024-03-14T09:03:12.795634+0000","last_undegraded":"2024-03-14T09:03:12.795634+0000","last_fullsized":"2024-03-14T09:03:12.795634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.813931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.797301+0000","last_change":"2024-03-14T09:02:59.988879+0000","last_active":"2024-03-14T09:03:12.797301+0000","last_peered":"2024-03-14T09:03:12.797301+0000","last_clean":"2024-03-14T09:03:12.797301+0000","last_became_active":"2024-03-14T09:02:59.988591+0000","last_became_peered":"2024-03-14T09:02:59.988591+0000","last_unstale":"2024-03-14T09:03:12.797301+0000","last_undegraded":"2024-03-14T09:03:12.797301+0000","last_fullsized":"2024-03-14T09:03:12.797301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:53.180020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.799555+0000","last_change":"2024-03-14T09:03:12.792470+0000","last_active":"2024-03-14T09:03:12.799555+0000","last_peered":"2024-03-14T09:03:12.799555+0000","last_clean":"2024-03-14T09:03:12.799555+0000","last_became_active":"2024-03-14T09:03:12.792196+0000","last_became_peered":"2024-03-14T09:03:12.792196+0000","last_unstale":"2024-03-14T09:03:12.799555+0000","last_undegraded":"2024-03-14T09:03:12.799555+0000","last_fullsized":"2024-03-14T09:03:12.799555+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772530+0000","last_change":"2024-03-14T09:02:59.990561+0000","last_active":"2024-03-14T09:03:12.772530+0000","last_peered":"2024-03-14T09:03:12.772530+0000","last_clean":"2024-03-14T09:03:12.772530+0000","last_became_active":"2024-03-14T09:02:59.990302+0000","last_became_peered":"2024-03-14T09:02:59.990302+0000","last_unstale":"2024-03-14T09:03:12.772530+0000","last_undegraded":"2024-03-14T09:03:12.772530+0000","last_fullsized":"2024-03-14T09:03:12.772530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:45:52.620104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802322+0000","last_change":"2024-03-14T09:03:12.793797+0000","last_active":"2024-03-14T09:03:12.802322+0000","last_peered":"2024-03-14T09:03:12.802322+0000","last_clean":"2024-03-14T09:03:12.802322+0000","last_became_active":"2024-03-14T09:03:12.793494+0000","last_became_peered":"2024-03-14T09:03:12.793494+0000","last_unstale":"2024-03-14T09:03:12.802322+0000","last_undegraded":"2024-03-14T09:03:12.802322+0000","last_fullsized":"2024-03-14T09:03:12.802322+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:13.967274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782327+0000","last_change":"2024-03-14T09:02:59.993150+0000","last_active":"2024-03-14T09:03:12.782327+0000","last_peered":"2024-03-14T09:03:12.782327+0000","last_clean":"2024-03-14T09:03:12.782327+0000","last_became_active":"2024-03-14T09:02:59.992909+0000","last_became_peered":"2024-03-14T09:02:59.992909+0000","last_unstale":"2024-03-14T09:03:12.782327+0000","last_undegraded":"2024-03-14T09:03:12.782327+0000","last_fullsized":"2024-03-14T09:03:12.782327+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:47.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792650+0000","last_change":"2024-03-14T09:03:12.784684+0000","last_active":"2024-03-14T09:03:12.792650+0000","last_peered":"2024-03-14T09:03:12.792650+0000","last_clean":"2024-03-14T09:03:12.792650+0000","last_became_active":"2024-03-14T09:03:12.784061+0000","last_became_peered":"2024-03-14T09:03:12.784061+0000","last_unstale":"2024-03-14T09:03:12.792650+0000","last_undegraded":"2024-03-14T09:03:12.792650+0000","last_fullsized":"2024-03-14T09:03:12.792650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:45.740244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768522+0000","last_change":"2024-03-14T09:02:59.988789+0000","last_active":"2024-03-14T09:03:12.768522+0000","last_peered":"2024-03-14T09:03:12.768522+0000","last_clean":"2024-03-14T09:03:12.768522+0000","last_became_active":"2024-03-14T09:02:59.988532+0000","last_became_peered":"2024-03-14T09:02:59.988532+0000","last_unstale":"2024-03-14T09:03:12.768522+0000","last_undegraded":"2024-03-14T09:03:12.768522+0000","last_fullsized":"2024-03-14T09:03:12.768522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:04.128354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786876+0000","last_change":"2024-03-14T09:03:12.781876+0000","last_active":"2024-03-14T09:03:12.786876+0000","last_peered":"2024-03-14T09:03:12.786876+0000","last_clean":"2024-03-14T09:03:12.786876+0000","last_became_active":"2024-03-14T09:03:12.781623+0000","last_became_peered":"2024-03-14T09:03:12.781623+0000","last_unstale":"2024-03-14T09:03:12.786876+0000","last_undegraded":"2024-03-14T09:03:12.786876+0000","last_fullsized":"2024-03-14T09:03:12.786876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:22.264224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768484+0000","last_change":"2024-03-14T09:02:59.988645+0000","last_active":"2024-03-14T09:03:12.768484+0000","last_peered":"2024-03-14T09:03:12.768484+0000","last_clean":"2024-03-14T09:03:12.768484+0000","last_became_active":"2024-03-14T09:02:59.988398+0000","last_became_peered":"2024-03-14T09:02:59.988398+0000","last_unstale":"2024-03-14T09:03:12.768484+0000","last_undegraded":"2024-03-14T09:03:12.768484+0000","last_fullsized":"2024-03-14T09:03:12.768484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:43.264373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802896+0000","last_change":"2024-03-14T09:03:12.797022+0000","last_active":"2024-03-14T09:03:12.802896+0000","last_peered":"2024-03-14T09:03:12.802896+0000","last_clean":"2024-03-14T09:03:12.802896+0000","last_became_active":"2024-03-14T09:03:12.796750+0000","last_became_peered":"2024-03-14T09:03:12.796750+0000","last_unstale":"2024-03-14T09:03:12.802896+0000","last_undegraded":"2024-03-14T09:03:12.802896+0000","last_fullsized":"2024-03-14T09:03:12.802896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:59.134538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773593+0000","last_change":"2024-03-14T09:02:59.994531+0000","last_active":"2024-03-14T09:03:12.773593+0000","last_peered":"2024-03-14T09:03:12.773593+0000","last_clean":"2024-03-14T09:03:12.773593+0000","last_became_active":"2024-03-14T09:02:59.994257+0000","last_became_peered":"2024-03-14T09:02:59.994257+0000","last_unstale":"2024-03-14T09:03:12.773593+0000","last_undegraded":"2024-03-14T09:03:12.773593+0000","last_fullsized":"2024-03-14T09:03:12.773593+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786837+0000","last_change":"2024-03-14T09:03:12.782735+0000","last_active":"2024-03-14T09:03:12.786837+0000","last_peered":"2024-03-14T09:03:12.786837+0000","last_clean":"2024-03-14T09:03:12.786837+0000","last_became_active":"2024-03-14T09:03:12.782477+0000","last_became_peered":"2024-03-14T09:03:12.782477+0000","last_unstale":"2024-03-14T09:03:12.786837+0000","last_undegraded":"2024-03-14T09:03:12.786837+0000","last_fullsized":"2024-03-14T09:03:12.786837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778748+0000","last_change":"2024-03-14T09:02:59.993144+0000","last_active":"2024-03-14T09:03:12.778748+0000","last_peered":"2024-03-14T09:03:12.778748+0000","last_clean":"2024-03-14T09:03:12.778748+0000","last_became_active":"2024-03-14T09:02:59.992871+0000","last_became_peered":"2024-03-14T09:02:59.992871+0000","last_unstale":"2024-03-14T09:03:12.778748+0000","last_undegraded":"2024-03-14T09:03:12.778748+0000","last_fullsized":"2024-03-14T09:03:12.778748+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:01:34.849358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.807468+0000","last_change":"2024-03-14T09:03:12.800456+0000","last_active":"2024-03-14T09:03:12.807468+0000","last_peered":"2024-03-14T09:03:12.807468+0000","last_clean":"2024-03-14T09:03:12.807468+0000","last_became_active":"2024-03-14T09:03:12.800149+0000","last_became_peered":"2024-03-14T09:03:12.800149+0000","last_unstale":"2024-03-14T09:03:12.807468+0000","last_undegraded":"2024-03-14T09:03:12.807468+0000","last_fullsized":"2024-03-14T09:03:12.807468+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:14:12.016422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782261+0000","last_change":"2024-03-14T09:02:59.994768+0000","last_active":"2024-03-14T09:03:12.782261+0000","last_peered":"2024-03-14T09:03:12.782261+0000","last_clean":"2024-03-14T09:03:12.782261+0000","last_became_active":"2024-03-14T09:02:59.994507+0000","last_became_peered":"2024-03-14T09:02:59.994507+0000","last_unstale":"2024-03-14T09:03:12.782261+0000","last_undegraded":"2024-03-14T09:03:12.782261+0000","last_fullsized":"2024-03-14T09:03:12.782261+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:54.366204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795450+0000","last_change":"2024-03-14T09:03:12.792056+0000","last_active":"2024-03-14T09:03:12.795450+0000","last_peered":"2024-03-14T09:03:12.795450+0000","last_clean":"2024-03-14T09:03:12.795450+0000","last_became_active":"2024-03-14T09:03:12.791876+0000","last_became_peered":"2024-03-14T09:03:12.791876+0000","last_unstale":"2024-03-14T09:03:12.795450+0000","last_undegraded":"2024-03-14T09:03:12.795450+0000","last_fullsized":"2024-03-14T09:03:12.795450+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:46:05.658711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.791724+0000","last_change":"2024-03-14T09:02:59.985044+0000","last_active":"2024-03-14T09:03:12.791724+0000","last_peered":"2024-03-14T09:03:12.791724+0000","last_clean":"2024-03-14T09:03:12.791724+0000","last_became_active":"2024-03-14T09:02:59.984748+0000","last_became_peered":"2024-03-14T09:02:59.984748+0000","last_unstale":"2024-03-14T09:03:12.791724+0000","last_undegraded":"2024-03-14T09:03:12.791724+0000","last_fullsized":"2024-03-14T09:03:12.791724+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:43.222381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795662+0000","last_change":"2024-03-14T09:03:12.791708+0000","last_active":"2024-03-14T09:03:12.795662+0000","last_peered":"2024-03-14T09:03:12.795662+0000","last_clean":"2024-03-14T09:03:12.795662+0000","last_became_active":"2024-03-14T09:03:12.791445+0000","last_became_peered":"2024-03-14T09:03:12.791445+0000","last_unstale":"2024-03-14T09:03:12.795662+0000","last_undegraded":"2024-03-14T09:03:12.795662+0000","last_fullsized":"2024-03-14T09:03:12.795662+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:10.615193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764684+0000","last_change":"2024-03-14T09:02:59.992264+0000","last_active":"2024-03-14T09:03:12.764684+0000","last_peered":"2024-03-14T09:03:12.764684+0000","last_clean":"2024-03-14T09:03:12.764684+0000","last_became_active":"2024-03-14T09:02:59.991842+0000","last_became_peered":"2024-03-14T09:02:59.991842+0000","last_unstale":"2024-03-14T09:03:12.764684+0000","last_undegraded":"2024-03-14T09:03:12.764684+0000","last_fullsized":"2024-03-14T09:03:12.764684+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:53:54.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.794464+0000","last_change":"2024-03-14T09:03:12.783301+0000","last_active":"2024-03-14T09:03:12.794464+0000","last_peered":"2024-03-14T09:03:12.794464+0000","last_clean":"2024-03-14T09:03:12.794464+0000","last_became_active":"2024-03-14T09:03:12.783002+0000","last_became_peered":"2024-03-14T09:03:12.783002+0000","last_unstale":"2024-03-14T09:03:12.794464+0000","last_undegraded":"2024-03-14T09:03:12.794464+0000","last_fullsized":"2024-03-14T09:03:12.794464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:37.841882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774684+0000","last_change":"2024-03-14T09:02:59.992379+0000","last_active":"2024-03-14T09:03:12.774684+0000","last_peered":"2024-03-14T09:03:12.774684+0000","last_clean":"2024-03-14T09:03:12.774684+0000","last_became_active":"2024-03-14T09:02:59.992063+0000","last_became_peered":"2024-03-14T09:02:59.992063+0000","last_unstale":"2024-03-14T09:03:12.774684+0000","last_undegraded":"2024-03-14T09:03:12.774684+0000","last_fullsized":"2024-03-14T09:03:12.774684+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:13:59.225791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795345+0000","last_change":"2024-03-14T09:03:12.789967+0000","last_active":"2024-03-14T09:03:12.795345+0000","last_peered":"2024-03-14T09:03:12.795345+0000","last_clean":"2024-03-14T09:03:12.795345+0000","last_became_active":"2024-03-14T09:03:12.789721+0000","last_became_peered":"2024-03-14T09:03:12.789721+0000","last_unstale":"2024-03-14T09:03:12.795345+0000","last_undegraded":"2024-03-14T09:03:12.795345+0000","last_fullsized":"2024-03-14T09:03:12.795345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T21:02:39.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785393+0000","last_change":"2024-03-14T09:03:00.005111+0000","last_active":"2024-03-14T09:03:12.785393+0000","last_peered":"2024-03-14T09:03:12.785393+0000","last_clean":"2024-03-14T09:03:12.785393+0000","last_became_active":"2024-03-14T09:03:00.004879+0000","last_became_peered":"2024-03-14T09:03:00.004879+0000","last_unstale":"2024-03-14T09:03:12.785393+0000","last_undegraded":"2024-03-14T09:03:12.785393+0000","last_fullsized":"2024-03-14T09:03:12.785393+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786860+0000","last_change":"2024-03-14T09:03:12.781432+0000","last_active":"2024-03-14T09:03:12.786860+0000","last_peered":"2024-03-14T09:03:12.786860+0000","last_clean":"2024-03-14T09:03:12.786860+0000","last_became_active":"2024-03-14T09:03:12.781162+0000","last_became_peered":"2024-03-14T09:03:12.781162+0000","last_unstale":"2024-03-14T09:03:12.786860+0000","last_undegraded":"2024-03-14T09:03:12.786860+0000","last_fullsized":"2024-03-14T09:03:12.786860+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:42.260750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775136+0000","last_change":"2024-03-14T09:02:59.996712+0000","last_active":"2024-03-14T09:03:12.775136+0000","last_peered":"2024-03-14T09:03:12.775136+0000","last_clean":"2024-03-14T09:03:12.775136+0000","last_became_active":"2024-03-14T09:02:59.996399+0000","last_became_peered":"2024-03-14T09:02:59.996399+0000","last_unstale":"2024-03-14T09:03:12.775136+0000","last_undegraded":"2024-03-14T09:03:12.775136+0000","last_fullsized":"2024-03-14T09:03:12.775136+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:36.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798138+0000","last_change":"2024-03-14T09:03:12.789809+0000","last_active":"2024-03-14T09:03:12.798138+0000","last_peered":"2024-03-14T09:03:12.798138+0000","last_clean":"2024-03-14T09:03:12.798138+0000","last_became_active":"2024-03-14T09:03:12.789503+0000","last_became_peered":"2024-03-14T09:03:12.789503+0000","last_unstale":"2024-03-14T09:03:12.798138+0000","last_undegraded":"2024-03-14T09:03:12.798138+0000","last_fullsized":"2024-03-14T09:03:12.798138+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:39.707136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773318+0000","last_change":"2024-03-14T09:02:59.991769+0000","last_active":"2024-03-14T09:03:12.773318+0000","last_peered":"2024-03-14T09:03:12.773318+0000","last_clean":"2024-03-14T09:03:12.773318+0000","last_became_active":"2024-03-14T09:02:59.991530+0000","last_became_peered":"2024-03-14T09:02:59.991530+0000","last_unstale":"2024-03-14T09:03:12.773318+0000","last_undegraded":"2024-03-14T09:03:12.773318+0000","last_fullsized":"2024-03-14T09:03:12.773318+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:25.051250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795520+0000","last_change":"2024-03-14T09:03:12.788254+0000","last_active":"2024-03-14T09:03:12.795520+0000","last_peered":"2024-03-14T09:03:12.795520+0000","last_clean":"2024-03-14T09:03:12.795520+0000","last_became_active":"2024-03-14T09:03:12.787985+0000","last_became_peered":"2024-03-14T09:03:12.787985+0000","last_unstale":"2024-03-14T09:03:12.795520+0000","last_undegraded":"2024-03-14T09:03:12.795520+0000","last_fullsized":"2024-03-14T09:03:12.795520+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:03:32.019275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780792+0000","last_change":"2024-03-14T09:02:59.994062+0000","last_active":"2024-03-14T09:03:12.780792+0000","last_peered":"2024-03-14T09:03:12.780792+0000","last_clean":"2024-03-14T09:03:12.780792+0000","last_became_active":"2024-03-14T09:02:59.993787+0000","last_became_peered":"2024-03-14T09:02:59.993787+0000","last_unstale":"2024-03-14T09:03:12.780792+0000","last_undegraded":"2024-03-14T09:03:12.780792+0000","last_fullsized":"2024-03-14T09:03:12.780792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:33.757952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805459+0000","last_change":"2024-03-14T09:03:12.800195+0000","last_active":"2024-03-14T09:03:12.805459+0000","last_peered":"2024-03-14T09:03:12.805459+0000","last_clean":"2024-03-14T09:03:12.805459+0000","last_became_active":"2024-03-14T09:03:12.799676+0000","last_became_peered":"2024-03-14T09:03:12.799676+0000","last_unstale":"2024-03-14T09:03:12.805459+0000","last_undegraded":"2024-03-14T09:03:12.805459+0000","last_fullsized":"2024-03-14T09:03:12.805459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:44:09.365834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774501+0000","last_change":"2024-03-14T09:02:59.995639+0000","last_active":"2024-03-14T09:03:12.774501+0000","last_peered":"2024-03-14T09:03:12.774501+0000","last_clean":"2024-03-14T09:03:12.774501+0000","last_became_active":"2024-03-14T09:02:59.995353+0000","last_became_peered":"2024-03-14T09:02:59.995353+0000","last_unstale":"2024-03-14T09:03:12.774501+0000","last_undegraded":"2024-03-14T09:03:12.774501+0000","last_fullsized":"2024-03-14T09:03:12.774501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:24:57.493850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.807446+0000","last_change":"2024-03-14T09:03:12.800013+0000","last_active":"2024-03-14T09:03:12.807446+0000","last_peered":"2024-03-14T09:03:12.807446+0000","last_clean":"2024-03-14T09:03:12.807446+0000","last_became_active":"2024-03-14T09:03:12.796503+0000","last_became_peered":"2024-03-14T09:03:12.796503+0000","last_unstale":"2024-03-14T09:03:12.807446+0000","last_undegraded":"2024-03-14T09:03:12.807446+0000","last_fullsized":"2024-03-14T09:03:12.807446+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:33.336007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780210+0000","last_change":"2024-03-14T09:03:00.000150+0000","last_active":"2024-03-14T09:03:12.780210+0000","last_peered":"2024-03-14T09:03:12.780210+0000","last_clean":"2024-03-14T09:03:12.780210+0000","last_became_active":"2024-03-14T09:02:59.999849+0000","last_became_peered":"2024-03-14T09:02:59.999849+0000","last_unstale":"2024-03-14T09:03:12.780210+0000","last_undegraded":"2024-03-14T09:03:12.780210+0000","last_fullsized":"2024-03-14T09:03:12.780210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:41.379108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802780+0000","last_change":"2024-03-14T09:03:12.796550+0000","last_active":"2024-03-14T09:03:12.802780+0000","last_peered":"2024-03-14T09:03:12.802780+0000","last_clean":"2024-03-14T09:03:12.802780+0000","last_became_active":"2024-03-14T09:03:12.796116+0000","last_became_peered":"2024-03-14T09:03:12.796116+0000","last_unstale":"2024-03-14T09:03:12.802780+0000","last_undegraded":"2024-03-14T09:03:12.802780+0000","last_fullsized":"2024-03-14T09:03:12.802780+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:24.255695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778735+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:12.778735+0000","last_peered":"2024-03-14T09:03:12.778735+0000","last_clean":"2024-03-14T09:03:12.778735+0000","last_became_active":"2024-03-14T09:02:59.992766+0000","last_became_peered":"2024-03-14T09:02:59.992766+0000","last_unstale":"2024-03-14T09:03:12.778735+0000","last_undegraded":"2024-03-14T09:03:12.778735+0000","last_fullsized":"2024-03-14T09:03:12.778735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.790081+0000","last_change":"2024-03-14T09:03:12.785045+0000","last_active":"2024-03-14T09:03:12.790081+0000","last_peered":"2024-03-14T09:03:12.790081+0000","last_clean":"2024-03-14T09:03:12.790081+0000","last_became_active":"2024-03-14T09:03:12.784799+0000","last_became_peered":"2024-03-14T09:03:12.784799+0000","last_unstale":"2024-03-14T09:03:12.790081+0000","last_undegraded":"2024-03-14T09:03:12.790081+0000","last_fullsized":"2024-03-14T09:03:12.790081+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:59:39.800096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775125+0000","last_change":"2024-03-14T09:03:00.001238+0000","last_active":"2024-03-14T09:03:12.775125+0000","last_peered":"2024-03-14T09:03:12.775125+0000","last_clean":"2024-03-14T09:03:12.775125+0000","last_became_active":"2024-03-14T09:03:00.000923+0000","last_became_peered":"2024-03-14T09:03:00.000923+0000","last_unstale":"2024-03-14T09:03:12.775125+0000","last_undegraded":"2024-03-14T09:03:12.775125+0000","last_fullsized":"2024-03-14T09:03:12.775125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:02:36.743869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.807464+0000","last_change":"2024-03-14T09:03:12.800191+0000","last_active":"2024-03-14T09:03:12.807464+0000","last_peered":"2024-03-14T09:03:12.807464+0000","last_clean":"2024-03-14T09:03:12.807464+0000","last_became_active":"2024-03-14T09:03:12.797387+0000","last_became_peered":"2024-03-14T09:03:12.797387+0000","last_unstale":"2024-03-14T09:03:12.807464+0000","last_undegraded":"2024-03-14T09:03:12.807464+0000","last_fullsized":"2024-03-14T09:03:12.807464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:10.284481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774947+0000","last_change":"2024-03-14T09:02:59.991915+0000","last_active":"2024-03-14T09:03:12.774947+0000","last_peered":"2024-03-14T09:03:12.774947+0000","last_clean":"2024-03-14T09:03:12.774947+0000","last_became_active":"2024-03-14T09:02:59.991492+0000","last_became_peered":"2024-03-14T09:02:59.991492+0000","last_unstale":"2024-03-14T09:03:12.774947+0000","last_undegraded":"2024-03-14T09:03:12.774947+0000","last_fullsized":"2024-03-14T09:03:12.774947+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:06:09.473593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802651+0000","last_change":"2024-03-14T09:03:12.793601+0000","last_active":"2024-03-14T09:03:12.802651+0000","last_peered":"2024-03-14T09:03:12.802651+0000","last_clean":"2024-03-14T09:03:12.802651+0000","last_became_active":"2024-03-14T09:03:12.793332+0000","last_became_peered":"2024-03-14T09:03:12.793332+0000","last_unstale":"2024-03-14T09:03:12.802651+0000","last_undegraded":"2024-03-14T09:03:12.802651+0000","last_fullsized":"2024-03-14T09:03:12.802651+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:54.169331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764862+0000","last_change":"2024-03-14T09:02:59.996114+0000","last_active":"2024-03-14T09:03:12.764862+0000","last_peered":"2024-03-14T09:03:12.764862+0000","last_clean":"2024-03-14T09:03:12.764862+0000","last_became_active":"2024-03-14T09:02:59.995850+0000","last_became_peered":"2024-03-14T09:02:59.995850+0000","last_unstale":"2024-03-14T09:03:12.764862+0000","last_undegraded":"2024-03-14T09:03:12.764862+0000","last_fullsized":"2024-03-14T09:03:12.764862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798801+0000","last_change":"2024-03-14T09:03:12.794173+0000","last_active":"2024-03-14T09:03:12.798801+0000","last_peered":"2024-03-14T09:03:12.798801+0000","last_clean":"2024-03-14T09:03:12.798801+0000","last_became_active":"2024-03-14T09:03:12.793848+0000","last_became_peered":"2024-03-14T09:03:12.793848+0000","last_unstale":"2024-03-14T09:03:12.798801+0000","last_undegraded":"2024-03-14T09:03:12.798801+0000","last_fullsized":"2024-03-14T09:03:12.798801+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:28:47.355578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775168+0000","last_change":"2024-03-14T09:02:59.996079+0000","last_active":"2024-03-14T09:03:12.775168+0000","last_peered":"2024-03-14T09:03:12.775168+0000","last_clean":"2024-03-14T09:03:12.775168+0000","last_became_active":"2024-03-14T09:02:59.995828+0000","last_became_peered":"2024-03-14T09:02:59.995828+0000","last_unstale":"2024-03-14T09:03:12.775168+0000","last_undegraded":"2024-03-14T09:03:12.775168+0000","last_fullsized":"2024-03-14T09:03:12.775168+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:05:21.595863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.797224+0000","last_change":"2024-03-14T09:03:12.788361+0000","last_active":"2024-03-14T09:03:12.797224+0000","last_peered":"2024-03-14T09:03:12.797224+0000","last_clean":"2024-03-14T09:03:12.797224+0000","last_became_active":"2024-03-14T09:03:12.788104+0000","last_became_peered":"2024-03-14T09:03:12.788104+0000","last_unstale":"2024-03-14T09:03:12.797224+0000","last_undegraded":"2024-03-14T09:03:12.797224+0000","last_fullsized":"2024-03-14T09:03:12.797224+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:58:02.555656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774222+0000","last_change":"2024-03-14T09:02:59.987925+0000","last_active":"2024-03-14T09:03:12.774222+0000","last_peered":"2024-03-14T09:03:12.774222+0000","last_clean":"2024-03-14T09:03:12.774222+0000","last_became_active":"2024-03-14T09:02:59.987659+0000","last_became_peered":"2024-03-14T09:02:59.987659+0000","last_unstale":"2024-03-14T09:03:12.774222+0000","last_undegraded":"2024-03-14T09:03:12.774222+0000","last_fullsized":"2024-03-14T09:03:12.774222+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:32.090611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.794719+0000","last_change":"2024-03-14T09:03:12.788186+0000","last_active":"2024-03-14T09:03:12.794719+0000","last_peered":"2024-03-14T09:03:12.794719+0000","last_clean":"2024-03-14T09:03:12.794719+0000","last_became_active":"2024-03-14T09:03:12.787920+0000","last_became_peered":"2024-03-14T09:03:12.787920+0000","last_unstale":"2024-03-14T09:03:12.794719+0000","last_undegraded":"2024-03-14T09:03:12.794719+0000","last_fullsized":"2024-03-14T09:03:12.794719+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:13.654491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780309+0000","last_change":"2024-03-14T09:03:00.004269+0000","last_active":"2024-03-14T09:03:12.780309+0000","last_peered":"2024-03-14T09:03:12.780309+0000","last_clean":"2024-03-14T09:03:12.780309+0000","last_became_active":"2024-03-14T09:03:00.004095+0000","last_became_peered":"2024-03-14T09:03:00.004095+0000","last_unstale":"2024-03-14T09:03:12.780309+0000","last_undegraded":"2024-03-14T09:03:12.780309+0000","last_fullsized":"2024-03-14T09:03:12.780309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795391+0000","last_change":"2024-03-14T09:03:12.790219+0000","last_active":"2024-03-14T09:03:12.795391+0000","last_peered":"2024-03-14T09:03:12.795391+0000","last_clean":"2024-03-14T09:03:12.795391+0000","last_became_active":"2024-03-14T09:03:12.789879+0000","last_became_peered":"2024-03-14T09:03:12.789879+0000","last_unstale":"2024-03-14T09:03:12.795391+0000","last_undegraded":"2024-03-14T09:03:12.795391+0000","last_fullsized":"2024-03-14T09:03:12.795391+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:35:08.337267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.770597+0000","last_change":"2024-03-14T09:03:00.001709+0000","last_active":"2024-03-14T09:03:12.770597+0000","last_peered":"2024-03-14T09:03:12.770597+0000","last_clean":"2024-03-14T09:03:12.770597+0000","last_became_active":"2024-03-14T09:03:00.001383+0000","last_became_peered":"2024-03-14T09:03:00.001383+0000","last_unstale":"2024-03-14T09:03:12.770597+0000","last_undegraded":"2024-03-14T09:03:12.770597+0000","last_fullsized":"2024-03-14T09:03:12.770597+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:38:34.607023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798681+0000","last_change":"2024-03-14T09:03:12.793887+0000","last_active":"2024-03-14T09:03:12.798681+0000","last_peered":"2024-03-14T09:03:12.798681+0000","last_clean":"2024-03-14T09:03:12.798681+0000","last_became_active":"2024-03-14T09:03:12.793595+0000","last_became_peered":"2024-03-14T09:03:12.793595+0000","last_unstale":"2024-03-14T09:03:12.798681+0000","last_undegraded":"2024-03-14T09:03:12.798681+0000","last_fullsized":"2024-03-14T09:03:12.798681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764764+0000","last_change":"2024-03-14T09:02:59.998723+0000","last_active":"2024-03-14T09:03:12.764764+0000","last_peered":"2024-03-14T09:03:12.764764+0000","last_clean":"2024-03-14T09:03:12.764764+0000","last_became_active":"2024-03-14T09:02:59.998445+0000","last_became_peered":"2024-03-14T09:02:59.998445+0000","last_unstale":"2024-03-14T09:03:12.764764+0000","last_undegraded":"2024-03-14T09:03:12.764764+0000","last_fullsized":"2024-03-14T09:03:12.764764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:00.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.797866+0000","last_change":"2024-03-14T09:03:12.788823+0000","last_active":"2024-03-14T09:03:12.797866+0000","last_peered":"2024-03-14T09:03:12.797866+0000","last_clean":"2024-03-14T09:03:12.797866+0000","last_became_active":"2024-03-14T09:03:12.788559+0000","last_became_peered":"2024-03-14T09:03:12.788559+0000","last_unstale":"2024-03-14T09:03:12.797866+0000","last_undegraded":"2024-03-14T09:03:12.797866+0000","last_fullsized":"2024-03-14T09:03:12.797866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:54.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.796132+0000","last_change":"2024-03-14T09:03:12.371272+0000","last_active":"2024-03-14T09:03:12.796132+0000","last_peered":"2024-03-14T09:03:12.796132+0000","last_clean":"2024-03-14T09:03:12.796132+0000","last_became_active":"2024-03-14T09:03:12.370996+0000","last_became_peered":"2024-03-14T09:03:12.370996+0000","last_unstale":"2024-03-14T09:03:12.796132+0000","last_undegraded":"2024-03-14T09:03:12.796132+0000","last_fullsized":"2024-03-14T09:03:12.796132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:19:44.738510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764791+0000","last_change":"2024-03-14T09:03:00.003564+0000","last_active":"2024-03-14T09:03:12.764791+0000","last_peered":"2024-03-14T09:03:12.764791+0000","last_clean":"2024-03-14T09:03:12.764791+0000","last_became_active":"2024-03-14T09:03:00.003327+0000","last_became_peered":"2024-03-14T09:03:00.003327+0000","last_unstale":"2024-03-14T09:03:12.764791+0000","last_undegraded":"2024-03-14T09:03:12.764791+0000","last_fullsized":"2024-03-14T09:03:12.764791+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764731+0000","last_change":"2024-03-14T09:02:59.584671+0000","last_active":"2024-03-14T09:03:12.764731+0000","last_peered":"2024-03-14T09:03:12.764731+0000","last_clean":"2024-03-14T09:03:12.764731+0000","last_became_active":"2024-03-14T09:02:59.584365+0000","last_became_peered":"2024-03-14T09:02:59.584365+0000","last_unstale":"2024-03-14T09:03:12.764731+0000","last_undegraded":"2024-03-14T09:03:12.764731+0000","last_fullsized":"2024-03-14T09:03:12.764731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805409+0000","last_change":"2024-03-14T09:03:12.798922+0000","last_active":"2024-03-14T09:03:12.805409+0000","last_peered":"2024-03-14T09:03:12.805409+0000","last_clean":"2024-03-14T09:03:12.805409+0000","last_became_active":"2024-03-14T09:03:12.797976+0000","last_became_peered":"2024-03-14T09:03:12.797976+0000","last_unstale":"2024-03-14T09:03:12.805409+0000","last_undegraded":"2024-03-14T09:03:12.805409+0000","last_fullsized":"2024-03-14T09:03:12.805409+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:29:41.119050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798569+0000","last_change":"2024-03-14T09:03:12.374278+0000","last_active":"2024-03-14T09:03:12.798569+0000","last_peered":"2024-03-14T09:03:12.798569+0000","last_clean":"2024-03-14T09:03:12.798569+0000","last_became_active":"2024-03-14T09:03:12.373968+0000","last_became_peered":"2024-03-14T09:03:12.373968+0000","last_unstale":"2024-03-14T09:03:12.798569+0000","last_undegraded":"2024-03-14T09:03:12.798569+0000","last_fullsized":"2024-03-14T09:03:12.798569+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:40.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768519+0000","last_change":"2024-03-14T09:02:59.998579+0000","last_active":"2024-03-14T09:03:12.768519+0000","last_peered":"2024-03-14T09:03:12.768519+0000","last_clean":"2024-03-14T09:03:12.768519+0000","last_became_active":"2024-03-14T09:02:59.998316+0000","last_became_peered":"2024-03-14T09:02:59.998316+0000","last_unstale":"2024-03-14T09:03:12.768519+0000","last_undegraded":"2024-03-14T09:03:12.768519+0000","last_fullsized":"2024-03-14T09:03:12.768519+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:53.670276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773421+0000","last_change":"2024-03-14T09:02:59.580673+0000","last_active":"2024-03-14T09:03:12.773421+0000","last_peered":"2024-03-14T09:03:12.773421+0000","last_clean":"2024-03-14T09:03:12.773421+0000","last_became_active":"2024-03-14T09:02:59.580416+0000","last_became_peered":"2024-03-14T09:02:59.580416+0000","last_unstale":"2024-03-14T09:03:12.773421+0000","last_undegraded":"2024-03-14T09:03:12.773421+0000","last_fullsized":"2024-03-14T09:03:12.773421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:54.518315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805346+0000","last_change":"2024-03-14T09:03:12.798203+0000","last_active":"2024-03-14T09:03:12.805346+0000","last_peered":"2024-03-14T09:03:12.805346+0000","last_clean":"2024-03-14T09:03:12.805346+0000","last_became_active":"2024-03-14T09:03:12.796089+0000","last_became_peered":"2024-03-14T09:03:12.796089+0000","last_unstale":"2024-03-14T09:03:12.805346+0000","last_undegraded":"2024-03-14T09:03:12.805346+0000","last_fullsized":"2024-03-14T09:03:12.805346+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"54'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802545+0000","last_change":"2024-03-14T09:03:12.373911+0000","last_active":"2024-03-14T09:03:12.802545+0000","last_peered":"2024-03-14T09:03:12.802545+0000","last_clean":"2024-03-14T09:03:12.802545+0000","last_became_active":"2024-03-14T09:03:12.373633+0000","last_became_peered":"2024-03-14T09:03:12.373633+0000","last_unstale":"2024-03-14T09:03:12.802545+0000","last_undegraded":"2024-03-14T09:03:12.802545+0000","last_fullsized":"2024-03-14T09:03:12.802545+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764756+0000","last_change":"2024-03-14T09:03:00.003942+0000","last_active":"2024-03-14T09:03:12.764756+0000","last_peered":"2024-03-14T09:03:12.764756+0000","last_clean":"2024-03-14T09:03:12.764756+0000","last_became_active":"2024-03-14T09:03:00.003629+0000","last_became_peered":"2024-03-14T09:03:00.003629+0000","last_unstale":"2024-03-14T09:03:12.764756+0000","last_undegraded":"2024-03-14T09:03:12.764756+0000","last_fullsized":"2024-03-14T09:03:12.764756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764611+0000","last_change":"2024-03-14T09:02:59.582367+0000","last_active":"2024-03-14T09:03:12.764611+0000","last_peered":"2024-03-14T09:03:12.764611+0000","last_clean":"2024-03-14T09:03:12.764611+0000","last_became_active":"2024-03-14T09:02:59.582107+0000","last_became_peered":"2024-03-14T09:02:59.582107+0000","last_unstale":"2024-03-14T09:03:12.764611+0000","last_undegraded":"2024-03-14T09:03:12.764611+0000","last_fullsized":"2024-03-14T09:03:12.764611+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786945+0000","last_change":"2024-03-14T09:03:12.783128+0000","last_active":"2024-03-14T09:03:12.786945+0000","last_peered":"2024-03-14T09:03:12.786945+0000","last_clean":"2024-03-14T09:03:12.786945+0000","last_became_active":"2024-03-14T09:03:12.782861+0000","last_became_peered":"2024-03-14T09:03:12.782861+0000","last_unstale":"2024-03-14T09:03:12.786945+0000","last_undegraded":"2024-03-14T09:03:12.786945+0000","last_fullsized":"2024-03-14T09:03:12.786945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:46.521655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778154+0000","last_change":"2024-03-14T09:03:12.370015+0000","last_active":"2024-03-14T09:03:12.778154+0000","last_peered":"2024-03-14T09:03:12.778154+0000","last_clean":"2024-03-14T09:03:12.778154+0000","last_became_active":"2024-03-14T09:03:12.369735+0000","last_became_peered":"2024-03-14T09:03:12.369735+0000","last_unstale":"2024-03-14T09:03:12.778154+0000","last_undegraded":"2024-03-14T09:03:12.778154+0000","last_fullsized":"2024-03-14T09:03:12.778154+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:25:09.282747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764810+0000","last_change":"2024-03-14T09:02:59.993101+0000","last_active":"2024-03-14T09:03:12.764810+0000","last_peered":"2024-03-14T09:03:12.764810+0000","last_clean":"2024-03-14T09:03:12.764810+0000","last_became_active":"2024-03-14T09:02:59.992716+0000","last_became_peered":"2024-03-14T09:02:59.992716+0000","last_unstale":"2024-03-14T09:03:12.764810+0000","last_undegraded":"2024-03-14T09:03:12.764810+0000","last_fullsized":"2024-03-14T09:03:12.764810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:33.117417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773711+0000","last_change":"2024-03-14T09:02:59.580357+0000","last_active":"2024-03-14T09:03:12.773711+0000","last_peered":"2024-03-14T09:03:12.773711+0000","last_clean":"2024-03-14T09:03:12.773711+0000","last_became_active":"2024-03-14T09:02:59.579606+0000","last_became_peered":"2024-03-14T09:02:59.579606+0000","last_unstale":"2024-03-14T09:03:12.773711+0000","last_undegraded":"2024-03-14T09:03:12.773711+0000","last_fullsized":"2024-03-14T09:03:12.773711+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798393+0000","last_change":"2024-03-14T09:03:12.793590+0000","last_active":"2024-03-14T09:03:12.798393+0000","last_peered":"2024-03-14T09:03:12.798393+0000","last_clean":"2024-03-14T09:03:12.798393+0000","last_became_active":"2024-03-14T09:03:12.793304+0000","last_became_peered":"2024-03-14T09:03:12.793304+0000","last_unstale":"2024-03-14T09:03:12.798393+0000","last_undegraded":"2024-03-14T09:03:12.798393+0000","last_fullsized":"2024-03-14T09:03:12.798393+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:44.778260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.767143+0000","last_change":"2024-03-14T09:03:12.372188+0000","last_active":"2024-03-14T09:03:12.767143+0000","last_peered":"2024-03-14T09:03:12.767143+0000","last_clean":"2024-03-14T09:03:12.767143+0000","last_became_active":"2024-03-14T09:03:12.371919+0000","last_became_peered":"2024-03-14T09:03:12.371919+0000","last_unstale":"2024-03-14T09:03:12.767143+0000","last_undegraded":"2024-03-14T09:03:12.767143+0000","last_fullsized":"2024-03-14T09:03:12.767143+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774545+0000","last_change":"2024-03-14T09:02:59.995203+0000","last_active":"2024-03-14T09:03:12.774545+0000","last_peered":"2024-03-14T09:03:12.774545+0000","last_clean":"2024-03-14T09:03:12.774545+0000","last_became_active":"2024-03-14T09:02:59.994834+0000","last_became_peered":"2024-03-14T09:02:59.994834+0000","last_unstale":"2024-03-14T09:03:12.774545+0000","last_undegraded":"2024-03-14T09:03:12.774545+0000","last_fullsized":"2024-03-14T09:03:12.774545+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:01:11.465064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785124+0000","last_change":"2024-03-14T09:02:59.576916+0000","last_active":"2024-03-14T09:03:12.785124+0000","last_peered":"2024-03-14T09:03:12.785124+0000","last_clean":"2024-03-14T09:03:12.785124+0000","last_became_active":"2024-03-14T09:02:59.576632+0000","last_became_peered":"2024-03-14T09:02:59.576632+0000","last_unstale":"2024-03-14T09:03:12.785124+0000","last_undegraded":"2024-03-14T09:03:12.785124+0000","last_fullsized":"2024-03-14T09:03:12.785124+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:26.703840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792536+0000","last_change":"2024-03-14T09:03:12.784445+0000","last_active":"2024-03-14T09:03:12.792536+0000","last_peered":"2024-03-14T09:03:12.792536+0000","last_clean":"2024-03-14T09:03:12.792536+0000","last_became_active":"2024-03-14T09:03:12.784214+0000","last_became_peered":"2024-03-14T09:03:12.784214+0000","last_unstale":"2024-03-14T09:03:12.792536+0000","last_undegraded":"2024-03-14T09:03:12.792536+0000","last_fullsized":"2024-03-14T09:03:12.792536+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.769709+0000","last_change":"2024-03-14T09:03:12.369918+0000","last_active":"2024-03-14T09:03:12.769709+0000","last_peered":"2024-03-14T09:03:12.769709+0000","last_clean":"2024-03-14T09:03:12.769709+0000","last_became_active":"2024-03-14T09:03:12.369599+0000","last_became_peered":"2024-03-14T09:03:12.369599+0000","last_unstale":"2024-03-14T09:03:12.769709+0000","last_undegraded":"2024-03-14T09:03:12.769709+0000","last_fullsized":"2024-03-14T09:03:12.769709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:09:52.629290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774307+0000","last_change":"2024-03-14T09:02:59.994318+0000","last_active":"2024-03-14T09:03:12.774307+0000","last_peered":"2024-03-14T09:03:12.774307+0000","last_clean":"2024-03-14T09:03:12.774307+0000","last_became_active":"2024-03-14T09:02:59.994061+0000","last_became_peered":"2024-03-14T09:02:59.994061+0000","last_unstale":"2024-03-14T09:03:12.774307+0000","last_undegraded":"2024-03-14T09:03:12.774307+0000","last_fullsized":"2024-03-14T09:03:12.774307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:34:46.343906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.765055+0000","last_change":"2024-03-14T09:02:59.577008+0000","last_active":"2024-03-14T09:03:12.765055+0000","last_peered":"2024-03-14T09:03:12.765055+0000","last_clean":"2024-03-14T09:03:12.765055+0000","last_became_active":"2024-03-14T09:02:59.576724+0000","last_became_peered":"2024-03-14T09:02:59.576724+0000","last_unstale":"2024-03-14T09:03:12.765055+0000","last_undegraded":"2024-03-14T09:03:12.765055+0000","last_fullsized":"2024-03-14T09:03:12.765055+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:13:27.225529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.788467+0000","last_change":"2024-03-14T09:03:12.780975+0000","last_active":"2024-03-14T09:03:12.788467+0000","last_peered":"2024-03-14T09:03:12.788467+0000","last_clean":"2024-03-14T09:03:12.788467+0000","last_became_active":"2024-03-14T09:03:12.780681+0000","last_became_peered":"2024-03-14T09:03:12.780681+0000","last_unstale":"2024-03-14T09:03:12.788467+0000","last_undegraded":"2024-03-14T09:03:12.788467+0000","last_fullsized":"2024-03-14T09:03:12.788467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:47.894294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.784811+0000","last_change":"2024-03-14T09:03:12.373841+0000","last_active":"2024-03-14T09:03:12.784811+0000","last_peered":"2024-03-14T09:03:12.784811+0000","last_clean":"2024-03-14T09:03:12.784811+0000","last_became_active":"2024-03-14T09:03:12.373551+0000","last_became_peered":"2024-03-14T09:03:12.373551+0000","last_unstale":"2024-03-14T09:03:12.784811+0000","last_undegraded":"2024-03-14T09:03:12.784811+0000","last_fullsized":"2024-03-14T09:03:12.784811+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:35:12.965540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778517+0000","last_change":"2024-03-14T09:03:00.000645+0000","last_active":"2024-03-14T09:03:12.778517+0000","last_peered":"2024-03-14T09:03:12.778517+0000","last_clean":"2024-03-14T09:03:12.778517+0000","last_became_active":"2024-03-14T09:03:00.000365+0000","last_became_peered":"2024-03-14T09:03:00.000365+0000","last_unstale":"2024-03-14T09:03:12.778517+0000","last_undegraded":"2024-03-14T09:03:12.778517+0000","last_fullsized":"2024-03-14T09:03:12.778517+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:03:38.802121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774069+0000","last_change":"2024-03-14T09:02:59.570165+0000","last_active":"2024-03-14T09:03:12.774069+0000","last_peered":"2024-03-14T09:03:12.774069+0000","last_clean":"2024-03-14T09:03:12.774069+0000","last_became_active":"2024-03-14T09:02:59.569815+0000","last_became_peered":"2024-03-14T09:02:59.569815+0000","last_unstale":"2024-03-14T09:03:12.774069+0000","last_undegraded":"2024-03-14T09:03:12.774069+0000","last_fullsized":"2024-03-14T09:03:12.774069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:29:40.490280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802889+0000","last_change":"2024-03-14T09:03:12.796818+0000","last_active":"2024-03-14T09:03:12.802889+0000","last_peered":"2024-03-14T09:03:12.802889+0000","last_clean":"2024-03-14T09:03:12.802889+0000","last_became_active":"2024-03-14T09:03:12.796213+0000","last_became_peered":"2024-03-14T09:03:12.796213+0000","last_unstale":"2024-03-14T09:03:12.802889+0000","last_undegraded":"2024-03-14T09:03:12.802889+0000","last_fullsized":"2024-03-14T09:03:12.802889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.790230+0000","last_change":"2024-03-14T09:03:12.366900+0000","last_active":"2024-03-14T09:03:12.790230+0000","last_peered":"2024-03-14T09:03:12.790230+0000","last_clean":"2024-03-14T09:03:12.790230+0000","last_became_active":"2024-03-14T09:03:12.366301+0000","last_became_peered":"2024-03-14T09:03:12.366301+0000","last_unstale":"2024-03-14T09:03:12.790230+0000","last_undegraded":"2024-03-14T09:03:12.790230+0000","last_fullsized":"2024-03-14T09:03:12.790230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:25:11.884116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773893+0000","last_change":"2024-03-14T09:02:59.989907+0000","last_active":"2024-03-14T09:03:12.773893+0000","last_peered":"2024-03-14T09:03:12.773893+0000","last_clean":"2024-03-14T09:03:12.773893+0000","last_became_active":"2024-03-14T09:02:59.989652+0000","last_became_peered":"2024-03-14T09:02:59.989652+0000","last_unstale":"2024-03-14T09:03:12.773893+0000","last_undegraded":"2024-03-14T09:03:12.773893+0000","last_fullsized":"2024-03-14T09:03:12.773893+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:20:41.378699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785408+0000","last_change":"2024-03-14T09:02:59.586262+0000","last_active":"2024-03-14T09:03:12.785408+0000","last_peered":"2024-03-14T09:03:12.785408+0000","last_clean":"2024-03-14T09:03:12.785408+0000","last_became_active":"2024-03-14T09:02:59.585946+0000","last_became_peered":"2024-03-14T09:02:59.585946+0000","last_unstale":"2024-03-14T09:03:12.785408+0000","last_undegraded":"2024-03-14T09:03:12.785408+0000","last_fullsized":"2024-03-14T09:03:12.785408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:57.298932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.799938+0000","last_change":"2024-03-14T09:03:12.796856+0000","last_active":"2024-03-14T09:03:12.799938+0000","last_peered":"2024-03-14T09:03:12.799938+0000","last_clean":"2024-03-14T09:03:12.799938+0000","last_became_active":"2024-03-14T09:03:12.794750+0000","last_became_peered":"2024-03-14T09:03:12.794750+0000","last_unstale":"2024-03-14T09:03:12.799938+0000","last_undegraded":"2024-03-14T09:03:12.799938+0000","last_fullsized":"2024-03-14T09:03:12.799938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:04.178707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"54'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.787335+0000","last_change":"2024-03-14T09:03:12.367305+0000","last_active":"2024-03-14T09:03:12.787335+0000","last_peered":"2024-03-14T09:03:12.787335+0000","last_clean":"2024-03-14T09:03:12.787335+0000","last_became_active":"2024-03-14T09:03:12.367020+0000","last_became_peered":"2024-03-14T09:03:12.367020+0000","last_unstale":"2024-03-14T09:03:12.787335+0000","last_undegraded":"2024-03-14T09:03:12.787335+0000","last_fullsized":"2024-03-14T09:03:12.787335+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:26:39.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.783529+0000","last_change":"2024-03-14T09:02:59.993777+0000","last_active":"2024-03-14T09:03:12.783529+0000","last_peered":"2024-03-14T09:03:12.783529+0000","last_clean":"2024-03-14T09:03:12.783529+0000","last_became_active":"2024-03-14T09:02:59.993468+0000","last_became_peered":"2024-03-14T09:02:59.993468+0000","last_unstale":"2024-03-14T09:03:12.783529+0000","last_undegraded":"2024-03-14T09:03:12.783529+0000","last_fullsized":"2024-03-14T09:03:12.783529+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:52.273680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786447+0000","last_change":"2024-03-14T09:02:59.592728+0000","last_active":"2024-03-14T09:03:12.786447+0000","last_peered":"2024-03-14T09:03:12.786447+0000","last_clean":"2024-03-14T09:03:12.786447+0000","last_became_active":"2024-03-14T09:02:59.592466+0000","last_became_peered":"2024-03-14T09:02:59.592466+0000","last_unstale":"2024-03-14T09:03:12.786447+0000","last_undegraded":"2024-03-14T09:03:12.786447+0000","last_fullsized":"2024-03-14T09:03:12.786447+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786881+0000","last_change":"2024-03-14T09:03:12.782365+0000","last_active":"2024-03-14T09:03:12.786881+0000","last_peered":"2024-03-14T09:03:12.786881+0000","last_clean":"2024-03-14T09:03:12.786881+0000","last_became_active":"2024-03-14T09:03:12.782108+0000","last_became_peered":"2024-03-14T09:03:12.782108+0000","last_unstale":"2024-03-14T09:03:12.786881+0000","last_undegraded":"2024-03-14T09:03:12.786881+0000","last_fullsized":"2024-03-14T09:03:12.786881+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:07.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"54'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795880+0000","last_change":"2024-03-14T09:03:12.370627+0000","last_active":"2024-03-14T09:03:12.795880+0000","last_peered":"2024-03-14T09:03:12.795880+0000","last_clean":"2024-03-14T09:03:12.795880+0000","last_became_active":"2024-03-14T09:03:12.370314+0000","last_became_peered":"2024-03-14T09:03:12.370314+0000","last_unstale":"2024-03-14T09:03:12.795880+0000","last_undegraded":"2024-03-14T09:03:12.795880+0000","last_fullsized":"2024-03-14T09:03:12.795880+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:09.613459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786226+0000","last_change":"2024-03-14T09:03:00.002147+0000","last_active":"2024-03-14T09:03:12.786226+0000","last_peered":"2024-03-14T09:03:12.786226+0000","last_clean":"2024-03-14T09:03:12.786226+0000","last_became_active":"2024-03-14T09:03:00.001865+0000","last_became_peered":"2024-03-14T09:03:00.001865+0000","last_unstale":"2024-03-14T09:03:12.786226+0000","last_undegraded":"2024-03-14T09:03:12.786226+0000","last_fullsized":"2024-03-14T09:03:12.786226+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:28.776151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773441+0000","last_change":"2024-03-14T09:02:59.580724+0000","last_active":"2024-03-14T09:03:12.773441+0000","last_peered":"2024-03-14T09:03:12.773441+0000","last_clean":"2024-03-14T09:03:12.773441+0000","last_became_active":"2024-03-14T09:02:59.580469+0000","last_became_peered":"2024-03-14T09:02:59.580469+0000","last_unstale":"2024-03-14T09:03:12.773441+0000","last_undegraded":"2024-03-14T09:03:12.773441+0000","last_fullsized":"2024-03-14T09:03:12.773441+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:27.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792746+0000","last_change":"2024-03-14T09:03:12.784113+0000","last_active":"2024-03-14T09:03:12.792746+0000","last_peered":"2024-03-14T09:03:12.792746+0000","last_clean":"2024-03-14T09:03:12.792746+0000","last_became_active":"2024-03-14T09:03:12.783844+0000","last_became_peered":"2024-03-14T09:03:12.783844+0000","last_unstale":"2024-03-14T09:03:12.792746+0000","last_undegraded":"2024-03-14T09:03:12.792746+0000","last_fullsized":"2024-03-14T09:03:12.792746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:43.845415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782069+0000","last_change":"2024-03-14T09:03:12.374028+0000","last_active":"2024-03-14T09:03:12.782069+0000","last_peered":"2024-03-14T09:03:12.782069+0000","last_clean":"2024-03-14T09:03:12.782069+0000","last_became_active":"2024-03-14T09:03:12.373770+0000","last_became_peered":"2024-03-14T09:03:12.373770+0000","last_unstale":"2024-03-14T09:03:12.782069+0000","last_undegraded":"2024-03-14T09:03:12.782069+0000","last_fullsized":"2024-03-14T09:03:12.782069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:51.114973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768339+0000","last_change":"2024-03-14T09:02:59.991430+0000","last_active":"2024-03-14T09:03:12.768339+0000","last_peered":"2024-03-14T09:03:12.768339+0000","last_clean":"2024-03-14T09:03:12.768339+0000","last_became_active":"2024-03-14T09:02:59.991194+0000","last_became_peered":"2024-03-14T09:02:59.991194+0000","last_unstale":"2024-03-14T09:03:12.768339+0000","last_undegraded":"2024-03-14T09:03:12.768339+0000","last_fullsized":"2024-03-14T09:03:12.768339+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:54.574682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785699+0000","last_change":"2024-03-14T09:02:59.581170+0000","last_active":"2024-03-14T09:03:12.785699+0000","last_peered":"2024-03-14T09:03:12.785699+0000","last_clean":"2024-03-14T09:03:12.785699+0000","last_became_active":"2024-03-14T09:02:59.580842+0000","last_became_peered":"2024-03-14T09:02:59.580842+0000","last_unstale":"2024-03-14T09:03:12.785699+0000","last_undegraded":"2024-03-14T09:03:12.785699+0000","last_fullsized":"2024-03-14T09:03:12.785699+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805383+0000","last_change":"2024-03-14T09:03:12.798525+0000","last_active":"2024-03-14T09:03:12.805383+0000","last_peered":"2024-03-14T09:03:12.805383+0000","last_clean":"2024-03-14T09:03:12.805383+0000","last_became_active":"2024-03-14T09:03:12.796144+0000","last_became_peered":"2024-03-14T09:03:12.796144+0000","last_unstale":"2024-03-14T09:03:12.805383+0000","last_undegraded":"2024-03-14T09:03:12.805383+0000","last_fullsized":"2024-03-14T09:03:12.805383+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:43:47.462319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.779209+0000","last_change":"2024-03-14T09:03:12.367742+0000","last_active":"2024-03-14T09:03:12.779209+0000","last_peered":"2024-03-14T09:03:12.779209+0000","last_clean":"2024-03-14T09:03:12.779209+0000","last_became_active":"2024-03-14T09:03:12.367471+0000","last_became_peered":"2024-03-14T09:03:12.367471+0000","last_unstale":"2024-03-14T09:03:12.779209+0000","last_undegraded":"2024-03-14T09:03:12.779209+0000","last_fullsized":"2024-03-14T09:03:12.779209+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:34.678842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785435+0000","last_change":"2024-03-14T09:03:00.003854+0000","last_active":"2024-03-14T09:03:12.785435+0000","last_peered":"2024-03-14T09:03:12.785435+0000","last_clean":"2024-03-14T09:03:12.785435+0000","last_became_active":"2024-03-14T09:03:00.003478+0000","last_became_peered":"2024-03-14T09:03:00.003478+0000","last_unstale":"2024-03-14T09:03:12.785435+0000","last_undegraded":"2024-03-14T09:03:12.785435+0000","last_fullsized":"2024-03-14T09:03:12.785435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:01.269806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'8","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782667+0000","last_change":"2024-03-14T09:02:59.578418+0000","last_active":"2024-03-14T09:03:12.782667+0000","last_peered":"2024-03-14T09:03:12.782667+0000","last_clean":"2024-03-14T09:03:12.782667+0000","last_became_active":"2024-03-14T09:02:59.577974+0000","last_became_peered":"2024-03-14T09:02:59.577974+0000","last_unstale":"2024-03-14T09:03:12.782667+0000","last_undegraded":"2024-03-14T09:03:12.782667+0000","last_fullsized":"2024-03-14T09:03:12.782667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:14:03.123322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795368+0000","last_change":"2024-03-14T09:03:12.791584+0000","last_active":"2024-03-14T09:03:12.795368+0000","last_peered":"2024-03-14T09:03:12.795368+0000","last_clean":"2024-03-14T09:03:12.795368+0000","last_became_active":"2024-03-14T09:03:12.791360+0000","last_became_peered":"2024-03-14T09:03:12.791360+0000","last_unstale":"2024-03-14T09:03:12.795368+0000","last_undegraded":"2024-03-14T09:03:12.795368+0000","last_fullsized":"2024-03-14T09:03:12.795368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:19:29.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774348+0000","last_change":"2024-03-14T09:03:12.366980+0000","last_active":"2024-03-14T09:03:12.774348+0000","last_peered":"2024-03-14T09:03:12.774348+0000","last_clean":"2024-03-14T09:03:12.774348+0000","last_became_active":"2024-03-14T09:03:12.366728+0000","last_became_peered":"2024-03-14T09:03:12.366728+0000","last_unstale":"2024-03-14T09:03:12.774348+0000","last_undegraded":"2024-03-14T09:03:12.774348+0000","last_fullsized":"2024-03-14T09:03:12.774348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:53.796599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772793+0000","last_change":"2024-03-14T09:03:00.001689+0000","last_active":"2024-03-14T09:03:12.772793+0000","last_peered":"2024-03-14T09:03:12.772793+0000","last_clean":"2024-03-14T09:03:12.772793+0000","last_became_active":"2024-03-14T09:03:00.001402+0000","last_became_peered":"2024-03-14T09:03:00.001402+0000","last_unstale":"2024-03-14T09:03:12.772793+0000","last_undegraded":"2024-03-14T09:03:12.772793+0000","last_fullsized":"2024-03-14T09:03:12.772793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:15:33.731023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774459+0000","last_change":"2024-03-14T09:02:59.581663+0000","last_active":"2024-03-14T09:03:12.774459+0000","last_peered":"2024-03-14T09:03:12.774459+0000","last_clean":"2024-03-14T09:03:12.774459+0000","last_became_active":"2024-03-14T09:02:59.581393+0000","last_became_peered":"2024-03-14T09:02:59.581393+0000","last_unstale":"2024-03-14T09:03:12.774459+0000","last_undegraded":"2024-03-14T09:03:12.774459+0000","last_fullsized":"2024-03-14T09:03:12.774459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:24:59.090186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786815+0000","last_change":"2024-03-14T09:03:12.781467+0000","last_active":"2024-03-14T09:03:12.786815+0000","last_peered":"2024-03-14T09:03:12.786815+0000","last_clean":"2024-03-14T09:03:12.786815+0000","last_became_active":"2024-03-14T09:03:12.781203+0000","last_became_peered":"2024-03-14T09:03:12.781203+0000","last_unstale":"2024-03-14T09:03:12.786815+0000","last_undegraded":"2024-03-14T09:03:12.786815+0000","last_fullsized":"2024-03-14T09:03:12.786815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:21:24.061643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"54'8","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.803464+0000","last_change":"2024-03-14T09:03:12.371369+0000","last_active":"2024-03-14T09:03:12.803464+0000","last_peered":"2024-03-14T09:03:12.803464+0000","last_clean":"2024-03-14T09:03:12.803464+0000","last_became_active":"2024-03-14T09:03:12.371087+0000","last_became_peered":"2024-03-14T09:03:12.371087+0000","last_unstale":"2024-03-14T09:03:12.803464+0000","last_undegraded":"2024-03-14T09:03:12.803464+0000","last_fullsized":"2024-03-14T09:03:12.803464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:55:38.705402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780694+0000","last_change":"2024-03-14T09:02:59.995857+0000","last_active":"2024-03-14T09:03:12.780694+0000","last_peered":"2024-03-14T09:03:12.780694+0000","last_clean":"2024-03-14T09:03:12.780694+0000","last_became_active":"2024-03-14T09:02:59.995533+0000","last_became_peered":"2024-03-14T09:02:59.995533+0000","last_unstale":"2024-03-14T09:03:12.780694+0000","last_undegraded":"2024-03-14T09:03:12.780694+0000","last_fullsized":"2024-03-14T09:03:12.780694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.779092+0000","last_change":"2024-03-14T09:02:59.580620+0000","last_active":"2024-03-14T09:03:12.779092+0000","last_peered":"2024-03-14T09:03:12.779092+0000","last_clean":"2024-03-14T09:03:12.779092+0000","last_became_active":"2024-03-14T09:02:59.580320+0000","last_became_peered":"2024-03-14T09:02:59.580320+0000","last_unstale":"2024-03-14T09:03:12.779092+0000","last_undegraded":"2024-03-14T09:03:12.779092+0000","last_fullsized":"2024-03-14T09:03:12.779092+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.789385+0000","last_change":"2024-03-14T09:03:12.786865+0000","last_active":"2024-03-14T09:03:12.789385+0000","last_peered":"2024-03-14T09:03:12.789385+0000","last_clean":"2024-03-14T09:03:12.789385+0000","last_became_active":"2024-03-14T09:03:12.786552+0000","last_became_peered":"2024-03-14T09:03:12.786552+0000","last_unstale":"2024-03-14T09:03:12.789385+0000","last_undegraded":"2024-03-14T09:03:12.789385+0000","last_fullsized":"2024-03-14T09:03:12.789385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:37:30.711918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.779523+0000","last_change":"2024-03-14T09:03:12.368944+0000","last_active":"2024-03-14T09:03:12.779523+0000","last_peered":"2024-03-14T09:03:12.779523+0000","last_clean":"2024-03-14T09:03:12.779523+0000","last_became_active":"2024-03-14T09:03:12.368631+0000","last_became_peered":"2024-03-14T09:03:12.368631+0000","last_unstale":"2024-03-14T09:03:12.779523+0000","last_undegraded":"2024-03-14T09:03:12.779523+0000","last_fullsized":"2024-03-14T09:03:12.779523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785326+0000","last_change":"2024-03-14T09:02:59.999593+0000","last_active":"2024-03-14T09:03:12.785326+0000","last_peered":"2024-03-14T09:03:12.785326+0000","last_clean":"2024-03-14T09:03:12.785326+0000","last_became_active":"2024-03-14T09:02:59.999284+0000","last_became_peered":"2024-03-14T09:02:59.999284+0000","last_unstale":"2024-03-14T09:03:12.785326+0000","last_undegraded":"2024-03-14T09:03:12.785326+0000","last_fullsized":"2024-03-14T09:03:12.785326+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:20.597873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773980+0000","last_change":"2024-03-14T09:02:59.579869+0000","last_active":"2024-03-14T09:03:12.773980+0000","last_peered":"2024-03-14T09:03:12.773980+0000","last_clean":"2024-03-14T09:03:12.773980+0000","last_became_active":"2024-03-14T09:02:59.579512+0000","last_became_peered":"2024-03-14T09:02:59.579512+0000","last_unstale":"2024-03-14T09:03:12.773980+0000","last_undegraded":"2024-03-14T09:03:12.773980+0000","last_fullsized":"2024-03-14T09:03:12.773980+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:38:14.987601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.799873+0000","last_change":"2024-03-14T09:03:12.797642+0000","last_active":"2024-03-14T09:03:12.799873+0000","last_peered":"2024-03-14T09:03:12.799873+0000","last_clean":"2024-03-14T09:03:12.799873+0000","last_became_active":"2024-03-14T09:03:12.797092+0000","last_became_peered":"2024-03-14T09:03:12.797092+0000","last_unstale":"2024-03-14T09:03:12.799873+0000","last_undegraded":"2024-03-14T09:03:12.799873+0000","last_fullsized":"2024-03-14T09:03:12.799873+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:45.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778808+0000","last_change":"2024-03-14T09:03:12.370316+0000","last_active":"2024-03-14T09:03:12.778808+0000","last_peered":"2024-03-14T09:03:12.778808+0000","last_clean":"2024-03-14T09:03:12.778808+0000","last_became_active":"2024-03-14T09:03:12.370035+0000","last_became_peered":"2024-03-14T09:03:12.370035+0000","last_unstale":"2024-03-14T09:03:12.778808+0000","last_undegraded":"2024-03-14T09:03:12.778808+0000","last_fullsized":"2024-03-14T09:03:12.778808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:14:07.312244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780309+0000","last_change":"2024-03-14T09:03:00.006109+0000","last_active":"2024-03-14T09:03:12.780309+0000","last_peered":"2024-03-14T09:03:12.780309+0000","last_clean":"2024-03-14T09:03:12.780309+0000","last_became_active":"2024-03-14T09:03:00.005804+0000","last_became_peered":"2024-03-14T09:03:00.005804+0000","last_unstale":"2024-03-14T09:03:12.780309+0000","last_undegraded":"2024-03-14T09:03:12.780309+0000","last_fullsized":"2024-03-14T09:03:12.780309+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:44.977696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.779682+0000","last_change":"2024-03-14T09:02:59.581160+0000","last_active":"2024-03-14T09:03:12.779682+0000","last_peered":"2024-03-14T09:03:12.779682+0000","last_clean":"2024-03-14T09:03:12.779682+0000","last_became_active":"2024-03-14T09:02:59.580896+0000","last_became_peered":"2024-03-14T09:02:59.580896+0000","last_unstale":"2024-03-14T09:03:12.779682+0000","last_undegraded":"2024-03-14T09:03:12.779682+0000","last_fullsized":"2024-03-14T09:03:12.779682+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792670+0000","last_change":"2024-03-14T09:03:12.784850+0000","last_active":"2024-03-14T09:03:12.792670+0000","last_peered":"2024-03-14T09:03:12.792670+0000","last_clean":"2024-03-14T09:03:12.792670+0000","last_became_active":"2024-03-14T09:03:12.784440+0000","last_became_peered":"2024-03-14T09:03:12.784440+0000","last_unstale":"2024-03-14T09:03:12.792670+0000","last_undegraded":"2024-03-14T09:03:12.792670+0000","last_fullsized":"2024-03-14T09:03:12.792670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:44:52.116707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.794358+0000","last_change":"2024-03-14T09:03:12.373196+0000","last_active":"2024-03-14T09:03:12.794358+0000","last_peered":"2024-03-14T09:03:12.794358+0000","last_clean":"2024-03-14T09:03:12.794358+0000","last_became_active":"2024-03-14T09:03:12.372917+0000","last_became_peered":"2024-03-14T09:03:12.372917+0000","last_unstale":"2024-03-14T09:03:12.794358+0000","last_undegraded":"2024-03-14T09:03:12.794358+0000","last_fullsized":"2024-03-14T09:03:12.794358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:28:56.766593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785862+0000","last_change":"2024-03-14T09:03:00.004752+0000","last_active":"2024-03-14T09:03:12.785862+0000","last_peered":"2024-03-14T09:03:12.785862+0000","last_clean":"2024-03-14T09:03:12.785862+0000","last_became_active":"2024-03-14T09:03:00.004477+0000","last_became_peered":"2024-03-14T09:03:00.004477+0000","last_unstale":"2024-03-14T09:03:12.785862+0000","last_undegraded":"2024-03-14T09:03:12.785862+0000","last_fullsized":"2024-03-14T09:03:12.785862+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:59:56.775866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774837+0000","last_change":"2024-03-14T09:02:59.578612+0000","last_active":"2024-03-14T09:03:12.774837+0000","last_peered":"2024-03-14T09:03:12.774837+0000","last_clean":"2024-03-14T09:03:12.774837+0000","last_became_active":"2024-03-14T09:02:59.578337+0000","last_became_peered":"2024-03-14T09:02:59.578337+0000","last_unstale":"2024-03-14T09:03:12.774837+0000","last_undegraded":"2024-03-14T09:03:12.774837+0000","last_fullsized":"2024-03-14T09:03:12.774837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:00.264774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798410+0000","last_change":"2024-03-14T09:03:12.793610+0000","last_active":"2024-03-14T09:03:12.798410+0000","last_peered":"2024-03-14T09:03:12.798410+0000","last_clean":"2024-03-14T09:03:12.798410+0000","last_became_active":"2024-03-14T09:03:12.793347+0000","last_became_peered":"2024-03-14T09:03:12.793347+0000","last_unstale":"2024-03-14T09:03:12.798410+0000","last_undegraded":"2024-03-14T09:03:12.798410+0000","last_fullsized":"2024-03-14T09:03:12.798410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:22.948318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785301+0000","last_change":"2024-03-14T09:03:12.373227+0000","last_active":"2024-03-14T09:03:12.785301+0000","last_peered":"2024-03-14T09:03:12.785301+0000","last_clean":"2024-03-14T09:03:12.785301+0000","last_became_active":"2024-03-14T09:03:12.372761+0000","last_became_peered":"2024-03-14T09:03:12.372761+0000","last_unstale":"2024-03-14T09:03:12.785301+0000","last_undegraded":"2024-03-14T09:03:12.785301+0000","last_fullsized":"2024-03-14T09:03:12.785301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773925+0000","last_change":"2024-03-14T09:03:00.001202+0000","last_active":"2024-03-14T09:03:12.773925+0000","last_peered":"2024-03-14T09:03:12.773925+0000","last_clean":"2024-03-14T09:03:12.773925+0000","last_became_active":"2024-03-14T09:03:00.000900+0000","last_became_peered":"2024-03-14T09:03:00.000900+0000","last_unstale":"2024-03-14T09:03:12.773925+0000","last_undegraded":"2024-03-14T09:03:12.773925+0000","last_fullsized":"2024-03-14T09:03:12.773925+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:18:31.987629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764885+0000","last_change":"2024-03-14T09:02:59.586039+0000","last_active":"2024-03-14T09:03:12.764885+0000","last_peered":"2024-03-14T09:03:12.764885+0000","last_clean":"2024-03-14T09:03:12.764885+0000","last_became_active":"2024-03-14T09:02:59.585737+0000","last_became_peered":"2024-03-14T09:02:59.585737+0000","last_unstale":"2024-03-14T09:03:12.764885+0000","last_undegraded":"2024-03-14T09:03:12.764885+0000","last_fullsized":"2024-03-14T09:03:12.764885+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:27:42.217349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802667+0000","last_change":"2024-03-14T09:03:12.793968+0000","last_active":"2024-03-14T09:03:12.802667+0000","last_peered":"2024-03-14T09:03:12.802667+0000","last_clean":"2024-03-14T09:03:12.802667+0000","last_became_active":"2024-03-14T09:03:12.793123+0000","last_became_peered":"2024-03-14T09:03:12.793123+0000","last_unstale":"2024-03-14T09:03:12.802667+0000","last_undegraded":"2024-03-14T09:03:12.802667+0000","last_fullsized":"2024-03-14T09:03:12.802667+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:51.388714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.793769+0000","last_change":"2024-03-14T09:03:12.365684+0000","last_active":"2024-03-14T09:03:12.793769+0000","last_peered":"2024-03-14T09:03:12.793769+0000","last_clean":"2024-03-14T09:03:12.793769+0000","last_became_active":"2024-03-14T09:03:12.365263+0000","last_became_peered":"2024-03-14T09:03:12.365263+0000","last_unstale":"2024-03-14T09:03:12.793769+0000","last_undegraded":"2024-03-14T09:03:12.793769+0000","last_fullsized":"2024-03-14T09:03:12.793769+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:45.546849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773355+0000","last_change":"2024-03-14T09:02:59.988479+0000","last_active":"2024-03-14T09:03:12.773355+0000","last_peered":"2024-03-14T09:03:12.773355+0000","last_clean":"2024-03-14T09:03:12.773355+0000","last_became_active":"2024-03-14T09:02:59.988232+0000","last_became_peered":"2024-03-14T09:02:59.988232+0000","last_unstale":"2024-03-14T09:03:12.773355+0000","last_undegraded":"2024-03-14T09:03:12.773355+0000","last_fullsized":"2024-03-14T09:03:12.773355+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:17:42.007701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773632+0000","last_change":"2024-03-14T09:02:59.577396+0000","last_active":"2024-03-14T09:03:12.773632+0000","last_peered":"2024-03-14T09:03:12.773632+0000","last_clean":"2024-03-14T09:03:12.773632+0000","last_became_active":"2024-03-14T09:02:59.577072+0000","last_became_peered":"2024-03-14T09:02:59.577072+0000","last_unstale":"2024-03-14T09:03:12.773632+0000","last_undegraded":"2024-03-14T09:03:12.773632+0000","last_fullsized":"2024-03-14T09:03:12.773632+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:44:29.229267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.802620+0000","last_change":"2024-03-14T09:03:12.795240+0000","last_active":"2024-03-14T09:03:12.802620+0000","last_peered":"2024-03-14T09:03:12.802620+0000","last_clean":"2024-03-14T09:03:12.802620+0000","last_became_active":"2024-03-14T09:03:12.793477+0000","last_became_peered":"2024-03-14T09:03:12.793477+0000","last_unstale":"2024-03-14T09:03:12.802620+0000","last_undegraded":"2024-03-14T09:03:12.802620+0000","last_fullsized":"2024-03-14T09:03:12.802620+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:23:41.433146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.797232+0000","last_change":"2024-03-14T09:03:12.371367+0000","last_active":"2024-03-14T09:03:12.797232+0000","last_peered":"2024-03-14T09:03:12.797232+0000","last_clean":"2024-03-14T09:03:12.797232+0000","last_became_active":"2024-03-14T09:03:12.371044+0000","last_became_peered":"2024-03-14T09:03:12.371044+0000","last_unstale":"2024-03-14T09:03:12.797232+0000","last_undegraded":"2024-03-14T09:03:12.797232+0000","last_fullsized":"2024-03-14T09:03:12.797232+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:20:22.371301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.785526+0000","last_change":"2024-03-14T09:03:00.004176+0000","last_active":"2024-03-14T09:03:12.785526+0000","last_peered":"2024-03-14T09:03:12.785526+0000","last_clean":"2024-03-14T09:03:12.785526+0000","last_became_active":"2024-03-14T09:03:00.003837+0000","last_became_peered":"2024-03-14T09:03:00.003837+0000","last_unstale":"2024-03-14T09:03:12.785526+0000","last_undegraded":"2024-03-14T09:03:12.785526+0000","last_fullsized":"2024-03-14T09:03:12.785526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.765144+0000","last_change":"2024-03-14T09:02:59.585576+0000","last_active":"2024-03-14T09:03:12.765144+0000","last_peered":"2024-03-14T09:03:12.765144+0000","last_clean":"2024-03-14T09:03:12.765144+0000","last_became_active":"2024-03-14T09:02:59.585291+0000","last_became_peered":"2024-03-14T09:02:59.585291+0000","last_unstale":"2024-03-14T09:03:12.765144+0000","last_undegraded":"2024-03-14T09:03:12.765144+0000","last_fullsized":"2024-03-14T09:03:12.765144+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:37:52.174988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798350+0000","last_change":"2024-03-14T09:03:12.793793+0000","last_active":"2024-03-14T09:03:12.798350+0000","last_peered":"2024-03-14T09:03:12.798350+0000","last_clean":"2024-03-14T09:03:12.798350+0000","last_became_active":"2024-03-14T09:03:12.793563+0000","last_became_peered":"2024-03-14T09:03:12.793563+0000","last_unstale":"2024-03-14T09:03:12.798350+0000","last_undegraded":"2024-03-14T09:03:12.798350+0000","last_fullsized":"2024-03-14T09:03:12.798350+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:03.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774178+0000","last_change":"2024-03-14T09:03:12.372117+0000","last_active":"2024-03-14T09:03:12.774178+0000","last_peered":"2024-03-14T09:03:12.774178+0000","last_clean":"2024-03-14T09:03:12.774178+0000","last_became_active":"2024-03-14T09:03:12.371774+0000","last_became_peered":"2024-03-14T09:03:12.371774+0000","last_unstale":"2024-03-14T09:03:12.774178+0000","last_undegraded":"2024-03-14T09:03:12.774178+0000","last_fullsized":"2024-03-14T09:03:12.774178+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:04.409001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772907+0000","last_change":"2024-03-14T09:03:00.000726+0000","last_active":"2024-03-14T09:03:12.772907+0000","last_peered":"2024-03-14T09:03:12.772907+0000","last_clean":"2024-03-14T09:03:12.772907+0000","last_became_active":"2024-03-14T09:03:00.000482+0000","last_became_peered":"2024-03-14T09:03:00.000482+0000","last_unstale":"2024-03-14T09:03:12.772907+0000","last_undegraded":"2024-03-14T09:03:12.772907+0000","last_fullsized":"2024-03-14T09:03:12.772907+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:08.936857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774559+0000","last_change":"2024-03-14T09:02:59.579775+0000","last_active":"2024-03-14T09:03:12.774559+0000","last_peered":"2024-03-14T09:03:12.774559+0000","last_clean":"2024-03-14T09:03:12.774559+0000","last_became_active":"2024-03-14T09:02:59.579533+0000","last_became_peered":"2024-03-14T09:02:59.579533+0000","last_unstale":"2024-03-14T09:03:12.774559+0000","last_undegraded":"2024-03-14T09:03:12.774559+0000","last_fullsized":"2024-03-14T09:03:12.774559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:32.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.795329+0000","last_change":"2024-03-14T09:03:12.790037+0000","last_active":"2024-03-14T09:03:12.795329+0000","last_peered":"2024-03-14T09:03:12.795329+0000","last_clean":"2024-03-14T09:03:12.795329+0000","last_became_active":"2024-03-14T09:03:12.789578+0000","last_became_peered":"2024-03-14T09:03:12.789578+0000","last_unstale":"2024-03-14T09:03:12.795329+0000","last_undegraded":"2024-03-14T09:03:12.795329+0000","last_fullsized":"2024-03-14T09:03:12.795329+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:29:35.125664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764554+0000","last_change":"2024-03-14T09:03:12.370993+0000","last_active":"2024-03-14T09:03:12.764554+0000","last_peered":"2024-03-14T09:03:12.764554+0000","last_clean":"2024-03-14T09:03:12.764554+0000","last_became_active":"2024-03-14T09:03:12.370745+0000","last_became_peered":"2024-03-14T09:03:12.370745+0000","last_unstale":"2024-03-14T09:03:12.764554+0000","last_undegraded":"2024-03-14T09:03:12.764554+0000","last_fullsized":"2024-03-14T09:03:12.764554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:50.386980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764425+0000","last_change":"2024-03-14T09:02:59.991096+0000","last_active":"2024-03-14T09:03:12.764425+0000","last_peered":"2024-03-14T09:03:12.764425+0000","last_clean":"2024-03-14T09:03:12.764425+0000","last_became_active":"2024-03-14T09:02:59.990814+0000","last_became_peered":"2024-03-14T09:02:59.990814+0000","last_unstale":"2024-03-14T09:03:12.764425+0000","last_undegraded":"2024-03-14T09:03:12.764425+0000","last_fullsized":"2024-03-14T09:03:12.764425+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:22:29.470118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768416+0000","last_change":"2024-03-14T09:02:59.576489+0000","last_active":"2024-03-14T09:03:12.768416+0000","last_peered":"2024-03-14T09:03:12.768416+0000","last_clean":"2024-03-14T09:03:12.768416+0000","last_became_active":"2024-03-14T09:02:59.576203+0000","last_became_peered":"2024-03-14T09:02:59.576203+0000","last_unstale":"2024-03-14T09:03:12.768416+0000","last_undegraded":"2024-03-14T09:03:12.768416+0000","last_fullsized":"2024-03-14T09:03:12.768416+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782266+0000","last_change":"2024-03-14T09:03:12.778776+0000","last_active":"2024-03-14T09:03:12.782266+0000","last_peered":"2024-03-14T09:03:12.782266+0000","last_clean":"2024-03-14T09:03:12.782266+0000","last_became_active":"2024-03-14T09:03:12.778512+0000","last_became_peered":"2024-03-14T09:03:12.778512+0000","last_unstale":"2024-03-14T09:03:12.782266+0000","last_undegraded":"2024-03-14T09:03:12.782266+0000","last_fullsized":"2024-03-14T09:03:12.782266+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:20:46.234924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778917+0000","last_change":"2024-03-14T09:03:12.366770+0000","last_active":"2024-03-14T09:03:12.778917+0000","last_peered":"2024-03-14T09:03:12.778917+0000","last_clean":"2024-03-14T09:03:12.778917+0000","last_became_active":"2024-03-14T09:03:12.366022+0000","last_became_peered":"2024-03-14T09:03:12.366022+0000","last_unstale":"2024-03-14T09:03:12.778917+0000","last_undegraded":"2024-03-14T09:03:12.778917+0000","last_fullsized":"2024-03-14T09:03:12.778917+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:12:44.906314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.765207+0000","last_change":"2024-03-14T09:02:59.997230+0000","last_active":"2024-03-14T09:03:12.765207+0000","last_peered":"2024-03-14T09:03:12.765207+0000","last_clean":"2024-03-14T09:03:12.765207+0000","last_became_active":"2024-03-14T09:02:59.996966+0000","last_became_peered":"2024-03-14T09:02:59.996966+0000","last_unstale":"2024-03-14T09:03:12.765207+0000","last_undegraded":"2024-03-14T09:03:12.765207+0000","last_fullsized":"2024-03-14T09:03:12.765207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:55.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772772+0000","last_change":"2024-03-14T09:02:59.586427+0000","last_active":"2024-03-14T09:03:12.772772+0000","last_peered":"2024-03-14T09:03:12.772772+0000","last_clean":"2024-03-14T09:03:12.772772+0000","last_became_active":"2024-03-14T09:02:59.586007+0000","last_became_peered":"2024-03-14T09:02:59.586007+0000","last_unstale":"2024-03-14T09:03:12.772772+0000","last_undegraded":"2024-03-14T09:03:12.772772+0000","last_fullsized":"2024-03-14T09:03:12.772772+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:42.259341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805428+0000","last_change":"2024-03-14T09:03:12.799731+0000","last_active":"2024-03-14T09:03:12.805428+0000","last_peered":"2024-03-14T09:03:12.805428+0000","last_clean":"2024-03-14T09:03:12.805428+0000","last_became_active":"2024-03-14T09:03:12.799447+0000","last_became_peered":"2024-03-14T09:03:12.799447+0000","last_unstale":"2024-03-14T09:03:12.805428+0000","last_undegraded":"2024-03-14T09:03:12.805428+0000","last_fullsized":"2024-03-14T09:03:12.805428+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:26:58.339933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775064+0000","last_change":"2024-03-14T09:03:12.373601+0000","last_active":"2024-03-14T09:03:12.775064+0000","last_peered":"2024-03-14T09:03:12.775064+0000","last_clean":"2024-03-14T09:03:12.775064+0000","last_became_active":"2024-03-14T09:03:12.373332+0000","last_became_peered":"2024-03-14T09:03:12.373332+0000","last_unstale":"2024-03-14T09:03:12.775064+0000","last_undegraded":"2024-03-14T09:03:12.775064+0000","last_fullsized":"2024-03-14T09:03:12.775064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:38:05.459173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.775029+0000","last_change":"2024-03-14T09:02:59.992723+0000","last_active":"2024-03-14T09:03:12.775029+0000","last_peered":"2024-03-14T09:03:12.775029+0000","last_clean":"2024-03-14T09:03:12.775029+0000","last_became_active":"2024-03-14T09:02:59.992406+0000","last_became_peered":"2024-03-14T09:02:59.992406+0000","last_unstale":"2024-03-14T09:03:12.775029+0000","last_undegraded":"2024-03-14T09:03:12.775029+0000","last_fullsized":"2024-03-14T09:03:12.775029+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768561+0000","last_change":"2024-03-14T09:02:59.586304+0000","last_active":"2024-03-14T09:03:12.768561+0000","last_peered":"2024-03-14T09:03:12.768561+0000","last_clean":"2024-03-14T09:03:12.768561+0000","last_became_active":"2024-03-14T09:02:59.586031+0000","last_became_peered":"2024-03-14T09:02:59.586031+0000","last_unstale":"2024-03-14T09:03:12.768561+0000","last_undegraded":"2024-03-14T09:03:12.768561+0000","last_fullsized":"2024-03-14T09:03:12.768561+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:08.001744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.805437+0000","last_change":"2024-03-14T09:03:12.799089+0000","last_active":"2024-03-14T09:03:12.805437+0000","last_peered":"2024-03-14T09:03:12.805437+0000","last_clean":"2024-03-14T09:03:12.805437+0000","last_became_active":"2024-03-14T09:03:12.798644+0000","last_became_peered":"2024-03-14T09:03:12.798644+0000","last_unstale":"2024-03-14T09:03:12.805437+0000","last_undegraded":"2024-03-14T09:03:12.805437+0000","last_fullsized":"2024-03-14T09:03:12.805437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:49:56.418493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764583+0000","last_change":"2024-03-14T09:03:12.373610+0000","last_active":"2024-03-14T09:03:12.764583+0000","last_peered":"2024-03-14T09:03:12.764583+0000","last_clean":"2024-03-14T09:03:12.764583+0000","last_became_active":"2024-03-14T09:03:12.373360+0000","last_became_peered":"2024-03-14T09:03:12.373360+0000","last_unstale":"2024-03-14T09:03:12.764583+0000","last_undegraded":"2024-03-14T09:03:12.764583+0000","last_fullsized":"2024-03-14T09:03:12.764583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:32.334273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773759+0000","last_change":"2024-03-14T09:02:59.994799+0000","last_active":"2024-03-14T09:03:12.773759+0000","last_peered":"2024-03-14T09:03:12.773759+0000","last_clean":"2024-03-14T09:03:12.773759+0000","last_became_active":"2024-03-14T09:02:59.994442+0000","last_became_peered":"2024-03-14T09:02:59.994442+0000","last_unstale":"2024-03-14T09:03:12.773759+0000","last_undegraded":"2024-03-14T09:03:12.773759+0000","last_fullsized":"2024-03-14T09:03:12.773759+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:35:10.157686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"51'1","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764976+0000","last_change":"2024-03-14T09:02:59.586226+0000","last_active":"2024-03-14T09:03:12.764976+0000","last_peered":"2024-03-14T09:03:12.764976+0000","last_clean":"2024-03-14T09:03:12.764976+0000","last_became_active":"2024-03-14T09:02:59.577632+0000","last_became_peered":"2024-03-14T09:02:59.577632+0000","last_unstale":"2024-03-14T09:03:12.764976+0000","last_undegraded":"2024-03-14T09:03:12.764976+0000","last_fullsized":"2024-03-14T09:03:12.764976+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:46:51.268648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786943+0000","last_change":"2024-03-14T09:03:12.783302+0000","last_active":"2024-03-14T09:03:12.786943+0000","last_peered":"2024-03-14T09:03:12.786943+0000","last_clean":"2024-03-14T09:03:12.786943+0000","last_became_active":"2024-03-14T09:03:12.783052+0000","last_became_peered":"2024-03-14T09:03:12.783052+0000","last_unstale":"2024-03-14T09:03:12.786943+0000","last_undegraded":"2024-03-14T09:03:12.786943+0000","last_fullsized":"2024-03-14T09:03:12.786943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:29:42.217340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792918+0000","last_change":"2024-03-14T09:03:12.365515+0000","last_active":"2024-03-14T09:03:12.792918+0000","last_peered":"2024-03-14T09:03:12.792918+0000","last_clean":"2024-03-14T09:03:12.792918+0000","last_became_active":"2024-03-14T09:03:12.364921+0000","last_became_peered":"2024-03-14T09:03:12.364921+0000","last_unstale":"2024-03-14T09:03:12.792918+0000","last_undegraded":"2024-03-14T09:03:12.792918+0000","last_fullsized":"2024-03-14T09:03:12.792918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.769207+0000","last_change":"2024-03-14T09:02:59.991690+0000","last_active":"2024-03-14T09:03:12.769207+0000","last_peered":"2024-03-14T09:03:12.769207+0000","last_clean":"2024-03-14T09:03:12.769207+0000","last_became_active":"2024-03-14T09:02:59.990981+0000","last_became_peered":"2024-03-14T09:02:59.990981+0000","last_unstale":"2024-03-14T09:03:12.769207+0000","last_undegraded":"2024-03-14T09:03:12.769207+0000","last_fullsized":"2024-03-14T09:03:12.769207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:16:58.303964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778887+0000","last_change":"2024-03-14T09:02:59.578600+0000","last_active":"2024-03-14T09:03:12.778887+0000","last_peered":"2024-03-14T09:03:12.778887+0000","last_clean":"2024-03-14T09:03:12.778887+0000","last_became_active":"2024-03-14T09:02:59.578136+0000","last_became_peered":"2024-03-14T09:02:59.578136+0000","last_unstale":"2024-03-14T09:03:12.778887+0000","last_undegraded":"2024-03-14T09:03:12.778887+0000","last_fullsized":"2024-03-14T09:03:12.778887+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.792707+0000","last_change":"2024-03-14T09:03:12.783680+0000","last_active":"2024-03-14T09:03:12.792707+0000","last_peered":"2024-03-14T09:03:12.792707+0000","last_clean":"2024-03-14T09:03:12.792707+0000","last_became_active":"2024-03-14T09:03:12.783420+0000","last_became_peered":"2024-03-14T09:03:12.783420+0000","last_unstale":"2024-03-14T09:03:12.792707+0000","last_undegraded":"2024-03-14T09:03:12.792707+0000","last_fullsized":"2024-03-14T09:03:12.792707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:04:25.889118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-14T09:03:13.777870+0000","last_change":"2024-03-14T09:03:12.367975+0000","last_active":"2024-03-14T09:03:13.777870+0000","last_peered":"2024-03-14T09:03:13.777870+0000","last_clean":"2024-03-14T09:03:13.777870+0000","last_became_active":"2024-03-14T09:03:12.367702+0000","last_became_peered":"2024-03-14T09:03:12.367702+0000","last_unstale":"2024-03-14T09:03:13.777870+0000","last_undegraded":"2024-03-14T09:03:13.777870+0000","last_fullsized":"2024-03-14T09:03:13.777870+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:12.652757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.764747+0000","last_change":"2024-03-14T09:02:59.992900+0000","last_active":"2024-03-14T09:03:12.764747+0000","last_peered":"2024-03-14T09:03:12.764747+0000","last_clean":"2024-03-14T09:03:12.764747+0000","last_became_active":"2024-03-14T09:02:59.992337+0000","last_became_peered":"2024-03-14T09:02:59.992337+0000","last_unstale":"2024-03-14T09:03:12.764747+0000","last_undegraded":"2024-03-14T09:03:12.764747+0000","last_fullsized":"2024-03-14T09:03:12.764747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:13:57.824561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773810+0000","last_change":"2024-03-14T09:02:59.570173+0000","last_active":"2024-03-14T09:03:12.773810+0000","last_peered":"2024-03-14T09:03:12.773810+0000","last_clean":"2024-03-14T09:03:12.773810+0000","last_became_active":"2024-03-14T09:02:59.569794+0000","last_became_peered":"2024-03-14T09:02:59.569794+0000","last_unstale":"2024-03-14T09:03:12.773810+0000","last_undegraded":"2024-03-14T09:03:12.773810+0000","last_fullsized":"2024-03-14T09:03:12.773810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:37:52.665244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.789642+0000","last_change":"2024-03-14T09:03:12.779791+0000","last_active":"2024-03-14T09:03:12.789642+0000","last_peered":"2024-03-14T09:03:12.789642+0000","last_clean":"2024-03-14T09:03:12.789642+0000","last_became_active":"2024-03-14T09:03:12.779533+0000","last_became_peered":"2024-03-14T09:03:12.779533+0000","last_unstale":"2024-03-14T09:03:12.789642+0000","last_undegraded":"2024-03-14T09:03:12.789642+0000","last_fullsized":"2024-03-14T09:03:12.789642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:35.680576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.783366+0000","last_change":"2024-03-14T09:03:12.371984+0000","last_active":"2024-03-14T09:03:12.783366+0000","last_peered":"2024-03-14T09:03:12.783366+0000","last_clean":"2024-03-14T09:03:12.783366+0000","last_became_active":"2024-03-14T09:03:12.371727+0000","last_became_peered":"2024-03-14T09:03:12.371727+0000","last_unstale":"2024-03-14T09:03:12.783366+0000","last_undegraded":"2024-03-14T09:03:12.783366+0000","last_fullsized":"2024-03-14T09:03:12.783366+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:13:23.845822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.774572+0000","last_change":"2024-03-14T09:02:59.989615+0000","last_active":"2024-03-14T09:03:12.774572+0000","last_peered":"2024-03-14T09:03:12.774572+0000","last_clean":"2024-03-14T09:03:12.774572+0000","last_became_active":"2024-03-14T09:02:59.988716+0000","last_became_peered":"2024-03-14T09:02:59.988716+0000","last_unstale":"2024-03-14T09:03:12.774572+0000","last_undegraded":"2024-03-14T09:03:12.774572+0000","last_fullsized":"2024-03-14T09:03:12.774572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:38.598082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.768224+0000","last_change":"2024-03-14T09:02:59.571144+0000","last_active":"2024-03-14T09:03:12.768224+0000","last_peered":"2024-03-14T09:03:12.768224+0000","last_clean":"2024-03-14T09:03:12.768224+0000","last_became_active":"2024-03-14T09:02:59.570856+0000","last_became_peered":"2024-03-14T09:02:59.570856+0000","last_unstale":"2024-03-14T09:03:12.768224+0000","last_undegraded":"2024-03-14T09:03:12.768224+0000","last_fullsized":"2024-03-14T09:03:12.768224+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:55:37.946335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798185+0000","last_change":"2024-03-14T09:03:12.790881+0000","last_active":"2024-03-14T09:03:12.798185+0000","last_peered":"2024-03-14T09:03:12.798185+0000","last_clean":"2024-03-14T09:03:12.798185+0000","last_became_active":"2024-03-14T09:03:12.790583+0000","last_became_peered":"2024-03-14T09:03:12.790583+0000","last_unstale":"2024-03-14T09:03:12.798185+0000","last_undegraded":"2024-03-14T09:03:12.798185+0000","last_fullsized":"2024-03-14T09:03:12.798185+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:46:05.410736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.778644+0000","last_change":"2024-03-14T09:03:12.363729+0000","last_active":"2024-03-14T09:03:12.778644+0000","last_peered":"2024-03-14T09:03:12.778644+0000","last_clean":"2024-03-14T09:03:12.778644+0000","last_became_active":"2024-03-14T09:03:12.363440+0000","last_became_peered":"2024-03-14T09:03:12.363440+0000","last_unstale":"2024-03-14T09:03:12.778644+0000","last_undegraded":"2024-03-14T09:03:12.778644+0000","last_fullsized":"2024-03-14T09:03:12.778644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:37:13.058703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.780594+0000","last_change":"2024-03-14T09:03:00.006119+0000","last_active":"2024-03-14T09:03:12.780594+0000","last_peered":"2024-03-14T09:03:12.780594+0000","last_clean":"2024-03-14T09:03:12.780594+0000","last_became_active":"2024-03-14T09:03:00.005793+0000","last_became_peered":"2024-03-14T09:03:00.005793+0000","last_unstale":"2024-03-14T09:03:12.780594+0000","last_undegraded":"2024-03-14T09:03:12.780594+0000","last_fullsized":"2024-03-14T09:03:12.780594+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:09:22.889945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.782381+0000","last_change":"2024-03-14T09:02:59.577902+0000","last_active":"2024-03-14T09:03:12.782381+0000","last_peered":"2024-03-14T09:03:12.782381+0000","last_clean":"2024-03-14T09:03:12.782381+0000","last_became_active":"2024-03-14T09:02:59.577568+0000","last_became_peered":"2024-03-14T09:02:59.577568+0000","last_unstale":"2024-03-14T09:03:12.782381+0000","last_undegraded":"2024-03-14T09:03:12.782381+0000","last_fullsized":"2024-03-14T09:03:12.782381+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:42:53.569650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.786840+0000","last_change":"2024-03-14T09:03:12.781504+0000","last_active":"2024-03-14T09:03:12.786840+0000","last_peered":"2024-03-14T09:03:12.786840+0000","last_clean":"2024-03-14T09:03:12.786840+0000","last_became_active":"2024-03-14T09:03:12.781231+0000","last_became_peered":"2024-03-14T09:03:12.781231+0000","last_unstale":"2024-03-14T09:03:12.786840+0000","last_undegraded":"2024-03-14T09:03:12.786840+0000","last_fullsized":"2024-03-14T09:03:12.786840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:29:21.727488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.769661+0000","last_change":"2024-03-14T09:03:12.370434+0000","last_active":"2024-03-14T09:03:12.769661+0000","last_peered":"2024-03-14T09:03:12.769661+0000","last_clean":"2024-03-14T09:03:12.769661+0000","last_became_active":"2024-03-14T09:03:12.370170+0000","last_became_peered":"2024-03-14T09:03:12.370170+0000","last_unstale":"2024-03-14T09:03:12.769661+0000","last_undegraded":"2024-03-14T09:03:12.769661+0000","last_fullsized":"2024-03-14T09:03:12.769661+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:54:55.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.767440+0000","last_change":"2024-03-14T09:02:59.987865+0000","last_active":"2024-03-14T09:03:12.767440+0000","last_peered":"2024-03-14T09:03:12.767440+0000","last_clean":"2024-03-14T09:03:12.767440+0000","last_became_active":"2024-03-14T09:02:59.987627+0000","last_became_peered":"2024-03-14T09:02:59.987627+0000","last_unstale":"2024-03-14T09:03:12.767440+0000","last_undegraded":"2024-03-14T09:03:12.767440+0000","last_fullsized":"2024-03-14T09:03:12.767440+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:27.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.767751+0000","last_change":"2024-03-14T09:02:59.577305+0000","last_active":"2024-03-14T09:03:12.767751+0000","last_peered":"2024-03-14T09:03:12.767751+0000","last_clean":"2024-03-14T09:03:12.767751+0000","last_became_active":"2024-03-14T09:02:59.577047+0000","last_became_peered":"2024-03-14T09:02:59.577047+0000","last_unstale":"2024-03-14T09:03:12.767751+0000","last_undegraded":"2024-03-14T09:03:12.767751+0000","last_fullsized":"2024-03-14T09:03:12.767751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:16:57.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.796777+0000","last_change":"2024-03-14T09:03:12.788684+0000","last_active":"2024-03-14T09:03:12.796777+0000","last_peered":"2024-03-14T09:03:12.796777+0000","last_clean":"2024-03-14T09:03:12.796777+0000","last_became_active":"2024-03-14T09:03:12.788407+0000","last_became_peered":"2024-03-14T09:03:12.788407+0000","last_unstale":"2024-03-14T09:03:12.796777+0000","last_undegraded":"2024-03-14T09:03:12.796777+0000","last_fullsized":"2024-03-14T09:03:12.796777+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:14:44.881243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.767879+0000","last_change":"2024-03-14T09:03:12.373132+0000","last_active":"2024-03-14T09:03:12.767879+0000","last_peered":"2024-03-14T09:03:12.767879+0000","last_clean":"2024-03-14T09:03:12.767879+0000","last_became_active":"2024-03-14T09:03:12.372802+0000","last_became_peered":"2024-03-14T09:03:12.372802+0000","last_unstale":"2024-03-14T09:03:12.767879+0000","last_undegraded":"2024-03-14T09:03:12.767879+0000","last_fullsized":"2024-03-14T09:03:12.767879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:00:08.564763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.773969+0000","last_change":"2024-03-14T09:02:59.989109+0000","last_active":"2024-03-14T09:03:12.773969+0000","last_peered":"2024-03-14T09:03:12.773969+0000","last_clean":"2024-03-14T09:03:12.773969+0000","last_became_active":"2024-03-14T09:02:59.988861+0000","last_became_peered":"2024-03-14T09:02:59.988861+0000","last_unstale":"2024-03-14T09:03:12.773969+0000","last_undegraded":"2024-03-14T09:03:12.773969+0000","last_fullsized":"2024-03-14T09:03:12.773969+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:43:51.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772208+0000","last_change":"2024-03-14T09:02:59.580838+0000","last_active":"2024-03-14T09:03:12.772208+0000","last_peered":"2024-03-14T09:03:12.772208+0000","last_clean":"2024-03-14T09:03:12.772208+0000","last_became_active":"2024-03-14T09:02:59.580570+0000","last_became_peered":"2024-03-14T09:02:59.580570+0000","last_unstale":"2024-03-14T09:03:12.772208+0000","last_undegraded":"2024-03-14T09:03:12.772208+0000","last_fullsized":"2024-03-14T09:03:12.772208+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:54:32.752920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.798323+0000","last_change":"2024-03-14T09:03:12.793571+0000","last_active":"2024-03-14T09:03:12.798323+0000","last_peered":"2024-03-14T09:03:12.798323+0000","last_clean":"2024-03-14T09:03:12.798323+0000","last_became_active":"2024-03-14T09:03:12.793335+0000","last_became_peered":"2024-03-14T09:03:12.793335+0000","last_unstale":"2024-03-14T09:03:12.798323+0000","last_undegraded":"2024-03-14T09:03:12.798323+0000","last_fullsized":"2024-03-14T09:03:12.798323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:12.347114+0000","last_clean_scrub_stamp":"2024-03-14T09:03:12.347114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:25:56.026382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"54'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.793170+0000","last_change":"2024-03-14T09:03:12.364039+0000","last_active":"2024-03-14T09:03:12.793170+0000","last_peered":"2024-03-14T09:03:12.793170+0000","last_clean":"2024-03-14T09:03:12.793170+0000","last_became_active":"2024-03-14T09:03:12.363778+0000","last_became_peered":"2024-03-14T09:03:12.363778+0000","last_unstale":"2024-03-14T09:03:12.793170+0000","last_undegraded":"2024-03-14T09:03:12.793170+0000","last_fullsized":"2024-03-14T09:03:12.793170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:11.345380+0000","last_clean_scrub_stamp":"2024-03-14T09:03:11.345380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:53.168005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.772392+0000","last_change":"2024-03-14T09:02:59.993748+0000","last_active":"2024-03-14T09:03:12.772392+0000","last_peered":"2024-03-14T09:03:12.772392+0000","last_clean":"2024-03-14T09:03:12.772392+0000","last_became_active":"2024-03-14T09:02:59.993511+0000","last_became_peered":"2024-03-14T09:02:59.993511+0000","last_unstale":"2024-03-14T09:03:12.772392+0000","last_undegraded":"2024-03-14T09:03:12.772392+0000","last_fullsized":"2024-03-14T09:03:12.772392+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:59.556483+0000","last_clean_scrub_stamp":"2024-03-14T09:02:59.556483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:34:31.054784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-14T09:03:12.779043+0000","last_change":"2024-03-14T09:02:59.579213+0000","last_active":"2024-03-14T09:03:12.779043+0000","last_peered":"2024-03-14T09:03:12.779043+0000","last_clean":"2024-03-14T09:03:12.779043+0000","last_became_active":"2024-03-14T09:02:59.578948+0000","last_became_peered":"2024-03-14T09:02:59.578948+0000","last_unstale":"2024-03-14T09:03:12.779043+0000","last_undegraded":"2024-03-14T09:03:12.779043+0000","last_fullsized":"2024-03-14T09:03:12.779043+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:02:58.551451+0000","last_clean_scrub_stamp":"2024-03-14T09:02:58.551451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:25:58.014714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-14T09:03:13.776632+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:13.776632+0000","last_peered":"2024-03-14T09:03:13.776632+0000","last_clean":"2024-03-14T09:03:13.776632+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:13.776632+0000","last_undegraded":"2024-03-14T09:03:13.776632+0000","last_fullsized":"2024-03-14T09:03:13.776632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":877129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.85699999999999998,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":6.5,"15min":6.5},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53100000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.436},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68100000000000005,"5min":0.878,"15min":0.878},"last":0.625}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":286077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64300000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Mar 14 09:02:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":288255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33900000000000002,"15min":0.34000000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.45700000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39400000000000002,"15min":0.39500000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.497,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":880010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.71599999999999997,"15min":0.78200000000000003},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.41799999999999998,"5min":7.383,"15min":7.383},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.79200000000000004,"15min":0.86599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51800000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.81699999999999995,"15min":0.89200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.48899999999999999,"5min":7.944,"15min":7.944},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.313},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.77200000000000002,"15min":0.83599999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0349999999999999,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.98399999999999999,"15min":1.0489999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0509999999999999,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80300000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":287605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34200000000000003,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.40400000000000003,"5min":1.111,"15min":1.111},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.246},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":2.5819999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.67300000000000004,"15min":0.72999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.41199999999999998,"5min":7.5,"15min":7.5},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.71599999999999997,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.48699999999999999,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":1.151,"15min":1.151},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.433},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":1.179,"15min":1.179},"last":0.36299999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71799999999999997,"5min":0.751,"15min":0.751},"last":0.38800000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.75900000000000001,"5min":1.165,"15min":1.165},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64900000000000002,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83199999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.41599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":286153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.309},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.628}]},{"osd":5,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:02:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":289130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51100000000000001,"15min":0.51600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.68999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.751,"15min":0.751},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.89100000000000001,"15min":0.94699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76000000000000001,"5min":7.702,"15min":7.702},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45300000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.753,"15min":0.753},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":286727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67900000000000005,"5min":0.876,"15min":0.876},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.436}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.377},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47899999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:03:21.872 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:03:21.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-03-14T09:03:22.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.034+0000 7f802f508700 1 Processor -- start 2024-03-14T09:03:22.034 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.034+0000 7f802f508700 1 -- start start 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802f508700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f80280c3250 0x7f802813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802f508700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802813ade0 con 0x7f80280c1780 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802f508700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802806fb30 con 0x7f80280a8540 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802f508700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802806ec30 con 0x7f80280c3250 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802caa3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f80280c3250 0x7f802813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802daa5700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f80280c1780 0x7f80280c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36258/0 (socket says 172.21.15.154:36258) 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802daa5700 1 -- 172.21.15.154:0/2390574294 learned_addr learned my addr 172.21.15.154:0/2390574294 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:22.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802caa3700 1 -- 172.21.15.154:0/2390574294 >> v1:172.21.15.191:6789/0 conn(0x7f80280a8540 legacy=0x7f8028154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802caa3700 1 -- 172.21.15.154:0/2390574294 >> v1:172.21.15.154:6789/0 conn(0x7f80280c1780 legacy=0x7f80280c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802caa3700 1 -- 172.21.15.154:0/2390574294 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802806ef10 con 0x7f80280c3250 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f801ffff700 1 -- 172.21.15.154:0/2390574294 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446612103 0 0) 0x7f8020001490 con 0x7f80280c1780 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.035+0000 7f802caa3700 1 --2- 172.21.15.154:0/2390574294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280c3250 0x7f802813a8a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8018009840 tx=0x7f8018009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfab30040141fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f801ffff700 1 -- 172.21.15.154:0/2390574294 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018011830 con 0x7f80280c3250 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f801ffff700 1 -- 172.21.15.154:0/2390574294 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8018011990 con 0x7f80280c3250 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f801ffff700 1 -- 172.21.15.154:0/2390574294 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018011d20 con 0x7f80280c3250 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 -- 172.21.15.154:0/2390574294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280c3250 msgr2=0x7f802813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 --2- 172.21.15.154:0/2390574294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280c3250 0x7f802813a8a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8018009840 tx=0x7f8018009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 -- 172.21.15.154:0/2390574294 shutdown_connections 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 --2- 172.21.15.154:0/2390574294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280c3250 0x7f802813a8a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 -- 172.21.15.154:0/2390574294 >> 172.21.15.154:0/2390574294 conn(0x7f80280c3880 msgr2=0x7f80280c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 -- 172.21.15.154:0/2390574294 shutdown_connections 2024-03-14T09:03:22.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.036+0000 7f802f508700 1 -- 172.21.15.154:0/2390574294 wait complete. 2024-03-14T09:03:22.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.037+0000 7f802f508700 1 Processor -- start 2024-03-14T09:03:22.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.037+0000 7f802f508700 1 -- start start 2024-03-14T09:03:22.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80280c1780 0x7f802814e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 0x7f802814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281ecad0 con 0x7f80280a8540 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281e87d0 con 0x7f80280c3250 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802f508700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80281e8ab0 con 0x7f80280c1780 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35334/0 (socket says 172.21.15.154:35334) 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 -- 172.21.15.154:0/2968290827 learned_addr learned my addr 172.21.15.154:0/2968290827 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802caa3700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80280c1780 0x7f802814e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802daa5700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 0x7f802814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 -- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80280c1780 msgr2=0x7f802814e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80280c1780 0x7f802814e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 -- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 msgr2=0x7f802814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 0x7f802814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802d2a4700 1 -- 172.21.15.154:0/2968290827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80180094a0 con 0x7f80280a8540 2024-03-14T09:03:22.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.038+0000 7f802daa5700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 0x7f802814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f802d2a4700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f802400c450 tx=0x7f802400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240052d0 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024005430 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240038c0 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80281e8d90 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.039+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8028139f90 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.040+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80280697d0 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.040+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8024003a20 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.040+0000 7f801dffb700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 0x7f800c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.041+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f80240884a0 con 0x7f80280a8540 2024-03-14T09:03:22.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.041+0000 7f802caa3700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 0x7f800c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.041+0000 7f802caa3700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 0x7f800c06ccf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f801801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.045+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8024053560 con 0x7f80280a8540 2024-03-14T09:03:22.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.322+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f802806d420 con 0x7f80280a8540 2024-03-14T09:03:22.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.330+0000 7f801dffb700 1 -- 172.21.15.154:0/2968290827 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v20) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8024056b80 con 0x7f80280a8540 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 msgr2=0x7f800c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 0x7f800c06ccf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8018000c00 tx=0x7f801801c040 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 msgr2=0x7f8028139a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f802400c450 tx=0x7f802400c760 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 shutdown_connections 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80280c3250 0x7f802814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f800c06a830 0x7f800c06ccf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80280a8540 0x7f8028139a50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 --2- 172.21.15.154:0/2968290827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80280c1780 0x7f802814e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 >> 172.21.15.154:0/2968290827 conn(0x7f80280c3880 msgr2=0x7f8028071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 shutdown_connections 2024-03-14T09:03:22.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.333+0000 7f802f508700 1 -- 172.21.15.154:0/2968290827 wait complete. 2024-03-14T09:03:22.334 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.1 2024-03-14T09:03:22.345 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:22.345 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:22.366 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:22.366 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:22.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:22.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.549+0000 7f0d88689700 1 Processor -- start 2024-03-14T09:03:22.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.549+0000 7f0d88689700 1 -- start start 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d88689700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0d8014c5c0 0x7f0d80152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d88689700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d80070e10 con 0x7f0d80072710 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d88689700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8014c9a0 con 0x7f0d80072290 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d88689700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c1bb0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d86425700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0d8014c5c0 0x7f0d80152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d86c26700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0d80072710 0x7f0d800c26e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36268/0 (socket says 172.21.15.154:36268) 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.550+0000 7f0d86c26700 1 -- 172.21.15.154:0/3626832235 learned_addr learned my addr 172.21.15.154:0/3626832235 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d86425700 1 -- 172.21.15.154:0/3626832235 >> v1:172.21.15.191:6789/0 conn(0x7f0d80072290 legacy=0x7f0d800c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d86425700 1 -- 172.21.15.154:0/3626832235 >> v1:172.21.15.154:6789/0 conn(0x7f0d80072710 legacy=0x7f0d800c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d86425700 1 -- 172.21.15.154:0/3626832235 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800c1580 con 0x7f0d8014c5c0 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d85423700 1 -- 172.21.15.154:0/3626832235 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935778000 0 0) 0x7f0d80070e10 con 0x7f0d80072710 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d86425700 1 --2- 172.21.15.154:0/3626832235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d80152f80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009870 tx=0x7f0d70009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff65827ac626d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d85423700 1 -- 172.21.15.154:0/3626832235 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70004230 con 0x7f0d8014c5c0 2024-03-14T09:03:22.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d85423700 1 -- 172.21.15.154:0/3626832235 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70004390 con 0x7f0d8014c5c0 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d85423700 1 -- 172.21.15.154:0/3626832235 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700036a0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 -- 172.21.15.154:0/3626832235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 msgr2=0x7f0d80152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 --2- 172.21.15.154:0/3626832235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d80152f80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009870 tx=0x7f0d70009b80 comp rx=0 tx=0).stop 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 -- 172.21.15.154:0/3626832235 shutdown_connections 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 --2- 172.21.15.154:0/3626832235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d80152f80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 -- 172.21.15.154:0/3626832235 >> 172.21.15.154:0/3626832235 conn(0x7f0d8006a770 msgr2=0x7f0d8006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.551+0000 7f0d88689700 1 -- 172.21.15.154:0/3626832235 shutdown_connections 2024-03-14T09:03:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.552+0000 7f0d88689700 1 -- 172.21.15.154:0/3626832235 wait complete. 2024-03-14T09:03:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 Processor -- start 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 -- start start 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d80072290 0x7f0d800bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d80072710 0x7f0d8006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801e9b80 con 0x7f0d8014c5c0 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8006c9e0 con 0x7f0d80072710 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.553+0000 7f0d88689700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8006ccc0 con 0x7f0d80072290 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35358/0 (socket says 172.21.15.154:35358) 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 -- 172.21.15.154:0/3302644521 learned_addr learned my addr 172.21.15.154:0/3302644521 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86425700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d80072290 0x7f0d800bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d85c24700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d80072710 0x7f0d8006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 -- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d80072290 msgr2=0x7f0d800bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d80072290 0x7f0d800bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 -- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d80072710 msgr2=0x7f0d8006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d80072710 0x7f0d8006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 -- 172.21.15.154:0/3302644521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700094a0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d86c26700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0d78002230 tx=0x7f0d7800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800aac0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800ac40 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8006cfa0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780070d0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.554+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d800c0170 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.555+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8006d4c0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.556+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0d78007250 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.556+0000 7f0d76ffd700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 0x7f0d6406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.556+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0d78088e60 con 0x7f0d8014c5c0 2024-03-14T09:03:22.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.556+0000 7f0d86425700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 0x7f0d6406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.557+0000 7f0d86425700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 0x7f0d6406ccf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d700090b0 tx=0x7f0d70008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.560 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.561+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0d8006d4c0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.758+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d8006d4c0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.760+0000 7f0d76ffd700 1 -- 172.21.15.154:0/3302644521 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7f0d8006d4c0 con 0x7f0d8014c5c0 2024-03-14T09:03:22.760 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:22.760 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 msgr2=0x7f0d6406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 0x7f0d6406ccf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d700090b0 tx=0x7f0d70008030 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 msgr2=0x7f0d8014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0d78002230 tx=0x7f0d7800cf50 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 shutdown_connections 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0d6406a830 0x7f0d6406ccf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d8014c5c0 0x7f0d8014fbd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d80072290 0x7f0d800bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 --2- 172.21.15.154:0/3302644521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d80072710 0x7f0d8006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 >> 172.21.15.154:0/3302644521 conn(0x7f0d8006a770 msgr2=0x7f0d800c1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:22.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 shutdown_connections 2024-03-14T09:03:22.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.762+0000 7f0d88689700 1 -- 172.21.15.154:0/3302644521 wait complete. 2024-03-14T09:03:22.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:57.920434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:58.934033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:03:11.026192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:03:11.722376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:22.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.898+0000 7fca37ff1700 1 Processor -- start 2024-03-14T09:03:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 -- start start 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca300a8540 0x7fca3013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca300c1780 0x7fca300c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3013ade0 con 0x7fca300c3250 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3006fb30 con 0x7fca300a8540 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.899+0000 7fca37ff1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3006ec30 con 0x7fca300c1780 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca300a8540 0x7fca3013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca300a8540 0x7fca3013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35374/0 (socket says 172.21.15.154:35374) 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 -- 172.21.15.154:0/1922342097 learned_addr learned my addr 172.21.15.154:0/1922342097 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca3658e700 1 --2- 172.21.15.154:0/1922342097 >> v2:172.21.15.191:3300/0 conn(0x7fca300c1780 0x7fca300c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 -- 172.21.15.154:0/1922342097 >> v2:172.21.15.191:3300/0 conn(0x7fca300c1780 msgr2=0x7fca300c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 --2- 172.21.15.154:0/1922342097 >> v2:172.21.15.191:3300/0 conn(0x7fca300c1780 0x7fca300c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 -- 172.21.15.154:0/1922342097 >> v1:172.21.15.191:6789/0 conn(0x7fca300c3250 legacy=0x7fca30154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 -- 172.21.15.154:0/1922342097 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3006ef10 con 0x7fca300a8540 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca35d8d700 1 --2- 172.21.15.154:0/1922342097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca3013a8a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fca30072e40 tx=0x7fca2000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5541b401444fb66 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca34d8b700 1 -- 172.21.15.154:0/1922342097 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000dcf0 con 0x7fca300a8540 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca34d8b700 1 -- 172.21.15.154:0/1922342097 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000de50 con 0x7fca300a8540 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.900+0000 7fca34d8b700 1 -- 172.21.15.154:0/1922342097 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20004250 con 0x7fca300a8540 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 -- 172.21.15.154:0/1922342097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 msgr2=0x7fca3013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 --2- 172.21.15.154:0/1922342097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca3013a8a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fca30072e40 tx=0x7fca2000be70 comp rx=0 tx=0).stop 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 -- 172.21.15.154:0/1922342097 shutdown_connections 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 --2- 172.21.15.154:0/1922342097 >> v2:172.21.15.191:3300/0 conn(0x7fca300c1780 0x7fca300c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 --2- 172.21.15.154:0/1922342097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca3013a8a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 -- 172.21.15.154:0/1922342097 >> 172.21.15.154:0/1922342097 conn(0x7fca300c3880 msgr2=0x7fca300c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:22.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 -- 172.21.15.154:0/1922342097 shutdown_connections 2024-03-14T09:03:22.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.901+0000 7fca37ff1700 1 -- 172.21.15.154:0/1922342097 wait complete. 2024-03-14T09:03:22.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.902+0000 7fca37ff1700 1 Processor -- start 2024-03-14T09:03:22.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.902+0000 7fca37ff1700 1 -- start start 2024-03-14T09:03:22.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300c1780 0x7fca3014e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca300c3250 0x7fca3014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca301ee4b0 con 0x7fca300a8540 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3014eff0 con 0x7fca300c3250 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca301ed830 con 0x7fca300c1780 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35378/0 (socket says 172.21.15.154:35378) 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 -- 172.21.15.154:0/608862217 learned_addr learned my addr 172.21.15.154:0/608862217 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca3658e700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca300c3250 0x7fca3014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca3558c700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300c1780 0x7fca3014e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 -- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300c1780 msgr2=0x7fca3014e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300c1780 0x7fca3014e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 -- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca300c3250 msgr2=0x7fca3014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca300c3250 0x7fca3014eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 -- 172.21.15.154:0/608862217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2000b7f0 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca35d8d700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fca300c43c0 tx=0x7fca20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20004230 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20010470 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca301edb10 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20010800 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.903+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca301ee0d0 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.904+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca301ee5f0 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.905+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fca20023440 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.905+0000 7fca267fc700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 0x7fca1406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.905+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fca2008ab10 con 0x7fca300a8540 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.905+0000 7fca3558c700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 0x7fca1406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:22.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.906+0000 7fca3558c700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 0x7fca1406ccf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fca300c13c0 tx=0x7fca2c0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:22.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:22.910+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fca301ee5f0 con 0x7fca300a8540 2024-03-14T09:03:23.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.101+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca3013ade0 con 0x7fca300a8540 2024-03-14T09:03:23.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.102+0000 7fca267fc700 1 -- 172.21.15.154:0/608862217 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca3013ade0 con 0x7fca300a8540 2024-03-14T09:03:23.102 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:23.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 msgr2=0x7fca1406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 0x7fca1406ccf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fca300c13c0 tx=0x7fca2c0090e0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 msgr2=0x7fca30154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fca300c43c0 tx=0x7fca20004750 comp rx=0 tx=0).stop 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 shutdown_connections 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca300c3250 0x7fca3014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca1406a830 0x7fca1406ccf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca300a8540 0x7fca30154db0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 --2- 172.21.15.154:0/608862217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca300c1780 0x7fca3014e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 >> 172.21.15.154:0/608862217 conn(0x7fca300c3880 msgr2=0x7fca300c55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 shutdown_connections 2024-03-14T09:03:23.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.104+0000 7fca37ff1700 1 -- 172.21.15.154:0/608862217 wait complete. 2024-03-14T09:03:23.116 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:03:23.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.242+0000 7ff0fd513700 1 Processor -- start 2024-03-14T09:03:23.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.242+0000 7ff0fd513700 1 -- start start 2024-03-14T09:03:23.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0fd513700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0f806ba10 0x7ff0f80c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0fd513700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f815f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0fd513700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f80c55e0 con 0x7ff0f80c4af0 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0fd513700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f80c5ac0 con 0x7ff0f806ba10 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0fd513700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80c8290 con 0x7ff0f815d370 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0f806ba10 0x7ff0f80c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0f806ba10 0x7ff0f80c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35392/0 (socket says 172.21.15.154:35392) 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 -- 172.21.15.154:0/3777184639 learned_addr learned my addr 172.21.15.154:0/3777184639 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f77fe700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 -- 172.21.15.154:0/3777184639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 msgr2=0x7ff0f815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f815f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 -- 172.21.15.154:0/3777184639 >> v1:172.21.15.191:6789/0 conn(0x7ff0f80c4af0 legacy=0x7ff0f806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.243+0000 7ff0f67fc700 1 -- 172.21.15.154:0/3777184639 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8154bf0 con 0x7ff0f806ba10 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0f67fc700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f806ba10 0x7ff0f80c4ed0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80098f0 tx=0x7ff0e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8562fda2b669ccd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0f5ffb700 1 -- 172.21.15.154:0/3777184639 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8011830 con 0x7ff0f806ba10 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0f5ffb700 1 -- 172.21.15.154:0/3777184639 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8011990 con 0x7ff0f806ba10 2024-03-14T09:03:23.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0f5ffb700 1 -- 172.21.15.154:0/3777184639 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8011d20 con 0x7ff0f806ba10 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 -- 172.21.15.154:0/3777184639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f806ba10 msgr2=0x7ff0f80c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f806ba10 0x7ff0f80c4ed0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80098f0 tx=0x7ff0e8009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 -- 172.21.15.154:0/3777184639 shutdown_connections 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 --2- 172.21.15.154:0/3777184639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f806ba10 0x7ff0f80c4ed0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 -- 172.21.15.154:0/3777184639 >> 172.21.15.154:0/3777184639 conn(0x7ff0f80abc60 msgr2=0x7ff0f80ac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 -- 172.21.15.154:0/3777184639 shutdown_connections 2024-03-14T09:03:23.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.244+0000 7ff0fd513700 1 -- 172.21.15.154:0/3777184639 wait complete. 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 Processor -- start 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 -- start start 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f806ba10 0x7ff0f81ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f8154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80c2630 con 0x7ff0f80c4af0 2024-03-14T09:03:23.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f81eada0 con 0x7ff0f806ba10 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.246+0000 7ff0fd513700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f814ed80 con 0x7ff0f815d370 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35398/0 (socket says 172.21.15.154:35398) 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 -- 172.21.15.154:0/2355537651 learned_addr learned my addr 172.21.15.154:0/2355537651 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f77fe700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f8154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f6ffd700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f806ba10 0x7ff0f81ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 -- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 msgr2=0x7ff0f8154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f8154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 -- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f806ba10 msgr2=0x7ff0f81ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f806ba10 0x7ff0f81ea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 -- 172.21.15.154:0/2355537651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8009550 con 0x7ff0f80c4af0 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f77fe700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f8154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:23.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0f67fc700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8072e80 tx=0x7ff0e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80036a0 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e801f070 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f8157620 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8003a40 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.247+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f8157340 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.248+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f80c5ea0 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.249+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff0e8003ba0 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.249+0000 7ff0df7fe700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 0x7ff0e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.249+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7ff0e8021070 con 0x7ff0f80c4af0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.249+0000 7ff0f6ffd700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 0x7ff0e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.250+0000 7ff0f6ffd700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 0x7ff0e406ccf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec007fd0 tx=0x7ff0ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.253+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff0e8055b10 con 0x7ff0f80c4af0 2024-03-14T09:03:23.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.446+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0f80697d0 con 0x7ff0f80c4af0 2024-03-14T09:03:23.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.447+0000 7ff0df7fe700 1 -- 172.21.15.154:0/2355537651 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0e802b020 con 0x7ff0f80c4af0 2024-03-14T09:03:23.447 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 msgr2=0x7ff0e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 0x7ff0e406ccf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec007fd0 tx=0x7ff0ec009040 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 msgr2=0x7ff0f80c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8072e80 tx=0x7ff0e8004750 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 shutdown_connections 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff0f815d370 0x7ff0f8154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0f806ba10 0x7ff0f81ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff0e406a830 0x7ff0e406ccf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 --2- 172.21.15.154:0/2355537651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f80c4af0 0x7ff0f80c1b30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 >> 172.21.15.154:0/2355537651 conn(0x7ff0f80abc60 msgr2=0x7ff0f806b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 shutdown_connections 2024-03-14T09:03:23.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.449+0000 7ff0fd513700 1 -- 172.21.15.154:0/2355537651 wait complete. 2024-03-14T09:03:23.461 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:03:23.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.586+0000 7f088a7af700 1 Processor -- start 2024-03-14T09:03:23.586 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.586+0000 7f088a7af700 1 -- start start 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f088a7af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 0x7f088406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f088a7af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f088415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f088a7af700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0884156bf0 con 0x7f08840a9650 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f088a7af700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0884156050 con 0x7f08840c1780 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f088a7af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08840c1c00 con 0x7f088406acb0 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0883fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f08840a9650 0x7f08841564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36274/0 (socket says 172.21.15.154:36274) 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0883fff700 1 -- 172.21.15.154:0/1007887991 learned_addr learned my addr 172.21.15.154:0/1007887991 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f08837fe700 1 --2- 172.21.15.154:0/1007887991 >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 0x7f088406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1007887991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f088415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389502853 0 0) 0x7f0874001110 con 0x7f08840a9650 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f086c0035b0 con 0x7f08840a9650 2024-03-14T09:03:23.589 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0888d4c700 1 -- 172.21.15.154:0/1007887991 >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 msgr2=0x7f088406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1007887991 >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 0x7f088406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0888d4c700 1 -- 172.21.15.154:0/1007887991 >> v1:172.21.15.154:6789/0 conn(0x7f08840a9650 legacy=0x7f08841564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517682465 0 0) 0x7f086c0035b0 con 0x7f08840a9650 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f0888d4c700 1 -- 172.21.15.154:0/1007887991 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840a86f0 con 0x7f088406acb0 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.587+0000 7f08837fe700 1 --2- 172.21.15.154:0/1007887991 >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 0x7f088406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1007887991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f088415cfd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f087c009d90 tx=0x7f087c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5f63f8c5bb6b013 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c00e070 con 0x7f088406acb0 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c003a20 con 0x7f088406acb0 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f0882ffd700 1 -- 172.21.15.154:0/1007887991 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874001800 con 0x7f088406acb0 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 -- 172.21.15.154:0/1007887991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 msgr2=0x7f088415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 --2- 172.21.15.154:0/1007887991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f088415cfd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f087c009d90 tx=0x7f087c00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 -- 172.21.15.154:0/1007887991 shutdown_connections 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 --2- 172.21.15.154:0/1007887991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f088415cfd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 --2- 172.21.15.154:0/1007887991 >> v2:172.21.15.191:3300/0 conn(0x7f08840c1780 0x7f088406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.590 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 -- 172.21.15.154:0/1007887991 >> 172.21.15.154:0/1007887991 conn(0x7f08840c3880 msgr2=0x7f08840c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.588+0000 7f088a7af700 1 -- 172.21.15.154:0/1007887991 shutdown_connections 2024-03-14T09:03:23.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.589+0000 7f088a7af700 1 -- 172.21.15.154:0/1007887991 wait complete. 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 Processor -- start 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 -- start start 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f08840c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 0x7f0884147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08841dc1c0 con 0x7f08840c1780 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884143f10 con 0x7f08840a9650 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.590+0000 7f088a7af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08841ec2b0 con 0x7f088406acb0 2024-03-14T09:03:23.594 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35414/0 (socket says 172.21.15.154:35414) 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 -- 172.21.15.154:0/1453287721 learned_addr learned my addr 172.21.15.154:0/1453287721 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0883fff700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f08840c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f08837fe700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 0x7f0884147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 -- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 msgr2=0x7f08840c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f08840c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 -- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 msgr2=0x7f0884147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 0x7f0884147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 -- 172.21.15.154:0/1453287721 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780094a0 con 0x7f08840c1780 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f08837fe700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 0x7f0884147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0888d4c700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f08840c1d40 tx=0x7f087c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c0045c0 con 0x7f08840c1780 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c004760 con 0x7f08840c1780 2024-03-14T09:03:23.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c009a70 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c01b430 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.591+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08841dbe50 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.592+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08841dbcb0 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.593+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f087c01b590 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.593+0000 7f0880ff9700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 0x7f087006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.593+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f087c08a750 con 0x7f08840c1780 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.593+0000 7f0883fff700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 0x7f087006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.594+0000 7f0883fff700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 0x7f087006ccf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f08840c6660 tx=0x7f0874001250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.598+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f08841dbcb0 con 0x7f08840c1780 2024-03-14T09:03:23.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.794+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08841dc370 con 0x7f08840c1780 2024-03-14T09:03:23.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.794+0000 7f0880ff9700 1 -- 172.21.15.154:0/1453287721 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08841dc370 con 0x7f08840c1780 2024-03-14T09:03:23.794 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 msgr2=0x7f087006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 0x7f087006ccf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f08840c6660 tx=0x7f0874001250 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 msgr2=0x7f08841e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f08840c1d40 tx=0x7f087c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 shutdown_connections 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f087006a830 0x7f087006ccf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08840c1780 0x7f08841e8290 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088406acb0 0x7f08840c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 --2- 172.21.15.154:0/1453287721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08840a9650 0x7f0884147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 >> 172.21.15.154:0/1453287721 conn(0x7f08840c3880 msgr2=0x7f0884071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 shutdown_connections 2024-03-14T09:03:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.796+0000 7f088a7af700 1 -- 172.21.15.154:0/1453287721 wait complete. 2024-03-14T09:03:23.808 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:03:23.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.932+0000 7f072aa89700 1 Processor -- start 2024-03-14T09:03:23.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.932+0000 7f072aa89700 1 -- start start 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f072aa89700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f072aa89700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f072aa89700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07240ac8e0 con 0x7f0724156330 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f072aa89700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072406b310 con 0x7f072414eaf0 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f072aa89700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072406de30 con 0x7f072406acb0 2024-03-14T09:03:23.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f0728825700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.933+0000 7f0723fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0728825700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:33752/0 (socket says 172.21.15.154:33752) 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0729026700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f0724156330 0x7f0724157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:34142/0 (socket says 172.21.15.154:34142) 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:36894/0 (socket says 172.21.15.154:36894) 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0728825700 1 -- 172.21.15.154:0/3173786691 learned_addr learned my addr 172.21.15.154:0/3173786691 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 -- 172.21.15.154:0/3173786691 >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 msgr2=0x7f072406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 --2- 172.21.15.154:0/3173786691 >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 -- 172.21.15.154:0/3173786691 >> v1:172.21.15.191:6789/0 conn(0x7f0724156330 legacy=0x7f0724157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f07237fe700 1 -- 172.21.15.154:0/3173786691 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193359748 0 0) 0x7f07240ac8e0 con 0x7f0724156330 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 -- 172.21.15.154:0/3173786691 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240acb30 con 0x7f072406acb0 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0728825700 1 --2- 172.21.15.154:0/3173786691 >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f0723fff700 1 --2- 172.21.15.154:0/3173786691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0710009840 tx=0x7f0710009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ebbad5ecc24aa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f07237fe700 1 -- 172.21.15.154:0/3173786691 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011830 con 0x7f072406acb0 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f07237fe700 1 -- 172.21.15.154:0/3173786691 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710011990 con 0x7f072406acb0 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.934+0000 7f07237fe700 1 -- 172.21.15.154:0/3173786691 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011d20 con 0x7f072406acb0 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 -- 172.21.15.154:0/3173786691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 msgr2=0x7f0724155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 --2- 172.21.15.154:0/3173786691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0710009840 tx=0x7f0710009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 -- 172.21.15.154:0/3173786691 shutdown_connections 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 --2- 172.21.15.154:0/3173786691 >> v2:172.21.15.191:3300/0 conn(0x7f072414eaf0 0x7f072406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 --2- 172.21.15.154:0/3173786691 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f072406acb0 0x7f0724155df0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 -- 172.21.15.154:0/3173786691 >> 172.21.15.154:0/3173786691 conn(0x7f07240c3880 msgr2=0x7f07240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 -- 172.21.15.154:0/3173786691 shutdown_connections 2024-03-14T09:03:23.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.935+0000 7f072aa89700 1 -- 172.21.15.154:0/3173786691 wait complete. 2024-03-14T09:03:23.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.936+0000 7f072aa89700 1 Processor -- start 2024-03-14T09:03:23.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.936+0000 7f072aa89700 1 -- start start 2024-03-14T09:03:23.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 0x7f07240c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0724156330 0x7f07241540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072414a560 con 0x7f072406acb0 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072414a280 con 0x7f072414eaf0 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f072aa89700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07241447e0 con 0x7f0724156330 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35426/0 (socket says 172.21.15.154:35426) 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 -- 172.21.15.154:0/2798120801 learned_addr learned my addr 172.21.15.154:0/2798120801 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0723fff700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 0x7f07240c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0729026700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0724156330 0x7f07241540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 -- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0724156330 msgr2=0x7f07241540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0724156330 0x7f07241540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 -- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 msgr2=0x7f07240c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 0x7f07240c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:23.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 -- 172.21.15.154:0/2798120801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718009550 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0723fff700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 0x7f07240c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.937+0000 7f0728825700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f071800e9b0 tx=0x7f071800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.938+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800c850 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.938+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071800c9b0 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.938+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07100094a0 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.938+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800cd20 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.938+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07240c5710 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.939+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0724144d00 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.939+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0718013470 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.939+0000 7f07217fa700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 0x7f070806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.939+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f0718089180 con 0x7f072406acb0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.940+0000 7f0723fff700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 0x7f070806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.940+0000 7f0723fff700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 0x7f070806ccf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07100090b0 tx=0x7f071001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:23.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:23.944+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f07180542d0 con 0x7f072406acb0 2024-03-14T09:03:24.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.141+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07240c58a0 con 0x7f072406acb0 2024-03-14T09:03:24.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.141+0000 7f07217fa700 1 -- 172.21.15.154:0/2798120801 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v55) v1 ==== 105+0+11 (secure 0 0 0) 0x7f07180578f0 con 0x7f072406acb0 2024-03-14T09:03:24.141 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:24.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 msgr2=0x7f070806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 0x7f070806ccf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07100090b0 tx=0x7f071001c040 comp rx=0 tx=0).stop 2024-03-14T09:03:24.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 msgr2=0x7f07240c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f071800e9b0 tx=0x7f071800ed70 comp rx=0 tx=0).stop 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 shutdown_connections 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0724156330 0x7f07241540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f070806a830 0x7f070806ccf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f072406acb0 0x7f07240c5c30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 --2- 172.21.15.154:0/2798120801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f072414eaf0 0x7f07240c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 >> 172.21.15.154:0/2798120801 conn(0x7f07240c3880 msgr2=0x7f0724155a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 shutdown_connections 2024-03-14T09:03:24.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.143+0000 7f072aa89700 1 -- 172.21.15.154:0/2798120801 wait complete. 2024-03-14T09:03:24.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:03:24.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.281+0000 7fe60b1e5700 1 Processor -- start 2024-03-14T09:03:24.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.281+0000 7fe60b1e5700 1 -- start start 2024-03-14T09:03:24.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe60b1e5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe60414fbd0 0x7fe60415df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe60b1e5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6040acd10 con 0x7fe6040702f0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe60b1e5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6040ac9c0 con 0x7fe6040c4380 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe60b1e5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6040ac7b0 con 0x7fe60414fbd0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe609782700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe60414fbd0 0x7fe60415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe608f81700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe6040702f0 0x7fe6040c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36286/0 (socket says 172.21.15.154:36286) 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe608f81700 1 -- 172.21.15.154:0/1192739301 learned_addr learned my addr 172.21.15.154:0/1192739301 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe609782700 1 -- 172.21.15.154:0/1192739301 >> v1:172.21.15.191:6789/0 conn(0x7fe6040c4380 legacy=0x7fe6040c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe609782700 1 -- 172.21.15.154:0/1192739301 >> v1:172.21.15.154:6789/0 conn(0x7fe6040702f0 legacy=0x7fe6040c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe6037fe700 1 -- 172.21.15.154:0/1192739301 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900066750 0 0) 0x7fe6040acd10 con 0x7fe6040702f0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.282+0000 7fe609782700 1 -- 172.21.15.154:0/1192739301 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040707f0 con 0x7fe60414fbd0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe609782700 1 --2- 172.21.15.154:0/1192739301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe60415df60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009d90 tx=0x7fe5fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368318c9d795439b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe6037fe700 1 -- 172.21.15.154:0/1192739301 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00e070 con 0x7fe60414fbd0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe6037fe700 1 -- 172.21.15.154:0/1192739301 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc00bbd0 con 0x7fe60414fbd0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe6037fe700 1 -- 172.21.15.154:0/1192739301 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc012430 con 0x7fe60414fbd0 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 -- 172.21.15.154:0/1192739301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 msgr2=0x7fe60415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/1192739301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe60415df60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009d90 tx=0x7fe5fc00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 -- 172.21.15.154:0/1192739301 shutdown_connections 2024-03-14T09:03:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/1192739301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe60415df60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 -- 172.21.15.154:0/1192739301 >> 172.21.15.154:0/1192739301 conn(0x7fe6040a9bf0 msgr2=0x7fe6040a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 -- 172.21.15.154:0/1192739301 shutdown_connections 2024-03-14T09:03:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.283+0000 7fe60b1e5700 1 -- 172.21.15.154:0/1192739301 wait complete. 2024-03-14T09:03:24.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.284+0000 7fe60b1e5700 1 Processor -- start 2024-03-14T09:03:24.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.284+0000 7fe60b1e5700 1 -- start start 2024-03-14T09:03:24.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6040702f0 0x7fe60406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6040c4380 0x7fe604145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041ee4b0 con 0x7fe60414fbd0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604145aa0 con 0x7fe6040702f0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe60b1e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe604067ff0 con 0x7fe6040c4380 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe609782700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe609782700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35456/0 (socket says 172.21.15.154:35456) 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe609782700 1 -- 172.21.15.154:0/3301855894 learned_addr learned my addr 172.21.15.154:0/3301855894 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.285+0000 7fe603fff700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6040c4380 0x7fe604145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe608f81700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6040702f0 0x7fe60406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 -- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6040c4380 msgr2=0x7fe604145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6040c4380 0x7fe604145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 -- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6040702f0 msgr2=0x7fe60406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6040702f0 0x7fe60406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 -- 172.21.15.154:0/3301855894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc009a70 con 0x7fe60414fbd0 2024-03-14T09:03:24.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe609782700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc014040 tx=0x7fe5fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc01b810 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc020070 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6041ed8a0 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc01bbb0 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.286+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041ede60 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.287+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604153730 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.288+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe5fc01bd10 con 0x7fe60414fbd0 2024-03-14T09:03:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.288+0000 7fe6017fa700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 0x7fe5e806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.288+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fe5fc022070 con 0x7fe60414fbd0 2024-03-14T09:03:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.288+0000 7fe608f81700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 0x7fe5e806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.289+0000 7fe608f81700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 0x7fe5e806ccf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8007b10 tx=0x7fe5f8007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.292+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe5fc056600 con 0x7fe60414fbd0 2024-03-14T09:03:24.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.488+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fe6040697d0 con 0x7fe60414fbd0 2024-03-14T09:03:24.488 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.488+0000 7fe6017fa700 1 -- 172.21.15.154:0/3301855894 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v55) v1 ==== 101+0+11 (secure 0 0 0) 0x7fe5fc059c20 con 0x7fe60414fbd0 2024-03-14T09:03:24.488 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 msgr2=0x7fe5e806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 0x7fe5e806ccf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8007b10 tx=0x7fe5f8007a60 comp rx=0 tx=0).stop 2024-03-14T09:03:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 msgr2=0x7fe604145560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc014040 tx=0x7fe5fc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 shutdown_connections 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5e806a830 0x7fe5e806ccf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe60414fbd0 0x7fe604145560 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6040702f0 0x7fe60406cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 --2- 172.21.15.154:0/3301855894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe6040c4380 0x7fe604145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 >> 172.21.15.154:0/3301855894 conn(0x7fe6040a9bf0 msgr2=0x7fe6040a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 shutdown_connections 2024-03-14T09:03:24.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.490+0000 7fe60b1e5700 1 -- 172.21.15.154:0/3301855894 wait complete. 2024-03-14T09:03:24.503 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:03:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.625+0000 7fce3c6dd700 1 Processor -- start 2024-03-14T09:03:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.625+0000 7fce3c6dd700 1 -- start start 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3c6dd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce34152f80 0x7fce340c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3c6dd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce34070e10 con 0x7fce340c2510 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3c6dd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3414c9a0 con 0x7fce3414c5c0 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3c6dd700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce340c1bb0 con 0x7fce34152f80 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce34152f80 0x7fce340c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3ac7a700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fce340c2510 0x7fce3415ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36302/0 (socket says 172.21.15.154:36302) 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce34152f80 0x7fce340c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35468/0 (socket says 172.21.15.154:35468) 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce3ac7a700 1 -- 172.21.15.154:0/3726450111 learned_addr learned my addr 172.21.15.154:0/3726450111 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:24.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 -- 172.21.15.154:0/3726450111 >> v1:172.21.15.191:6789/0 conn(0x7fce3414c5c0 legacy=0x7fce34072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 -- 172.21.15.154:0/3726450111 >> v1:172.21.15.154:6789/0 conn(0x7fce340c2510 legacy=0x7fce3415ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 -- 172.21.15.154:0/3726450111 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340c1580 con 0x7fce34152f80 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39c78700 1 --2- 172.21.15.154:0/3726450111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce34152f80 0x7fce340c1fd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce24009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ac83286afdf7de server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39477700 1 -- 172.21.15.154:0/3726450111 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24004230 con 0x7fce34152f80 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39477700 1 -- 172.21.15.154:0/3726450111 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24004390 con 0x7fce34152f80 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.626+0000 7fce39477700 1 -- 172.21.15.154:0/3726450111 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240036a0 con 0x7fce34152f80 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 -- 172.21.15.154:0/3726450111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce34152f80 msgr2=0x7fce340c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/3726450111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce34152f80 0x7fce340c1fd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce24009b80 comp rx=0 tx=0).stop 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 -- 172.21.15.154:0/3726450111 shutdown_connections 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/3726450111 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce34152f80 0x7fce340c1fd0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 -- 172.21.15.154:0/3726450111 >> 172.21.15.154:0/3726450111 conn(0x7fce3406a770 msgr2=0x7fce3406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 -- 172.21.15.154:0/3726450111 shutdown_connections 2024-03-14T09:03:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.627+0000 7fce3c6dd700 1 -- 172.21.15.154:0/3726450111 wait complete. 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.628+0000 7fce3c6dd700 1 Processor -- start 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.628+0000 7fce3c6dd700 1 -- start start 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce3414c5c0 0x7fce341ead30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce34152f80 0x7fce340ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3406e4b0 con 0x7fce340c2510 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce340c0470 con 0x7fce3414c5c0 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3c6dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce340c0750 con 0x7fce34152f80 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35480/0 (socket says 172.21.15.154:35480) 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 -- 172.21.15.154:0/499912 learned_addr learned my addr 172.21.15.154:0/499912 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 -- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce34152f80 msgr2=0x7fce340ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce34152f80 0x7fce340ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 -- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce3414c5c0 msgr2=0x7fce341ead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce3414c5c0 0x7fce341ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 -- 172.21.15.154:0/499912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240094a0 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce3a479700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fce3000c4d0 tx=0x7fce3000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.629+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30011790 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.630+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce300118f0 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.630+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce341ea250 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.630+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30011ca0 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.630+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce341ea810 con 0x7fce340c2510 2024-03-14T09:03:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.631+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34072d30 con 0x7fce340c2510 2024-03-14T09:03:24.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.631+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fce3001c460 con 0x7fce340c2510 2024-03-14T09:03:24.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.631+0000 7fce2affd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 0x7fce1806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:24.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.631+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fce30088e70 con 0x7fce340c2510 2024-03-14T09:03:24.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.632+0000 7fce39c78700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 0x7fce1806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:24.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.632+0000 7fce39c78700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 0x7fce1806ccf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:24.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.636+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fce30053fc0 con 0x7fce340c2510 2024-03-14T09:03:24.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.845+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce340697d0 con 0x7fce340c2510 2024-03-14T09:03:24.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.845+0000 7fce2affd700 1 -- 172.21.15.154:0/499912 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce300575e0 con 0x7fce340c2510 2024-03-14T09:03:24.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 msgr2=0x7fce1806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 0x7fce1806ccf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fce24000c00 tx=0x7fce24013040 comp rx=0 tx=0).stop 2024-03-14T09:03:24.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 msgr2=0x7fce3414fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:24.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fce3000c4d0 tx=0x7fce3000c890 comp rx=0 tx=0).stop 2024-03-14T09:03:24.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 shutdown_connections 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce34152f80 0x7fce340ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fce1806a830 0x7fce1806ccf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce340c2510 0x7fce3414fe00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 --2- 172.21.15.154:0/499912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce3414c5c0 0x7fce341ead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 >> 172.21.15.154:0/499912 conn(0x7fce3406a770 msgr2=0x7fce341dd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 shutdown_connections 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:24.847+0000 7fce3c6dd700 1 -- 172.21.15.154:0/499912 wait complete. 2024-03-14T09:03:24.848 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:03:24.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:03:24.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:24.859 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:24.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:03:24.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:24.905 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:03:24.924 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:24.925 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:24.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:03:24.985 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:03:25.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.107+0000 7f2fd5e8a700 1 Processor -- start 2024-03-14T09:03:25.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.107+0000 7f2fd5e8a700 1 -- start start 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fd5e8a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fd00702f0 0x7f2fd00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fd5e8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 0x7f2fd00c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fd5e8a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd00acd10 con 0x7f2fd014fbd0 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fd5e8a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00ac7b0 con 0x7f2fd00702f0 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fd5e8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0150050 con 0x7f2fd00c5d60 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fcf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fd00702f0 0x7f2fd00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.108+0000 7f2fcffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 0x7f2fd00c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fd00702f0 0x7f2fd00c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:33792/0 (socket says 172.21.15.154:33792) 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 -- 172.21.15.154:0/3687758106 learned_addr learned my addr 172.21.15.154:0/3687758106 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fceffd700 1 --1- 172.21.15.154:0/3687758106 >> v1:172.21.15.191:6789/0 conn(0x7f2fd014fbd0 0x7f2fd00c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:34176/0 (socket says 172.21.15.154:34176) 2024-03-14T09:03:25.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 -- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 msgr2=0x7f2fd00c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 0x7f2fd00c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 -- 172.21.15.154:0/3687758106 >> v1:172.21.15.191:6789/0 conn(0x7f2fd014fbd0 legacy=0x7f2fd00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 -- 172.21.15.154:0/3687758106 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00ac9c0 con 0x7f2fd00702f0 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcffff700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 0x7f2fd00c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fcf7fe700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd00c4380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098f0 tx=0x7f2fc0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a0c1b1e3971fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fce7fc700 1 -- 172.21.15.154:0/3687758106 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0011830 con 0x7f2fd00702f0 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fce7fc700 1 -- 172.21.15.154:0/3687758106 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc0011990 con 0x7f2fd00702f0 2024-03-14T09:03:25.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.109+0000 7f2fce7fc700 1 -- 172.21.15.154:0/3687758106 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0011d20 con 0x7f2fd00702f0 2024-03-14T09:03:25.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 msgr2=0x7f2fd00c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd00c4380 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00098f0 tx=0x7f2fc0009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:25.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3687758106 shutdown_connections 2024-03-14T09:03:25.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd00c5d60 0x7f2fd00c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3687758106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd00c4380 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3687758106 >> 172.21.15.154:0/3687758106 conn(0x7f2fd00a9bf0 msgr2=0x7f2fd00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3687758106 shutdown_connections 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.110+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3687758106 wait complete. 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.111+0000 7f2fd5e8a700 1 Processor -- start 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.111+0000 7f2fd5e8a700 1 -- start start 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd0145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd014fbd0 0x7f2fd0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01ec6f0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0153e50 con 0x7f2fd00702f0 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fd5e8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0145280 con 0x7f2fd014fbd0 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35488/0 (socket says 172.21.15.154:35488) 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 -- 172.21.15.154:0/3804890795 learned_addr learned my addr 172.21.15.154:0/3804890795 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fcf7fe700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd0145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fcffff700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd014fbd0 0x7f2fd0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 -- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd014fbd0 msgr2=0x7f2fd0151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd014fbd0 0x7f2fd0151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 -- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 msgr2=0x7f2fd0145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd0145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 -- 172.21.15.154:0/3804890795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009a70 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.112+0000 7f2fceffd700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c520 tx=0x7f2fc400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4005cb0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4005e10 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc0009550 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40072d0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd0152f60 con 0x7f2fd00c5d60 2024-03-14T09:03:25.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.113+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0070ab0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.114+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2fc4007430 con 0x7f2fd00c5d60 2024-03-14T09:03:25.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.115+0000 7f2fb7fff700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:25.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.115+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f2fc4088510 con 0x7f2fd00c5d60 2024-03-14T09:03:25.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.115+0000 7f2fcf7fe700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:25.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.115+0000 7f2fcf7fe700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0014040 tx=0x7f2fc0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:25.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.119+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2fc40537f0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.361+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fd0070ab0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.361+0000 7f2fb7fff700 1 -- 172.21.15.154:0/3804890795 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+149 (secure 0 0 0) 0x7f2fc40170b0 con 0x7f2fd00c5d60 2024-03-14T09:03:25.361 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:03:25.361 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 msgr2=0x7f2fbc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0014040 tx=0x7f2fc0008030 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 msgr2=0x7f2fd01eba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c520 tx=0x7f2fc400c830 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 shutdown_connections 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd014fbd0 0x7f2fd0151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd00702f0 0x7f2fd0145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 --2- 172.21.15.154:0/3804890795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd00c5d60 0x7f2fd01eba60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.363+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 >> 172.21.15.154:0/3804890795 conn(0x7f2fd00a9bf0 msgr2=0x7f2fd00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.364+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 shutdown_connections 2024-03-14T09:03:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:25.364+0000 7f2fd5e8a700 1 -- 172.21.15.154:0/3804890795 wait complete. 2024-03-14T09:03:25.375 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:03:25.376 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:25.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.376 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:25.398 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42259sec preferred_lft 42259sec 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:25.399 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:25.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.400 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:03:25.469 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:03:25.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.471 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:03:25.533 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:03:25.533 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:03:25.533 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:03:25.533 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:03:25.533 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:03:25.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:03:25.534 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:03:25.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.646 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:03:25.663 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:03:25.663 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:25.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.663 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:03:25.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:25.731 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:25.788 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:03:25.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:25.788 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:25.845 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-03-14T09:03:25.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:25.931 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:25.931 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:03:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:25.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:25.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:25.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:25.968 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:03:25.969 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:03:25.971 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:03:25.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.971 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:25.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:25.999 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:03:26.088 INFO:teuthology.orchestra.run.smithi191.stdout:client5653 2024-03-14T09:03:26.088 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:26.089 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:26.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth add client.testuser mon 'allow rw fsname=cephfs, allow rw fsname=cephfs2' osd 'allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2' mds 'allow rw fsname=cephfs' 2024-03-14T09:03:26.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:03:26.183 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:03:26.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 Processor -- start 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 -- start start 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9000a8540 0x7fd90013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9000c1780 0x7fd9000c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90013ade0 con 0x7fd9000c3250 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90006fb30 con 0x7fd9000a8540 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.242+0000 7fd905931700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90006ec30 con 0x7fd9000c1780 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9000a8540 0x7fd90013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9000a8540 0x7fd90013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35504/0 (socket says 172.21.15.154:35504) 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 -- 172.21.15.154:0/261171142 learned_addr learned my addr 172.21.15.154:0/261171142 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8ff7fe700 1 --2- 172.21.15.154:0/261171142 >> v2:172.21.15.191:3300/0 conn(0x7fd9000c1780 0x7fd9000c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 -- 172.21.15.154:0/261171142 >> v2:172.21.15.191:3300/0 conn(0x7fd9000c1780 msgr2=0x7fd9000c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 --2- 172.21.15.154:0/261171142 >> v2:172.21.15.191:3300/0 conn(0x7fd9000c1780 0x7fd9000c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 -- 172.21.15.154:0/261171142 >> v1:172.21.15.191:6789/0 conn(0x7fd9000c3250 legacy=0x7fd900154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 -- 172.21.15.154:0/261171142 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90006ef10 con 0x7fd9000a8540 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8feffd700 1 --2- 172.21.15.154:0/261171142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000a8540 0x7fd90013a8a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd900072e40 tx=0x7fd8f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e753c8fabe2470e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8fdffb700 1 -- 172.21.15.154:0/261171142 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000b870 con 0x7fd9000a8540 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8fdffb700 1 -- 172.21.15.154:0/261171142 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f000b9d0 con 0x7fd9000a8540 2024-03-14T09:03:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.243+0000 7fd8fdffb700 1 -- 172.21.15.154:0/261171142 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000bd60 con 0x7fd9000a8540 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 -- 172.21.15.154:0/261171142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000a8540 msgr2=0x7fd90013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 --2- 172.21.15.154:0/261171142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000a8540 0x7fd90013a8a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd900072e40 tx=0x7fd8f0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 -- 172.21.15.154:0/261171142 shutdown_connections 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 --2- 172.21.15.154:0/261171142 >> v2:172.21.15.191:3300/0 conn(0x7fd9000c1780 0x7fd9000c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 --2- 172.21.15.154:0/261171142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000a8540 0x7fd90013a8a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 -- 172.21.15.154:0/261171142 >> 172.21.15.154:0/261171142 conn(0x7fd9000c3880 msgr2=0x7fd9000c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 -- 172.21.15.154:0/261171142 shutdown_connections 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.244+0000 7fd905931700 1 -- 172.21.15.154:0/261171142 wait complete. 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.245+0000 7fd905931700 1 Processor -- start 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 -- start start 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9000a8540 0x7fd9001e8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9000c1780 0x7fd9001e9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000acbe0 con 0x7fd9000c3250 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001eb880 con 0x7fd9000a8540 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.246+0000 7fd905931700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000a9ea0 con 0x7fd9000c1780 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35518/0 (socket says 172.21.15.154:35518) 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 -- 172.21.15.154:0/784524838 learned_addr learned my addr 172.21.15.154:0/784524838 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8feffd700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9000a8540 0x7fd9001e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8fe7fc700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9000c1780 0x7fd9001e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 -- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9000c1780 msgr2=0x7fd9001e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9000c1780 0x7fd9001e9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 -- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9000a8540 msgr2=0x7fd9001e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9000a8540 0x7fd9001e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 -- 172.21.15.154:0/784524838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0009550 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8ff7fe700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8007e90 tx=0x7fd8f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800ede0 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f8014420 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9001eb510 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80147d0 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.247+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd900130a80 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.248+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9001eba10 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.249+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd8f8014930 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.249+0000 7fd8e7fff700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 0x7fd8ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.249+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd8f8089740 con 0x7fd9000c3250 2024-03-14T09:03:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.249+0000 7fd8feffd700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 0x7fd8ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.250+0000 7fd8feffd700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 0x7fd8ec06ccf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009c10 tx=0x7fd8f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.253+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd8f8054890 con 0x7fd9000c3250 2024-03-14T09:03:26.535 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.535+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]} v 0) v1 -- 0x7fd9000697d0 con 0x7fd9000c3250 2024-03-14T09:03:26.538 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.538+0000 7fd8e7fff700 1 -- 172.21.15.154:0/784524838 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow rw fsname=cephfs, allow rw fsname=cephfs2", "osd", "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2", "mds", "allow rw fsname=cephfs"]}]=0 added key for client.testuser v21) v1 ==== 291+0+0 (secure 0 0 0) 0x7fd8f8057eb0 con 0x7fd9000c3250 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 msgr2=0x7fd8ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 0x7fd8ec06ccf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009c10 tx=0x7fd8f0008030 comp rx=0 tx=0).stop 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 msgr2=0x7fd900151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8007e90 tx=0x7fd8f800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 shutdown_connections 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8ec06a830 0x7fd8ec06ccf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9000c3250 0x7fd900151870 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9000a8540 0x7fd9001e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 --2- 172.21.15.154:0/784524838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9000c1780 0x7fd9001e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 >> 172.21.15.154:0/784524838 conn(0x7fd9000c3880 msgr2=0x7fd900135f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:26.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 shutdown_connections 2024-03-14T09:03:26.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.541+0000 7fd905931700 1 -- 172.21.15.154:0/784524838 wait complete. 2024-03-14T09:03:26.542 INFO:teuthology.orchestra.run.smithi154.stderr:added key for client.testuser 2024-03-14T09:03:26.553 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.testuser 2024-03-14T09:03:26.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 Processor -- start 2024-03-14T09:03:26.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 -- start start 2024-03-14T09:03:26.719 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36fc0c5150 0x7f36fc154e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 0x7f36fc0c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc0707b0 con 0x7f36fc0c5150 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc0c6a20 con 0x7f36fc155350 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.717+0000 7f37018d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0c6460 con 0x7f36fc0c87e0 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36fc0c5150 0x7f36fc154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36fc0c5150 0x7f36fc154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35520/0 (socket says 172.21.15.154:35520) 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 -- 172.21.15.154:0/3747189851 learned_addr learned my addr 172.21.15.154:0/3747189851 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fb7fe700 1 --2- 172.21.15.154:0/3747189851 >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 0x7f36fc0c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36faffd700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 -- 172.21.15.154:0/3747189851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 msgr2=0x7f36fc0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:26.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 -- 172.21.15.154:0/3747189851 >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 msgr2=0x7f36fc0c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/3747189851 >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 0x7f36fc0c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 -- 172.21.15.154:0/3747189851 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0c6740 con 0x7f36fc0c5150 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fb7fe700 1 --2- 172.21.15.154:0/3747189851 >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 0x7f36fc0c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc154e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0aacc0 tx=0x7f36f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a4b922025db6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36f9ffb700 1 -- 172.21.15.154:0/3747189851 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000b870 con 0x7f36fc0c5150 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36f9ffb700 1 -- 172.21.15.154:0/3747189851 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000b9d0 con 0x7f36fc0c5150 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.718+0000 7f36f9ffb700 1 -- 172.21.15.154:0/3747189851 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000bd60 con 0x7f36fc0c5150 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 -- 172.21.15.154:0/3747189851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 msgr2=0x7f36fc154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc154e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0aacc0 tx=0x7f36f0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 -- 172.21.15.154:0/3747189851 shutdown_connections 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 --2- 172.21.15.154:0/3747189851 >> v2:172.21.15.191:3300/0 conn(0x7f36fc155350 0x7f36fc0c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 --2- 172.21.15.154:0/3747189851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc154e10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 -- 172.21.15.154:0/3747189851 >> 172.21.15.154:0/3747189851 conn(0x7f36fc06e4a0 msgr2=0x7f36fc06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:26.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 -- 172.21.15.154:0/3747189851 shutdown_connections 2024-03-14T09:03:26.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.719+0000 7f37018d8700 1 -- 172.21.15.154:0/3747189851 wait complete. 2024-03-14T09:03:26.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.720+0000 7f37018d8700 1 Processor -- start 2024-03-14T09:03:26.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.720+0000 7f37018d8700 1 -- start start 2024-03-14T09:03:26.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36fc155350 0x7f36fc140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc06d480 con 0x7f36fc0c5150 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc06d080 con 0x7f36fc155350 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f37018d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc140fd0 con 0x7f36fc0c87e0 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35530/0 (socket says 172.21.15.154:35530) 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 -- 172.21.15.154:0/910527882 learned_addr learned my addr 172.21.15.154:0/910527882 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36fb7fe700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36fc155350 0x7f36fc140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 -- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 msgr2=0x7f36fc140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 -- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36fc155350 msgr2=0x7f36fc140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36fc155350 0x7f36fc140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:26.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 -- 172.21.15.154:0/910527882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0009550 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.721+0000 7f36faffd700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00e810 tx=0x7f36ec00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.722+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec00c070 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.722+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec013610 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.722+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc06d6b0 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.722+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0139a0 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.722+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc14aa40 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.723+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc14adb0 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.723+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f36ec013b00 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.723+0000 7f36e3fff700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 0x7f36e806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.723+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f36ec088cb0 con 0x7f36fc0c5150 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.724+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 0x7f36e806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:26.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.724+0000 7f36fa7fc700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 0x7f36e806ccf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f36f0014040 tx=0x7f36f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:26.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:26.728+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f36ec053e00 con 0x7f36fc0c5150 2024-03-14T09:03:27.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.000+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.testuser"} v 0) v1 -- 0x7f36fc1412d0 con 0x7f36fc0c5150 2024-03-14T09:03:27.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.001+0000 7f36e3fff700 1 -- 172.21.15.154:0/910527882 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.testuser"}]=0 v21) v1 ==== 85+0+245 (secure 0 0 0) 0x7f36ec009440 con 0x7f36fc0c5150 2024-03-14T09:03:27.001 INFO:teuthology.orchestra.run.smithi154.stdout:[client.testuser] 2024-03-14T09:03:27.001 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA== 2024-03-14T09:03:27.001 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow rw fsname=cephfs" 2024-03-14T09:03:27.001 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow rw fsname=cephfs, allow rw fsname=cephfs2" 2024-03-14T09:03:27.001 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-03-14T09:03:27.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 msgr2=0x7f36e806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 0x7f36e806ccf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f36f0014040 tx=0x7f36f0008030 comp rx=0 tx=0).stop 2024-03-14T09:03:27.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 msgr2=0x7f36fc06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00e810 tx=0x7f36ec00eb20 comp rx=0 tx=0).stop 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 shutdown_connections 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36fc155350 0x7f36fc140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f36e806a830 0x7f36e806ccf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36fc0c5150 0x7f36fc06b880 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 --2- 172.21.15.154:0/910527882 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36fc0c87e0 0x7f36fc140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 >> 172.21.15.154:0/910527882 conn(0x7f36fc06e4a0 msgr2=0x7f36fc14d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 shutdown_connections 2024-03-14T09:03:27.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.003+0000 7f37018d8700 1 -- 172.21.15.154:0/910527882 wait complete. 2024-03-14T09:03:27.015 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:03:27.032 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.DmznG5J3zd 2024-03-14T09:03:27.032 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:27.032 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/tmp/tmp.DmznG5J3zd 2024-03-14T09:03:27.089 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:27.089 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:27.147 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:27.147 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:27.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:27.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.369+0000 7f4d4991a700 1 Processor -- start 2024-03-14T09:03:27.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.369+0000 7f4d4991a700 1 -- start start 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d4991a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d440c3250 0x7f4d4413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d4991a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 0x7f4d440c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d4991a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4413ade0 con 0x7f4d440a8540 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d4991a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4406fb30 con 0x7f4d440c3250 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d4991a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4406ec30 con 0x7f4d440c1780 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d440c3250 0x7f4d4413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d440c3250 0x7f4d4413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35546/0 (socket says 172.21.15.154:35546) 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 -- 172.21.15.154:0/2625118038 learned_addr learned my addr 172.21.15.154:0/2625118038 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d42ffd700 1 --1- 172.21.15.154:0/2625118038 >> v1:172.21.15.154:6789/0 conn(0x7f4d440a8540 0x7f4d44154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36314/0 (socket says 172.21.15.154:36314) 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d437fe700 1 --2- 172.21.15.154:0/2625118038 >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 0x7f4d440c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 -- 172.21.15.154:0/2625118038 >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 msgr2=0x7f4d440c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 --2- 172.21.15.154:0/2625118038 >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 0x7f4d440c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.370+0000 7f4d427fc700 1 -- 172.21.15.154:0/2625118038 >> v1:172.21.15.154:6789/0 conn(0x7f4d440a8540 legacy=0x7f4d44154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d427fc700 1 -- 172.21.15.154:0/2625118038 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4406ef10 con 0x7f4d440c3250 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d41ffb700 1 -- 172.21.15.154:0/2625118038 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588096896 0 0) 0x7f4d38001110 con 0x7f4d440a8540 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d437fe700 1 --2- 172.21.15.154:0/2625118038 >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 0x7f4d440c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d427fc700 1 --2- 172.21.15.154:0/2625118038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d4413a8a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4d44072e40 tx=0x7f4d3400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b241858d0e4f238 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d41ffb700 1 -- 172.21.15.154:0/2625118038 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400dd30 con 0x7f4d440c3250 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d41ffb700 1 -- 172.21.15.154:0/2625118038 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3400de90 con 0x7f4d440c3250 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d41ffb700 1 -- 172.21.15.154:0/2625118038 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34010690 con 0x7f4d440c3250 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 -- 172.21.15.154:0/2625118038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 msgr2=0x7f4d4413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 --2- 172.21.15.154:0/2625118038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d4413a8a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4d44072e40 tx=0x7f4d3400be70 comp rx=0 tx=0).stop 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 -- 172.21.15.154:0/2625118038 shutdown_connections 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 --2- 172.21.15.154:0/2625118038 >> v2:172.21.15.191:3300/0 conn(0x7f4d440c1780 0x7f4d440c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 --2- 172.21.15.154:0/2625118038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d4413a8a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 -- 172.21.15.154:0/2625118038 >> 172.21.15.154:0/2625118038 conn(0x7f4d440c3880 msgr2=0x7f4d440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:27.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 -- 172.21.15.154:0/2625118038 shutdown_connections 2024-03-14T09:03:27.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.371+0000 7f4d4991a700 1 -- 172.21.15.154:0/2625118038 wait complete. 2024-03-14T09:03:27.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.373+0000 7f4d4991a700 1 Processor -- start 2024-03-14T09:03:27.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.373+0000 7f4d4991a700 1 -- start start 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d440a8540 0x7f4d441dd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d440c1780 0x7f4d44135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440ab9f0 con 0x7f4d440c3250 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440aaa40 con 0x7f4d440c1780 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d4991a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440aa760 con 0x7f4d440a8540 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35558/0 (socket says 172.21.15.154:35558) 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 -- 172.21.15.154:0/3369194939 learned_addr learned my addr 172.21.15.154:0/3369194939 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d42ffd700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d440a8540 0x7f4d441dd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d427fc700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d440c1780 0x7f4d44135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 -- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d440a8540 msgr2=0x7f4d441dd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d440a8540 0x7f4d441dd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 -- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d440c1780 msgr2=0x7f4d44135510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d440c1780 0x7f4d44135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.374+0000 7f4d437fe700 1 -- 172.21.15.154:0/3369194939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3400b7f0 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d437fe700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c007e70 tx=0x7f4d3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c00de40 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c00f040 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d44134a30 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c003c60 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.375+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d44134ff0 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.376+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44135210 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.376+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4d3c004030 con 0x7f4d440c3250 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.376+0000 7f4d2bfff700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 0x7f4d3006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.378 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.377+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f4d3c089130 con 0x7f4d440c3250 2024-03-14T09:03:27.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.377+0000 7f4d42ffd700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 0x7f4d3006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.377+0000 7f4d42ffd700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 0x7f4d3006cd20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f60 tx=0x7f4d38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.381+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4d3c054280 con 0x7f4d440c3250 2024-03-14T09:03:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.575+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d440697d0 con 0x7f4d440c3250 2024-03-14T09:03:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.576+0000 7f4d2bfff700 1 -- 172.21.15.154:0/3369194939 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7f4d3c0578a0 con 0x7f4d440c3250 2024-03-14T09:03:27.576 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:27.577 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 msgr2=0x7f4d3006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 0x7f4d3006cd20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f60 tx=0x7f4d38009040 comp rx=0 tx=0).stop 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 msgr2=0x7f4d44135a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c007e70 tx=0x7f4d3c00d3b0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 shutdown_connections 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4d3006a860 0x7f4d3006cd20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d440c3250 0x7f4d44135a50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d440a8540 0x7f4d441dd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 --2- 172.21.15.154:0/3369194939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d440c1780 0x7f4d44135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 >> 172.21.15.154:0/3369194939 conn(0x7f4d440c3880 msgr2=0x7f4d44139020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:27.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 shutdown_connections 2024-03-14T09:03:27.580 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.579+0000 7f4d4991a700 1 -- 172.21.15.154:0/3369194939 wait complete. 2024-03-14T09:03:27.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:57.920434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:58.934033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:03:11.026192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:03:11.722376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:27.592 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:27.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 Processor -- start 2024-03-14T09:03:27.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 -- start start 2024-03-14T09:03:27.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f31000a8540 0x7f310013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f3100154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310013ade0 con 0x7f31001550b0 2024-03-14T09:03:27.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310006fb30 con 0x7f31000a8540 2024-03-14T09:03:27.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f310658c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310006ec30 con 0x7f31000c3250 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.717+0000 7f3104b29700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f31001550b0 0x7f31000c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36324/0 (socket says 172.21.15.154:36324) 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f3104b29700 1 -- 172.21.15.154:0/4268325319 learned_addr learned my addr 172.21.15.154:0/4268325319 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30ff7fe700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f3100154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30fffff700 1 --2- 172.21.15.154:0/4268325319 >> v2:172.21.15.191:3300/0 conn(0x7f31000a8540 0x7f310013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959471990 0 0) 0x7f30f8001490 con 0x7f31001550b0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ec0035b0 con 0x7f31001550b0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49899299 0 0) 0x7f30ec0035b0 con 0x7f31001550b0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 msgr2=0x7f3100154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f3100154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 >> v2:172.21.15.191:3300/0 conn(0x7f31000a8540 msgr2=0x7f310013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 --2- 172.21.15.154:0/4268325319 >> v2:172.21.15.191:3300/0 conn(0x7f31000a8540 0x7f310013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310006ef10 con 0x7f31001550b0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f30f8001b80 con 0x7f31001550b0 2024-03-14T09:03:27.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 >> v1:172.21.15.154:6789/0 conn(0x7f31001550b0 legacy=0x7f31000c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f30ec003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ec003f80 0x7f30ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec004d90 0x7f30ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310006fb30 con 0x7f30ec004d90 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310006ec30 con 0x7f30ec003f80 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec00dd00 con 0x7f31000c3250 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f3104b29700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec004d90 0x7f30ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30ff7fe700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ec003f80 0x7f30ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f30fffff700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f30ec003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.718+0000 7f3104b29700 1 -- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 msgr2=0x7f30ec003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31000c3250 0x7f30ec003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 -- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ec003f80 msgr2=0x7f30ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ec003f80 0x7f30ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 -- 172.21.15.154:0/4268325319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0035b0 con 0x7f30ec004d90 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec004d90 0x7f30ec005210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f30f800a830 tx=0x7f30ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4900d88a890ec904 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800a4c0 con 0x7f30ec004d90 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f8012600 con 0x7f30ec004d90 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f30feffd700 1 -- 172.21.15.154:0/4268325319 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f80129b0 con 0x7f30ec004d90 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 -- 172.21.15.154:0/4268325319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec004d90 msgr2=0x7f30ec005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 --2- 172.21.15.154:0/4268325319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30ec004d90 0x7f30ec005210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f30f800a830 tx=0x7f30ec005880 comp rx=0 tx=0).stop 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f3104b29700 1 -- 172.21.15.154:0/4268325319 reap_dead start 2024-03-14T09:03:27.723 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 -- 172.21.15.154:0/4268325319 shutdown_connections 2024-03-14T09:03:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 -- 172.21.15.154:0/4268325319 >> 172.21.15.154:0/4268325319 conn(0x7f31000c3880 msgr2=0x7f31000c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 -- 172.21.15.154:0/4268325319 shutdown_connections 2024-03-14T09:03:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.719+0000 7f310658c700 1 -- 172.21.15.154:0/4268325319 wait complete. 2024-03-14T09:03:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 Processor -- start 2024-03-14T09:03:27.724 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 -- start start 2024-03-14T09:03:27.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3100153c10 0x7f31001daf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31001ee060 0x7f31001e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31001eb190 con 0x7f3100153ff0 2024-03-14T09:03:27.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31001dda40 con 0x7f3100153c10 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.721+0000 7f310658c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31000c1600 con 0x7f31001ee060 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35570/0 (socket says 172.21.15.154:35570) 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 -- 172.21.15.154:0/1351619804 learned_addr learned my addr 172.21.15.154:0/1351619804 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30fffff700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3100153c10 0x7f31001daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f3104b29700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31001ee060 0x7f31001e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 -- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31001ee060 msgr2=0x7f31001e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31001ee060 0x7f31001e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 -- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3100153c10 msgr2=0x7f31001daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3100153c10 0x7f31001daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 -- 172.21.15.154:0/1351619804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000c18e0 con 0x7f3100153ff0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30ff7fe700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30e80119a0 tx=0x7f30e8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800f940 con 0x7f3100153ff0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e800faa0 con 0x7f3100153ff0 2024-03-14T09:03:27.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31000c1bc0 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e800fe30 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.722+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31001edb40 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.723+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31001edd30 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.724+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f30e8014440 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.724+0000 7f30fcff9700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 0x7f30e006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.724+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f30e8089770 con 0x7f3100153ff0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.724+0000 7f30fffff700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 0x7f30e006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.725+0000 7f30fffff700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 0x7f30e006cd20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f31000c24c0 tx=0x7f30f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:27.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.728+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f30e8054970 con 0x7f3100153ff0 2024-03-14T09:03:27.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.922+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31000697d0 con 0x7f3100153ff0 2024-03-14T09:03:27.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.922+0000 7f30fcff9700 1 -- 172.21.15.154:0/1351619804 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30e8057f90 con 0x7f3100153ff0 2024-03-14T09:03:27.922 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 msgr2=0x7f30e006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 0x7f30e006cd20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f31000c24c0 tx=0x7f30f0008040 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 msgr2=0x7f31001eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f30e80119a0 tx=0x7f30e8011d60 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 shutdown_connections 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31001ee060 0x7f31001e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3100153c10 0x7f31001daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30e006a860 0x7f30e006cd20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 --2- 172.21.15.154:0/1351619804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3100153ff0 0x7f31001eac50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 >> 172.21.15.154:0/1351619804 conn(0x7f31000c3880 msgr2=0x7f310006e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 shutdown_connections 2024-03-14T09:03:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:27.924+0000 7f310658c700 1 -- 172.21.15.154:0/1351619804 wait complete. 2024-03-14T09:03:27.936 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:03:28.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.061+0000 7f820895b700 1 Processor -- start 2024-03-14T09:03:28.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.061+0000 7f820895b700 1 -- start start 2024-03-14T09:03:28.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820895b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040a8540 0x7f820413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820895b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82040c1780 0x7f82040c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820895b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820413ade0 con 0x7f82040c3250 2024-03-14T09:03:28.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820895b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820406fb30 con 0x7f82040c1780 2024-03-14T09:03:28.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820895b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820406ec30 con 0x7f82040a8540 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82040c1780 0x7f82040c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82040c1780 0x7f82040c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35584/0 (socket says 172.21.15.154:35584) 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 -- 172.21.15.154:0/2799738121 learned_addr learned my addr 172.21.15.154:0/2799738121 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f820259c700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040a8540 0x7f820413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 -- 172.21.15.154:0/2799738121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040a8540 msgr2=0x7f820413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040a8540 0x7f820413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 -- 172.21.15.154:0/2799738121 >> v1:172.21.15.191:6789/0 conn(0x7f82040c3250 legacy=0x7f8204154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.062+0000 7f8202d9d700 1 -- 172.21.15.154:0/2799738121 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820406ef10 con 0x7f82040c1780 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f8202d9d700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c1780 0x7f82040c1c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009e50 tx=0x7f81fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc17a83c7b8136 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820159a700 1 -- 172.21.15.154:0/2799738121 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00e070 con 0x7f82040c1780 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820159a700 1 -- 172.21.15.154:0/2799738121 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc003a20 con 0x7f82040c1780 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820159a700 1 -- 172.21.15.154:0/2799738121 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc003dd0 con 0x7f82040c1780 2024-03-14T09:03:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 -- 172.21.15.154:0/2799738121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c1780 msgr2=0x7f82040c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c1780 0x7f82040c1c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009e50 tx=0x7f81fc00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 -- 172.21.15.154:0/2799738121 shutdown_connections 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040a8540 0x7f820413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 --2- 172.21.15.154:0/2799738121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c1780 0x7f82040c1c10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 -- 172.21.15.154:0/2799738121 >> 172.21.15.154:0/2799738121 conn(0x7f82040c3880 msgr2=0x7f82040c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 -- 172.21.15.154:0/2799738121 shutdown_connections 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.063+0000 7f820895b700 1 -- 172.21.15.154:0/2799738121 wait complete. 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.065+0000 7f820895b700 1 Processor -- start 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.065+0000 7f820895b700 1 -- start start 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82040a8540 0x7f820414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040c1780 0x7f8204136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040c5920 con 0x7f82040c3250 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820412d600 con 0x7f82040a8540 2024-03-14T09:03:28.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820895b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82041ead10 con 0x7f82040c1780 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35596/0 (socket says 172.21.15.154:35596) 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 -- 172.21.15.154:0/3747116079 learned_addr learned my addr 172.21.15.154:0/3747116079 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f820259c700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82040a8540 0x7f820414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8201d9b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040c1780 0x7f8204136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 -- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040c1780 msgr2=0x7f8204136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040c1780 0x7f8204136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 -- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82040a8540 msgr2=0x7f820414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82040a8540 0x7f820414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 -- 172.21.15.154:0/3747116079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc009b30 con 0x7f82040c3250 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.066+0000 7f8202d9d700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f81fc015040 tx=0x7f81fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.067+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc004440 con 0x7f82040c3250 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.067+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc0045c0 con 0x7f82040c3250 2024-03-14T09:03:28.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.067+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82041eafb0 con 0x7f82040c3250 2024-03-14T09:03:28.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.067+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc004970 con 0x7f82040c3250 2024-03-14T09:03:28.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.067+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82040abfb0 con 0x7f82040c3250 2024-03-14T09:03:28.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.068+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82040697d0 con 0x7f82040c3250 2024-03-14T09:03:28.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.068+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f81fc025450 con 0x7f82040c3250 2024-03-14T09:03:28.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.068+0000 7f81f2ffd700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 0x7f81e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.069+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f81fc08a450 con 0x7f82040c3250 2024-03-14T09:03:28.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.069+0000 7f820259c700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 0x7f81e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.069+0000 7f820259c700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 0x7f81e406ccf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f81f8006b20 tx=0x7f81f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.073+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f81fc0556c0 con 0x7f82040c3250 2024-03-14T09:03:28.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.266+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f820413ade0 con 0x7f82040c3250 2024-03-14T09:03:28.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.266+0000 7f81f2ffd700 1 -- 172.21.15.154:0/3747116079 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f820413ade0 con 0x7f82040c3250 2024-03-14T09:03:28.266 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:28.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 msgr2=0x7f81e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 0x7f81e406ccf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f81f8006b20 tx=0x7f81f8008040 comp rx=0 tx=0).stop 2024-03-14T09:03:28.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 msgr2=0x7f82040ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f81fc015040 tx=0x7f81fc004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 shutdown_connections 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f81e406a830 0x7f81e406ccf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82040c3250 0x7f82040ac4d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82040a8540 0x7f820414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 --2- 172.21.15.154:0/3747116079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82040c1780 0x7f8204136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 >> 172.21.15.154:0/3747116079 conn(0x7f82040c3880 msgr2=0x7f8204152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 shutdown_connections 2024-03-14T09:03:28.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.268+0000 7f820895b700 1 -- 172.21.15.154:0/3747116079 wait complete. 2024-03-14T09:03:28.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:03:28.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 Processor -- start 2024-03-14T09:03:28.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 -- start start 2024-03-14T09:03:28.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 0x7f1b6814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1b680c5a00 0x7f1b68151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b68155a40 con 0x7f1b680c55a0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680c5e70 con 0x7f1b680c5a00 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.409+0000 7f1b6f0a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c2f10 con 0x7f1b680c26c0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1b680c5a00 0x7f1b68151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6ce3d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 0x7f1b6814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1b680c5a00 0x7f1b68151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:33880/0 (socket says 172.21.15.154:33880) 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b67fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1b680c55a0 0x7f1b681518c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:34204/0 (socket says 172.21.15.154:34204) 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 -- 172.21.15.154:0/4033229836 learned_addr learned my addr 172.21.15.154:0/4033229836 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b677fe700 1 -- 172.21.15.154:0/4033229836 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213028629 0 0) 0x7f1b68155a40 con 0x7f1b680c55a0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b677fe700 1 -- 172.21.15.154:0/4033229836 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b580035b0 con 0x7f1b680c55a0 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 -- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 msgr2=0x7f1b6814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 0x7f1b6814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 -- 172.21.15.154:0/4033229836 >> v1:172.21.15.191:6789/0 conn(0x7f1b680c55a0 legacy=0x7f1b681518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 -- 172.21.15.154:0/4033229836 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68072a90 con 0x7f1b680c5a00 2024-03-14T09:03:28.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6ce3d700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 0x7f1b6814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.410+0000 7f1b6d63e700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c5a00 0x7f1b68151fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009e50 tx=0x7f1b6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af05fb82a792d3f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b677fe700 1 -- 172.21.15.154:0/4033229836 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000e070 con 0x7f1b680c5a00 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b677fe700 1 -- 172.21.15.154:0/4033229836 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6000b910 con 0x7f1b680c5a00 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b677fe700 1 -- 172.21.15.154:0/4033229836 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000bc80 con 0x7f1b680c5a00 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c5a00 msgr2=0x7f1b68151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c5a00 0x7f1b68151fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009e50 tx=0x7f1b6000b330 comp rx=0 tx=0).stop 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/4033229836 shutdown_connections 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c5a00 0x7f1b68151fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/4033229836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c26c0 0x7f1b6814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/4033229836 >> 172.21.15.154:0/4033229836 conn(0x7f1b6806a770 msgr2=0x7f1b6806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/4033229836 shutdown_connections 2024-03-14T09:03:28.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.411+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/4033229836 wait complete. 2024-03-14T09:03:28.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.412+0000 7f1b6f0a1700 1 Processor -- start 2024-03-14T09:03:28.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.412+0000 7f1b6f0a1700 1 -- start start 2024-03-14T09:03:28.416 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 0x7f1b681def40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c5a00 0x7f1b68153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681507d0 con 0x7f1b680c55a0 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681ee560 con 0x7f1b680c26c0 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b6f0a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681edf50 con 0x7f1b680c5a00 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35606/0 (socket says 172.21.15.154:35606) 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.413+0000 7f1b67fff700 1 -- 172.21.15.154:0/3115107695 learned_addr learned my addr 172.21.15.154:0/3115107695 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b6d63e700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c5a00 0x7f1b68153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b6ce3d700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 0x7f1b681def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 -- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c5a00 msgr2=0x7f1b68153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c5a00 0x7f1b68153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 -- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 msgr2=0x7f1b681def40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 0x7f1b681def40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 -- 172.21.15.154:0/3115107695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0094a0 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b6ce3d700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 0x7f1b681def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b67fff700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400c530 tx=0x7f1b5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54005cb0 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54005e10 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b60009b30 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54007310 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.414+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b681ee1f0 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.415+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b681ee6f0 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.415+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1b54007470 con 0x7f1b680c55a0 2024-03-14T09:03:28.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.416+0000 7f1b657fa700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 0x7f1b4c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.419 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.416+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1b54088600 con 0x7f1b680c55a0 2024-03-14T09:03:28.419 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.416+0000 7f1b6ce3d700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 0x7f1b4c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.419 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.416+0000 7f1b6ce3d700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 0x7f1b4c06ccf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009cf0 tx=0x7f1b5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.420 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.420+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1b54053790 con 0x7f1b680c55a0 2024-03-14T09:03:28.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.611+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b681ee3b0 con 0x7f1b680c55a0 2024-03-14T09:03:28.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.611+0000 7f1b657fa700 1 -- 172.21.15.154:0/3115107695 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b54007720 con 0x7f1b680c55a0 2024-03-14T09:03:28.611 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:28.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.613+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 msgr2=0x7f1b4c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.613+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 0x7f1b4c06ccf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009cf0 tx=0x7f1b5c008040 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.613+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 msgr2=0x7f1b681ea840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.613+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400c530 tx=0x7f1b5400c8f0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 shutdown_connections 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1b680c5a00 0x7f1b68153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1b680c26c0 0x7f1b681def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1b4c06a830 0x7f1b4c06ccf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 --2- 172.21.15.154:0/3115107695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b680c55a0 0x7f1b681ea840 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 >> 172.21.15.154:0/3115107695 conn(0x7f1b6806a770 msgr2=0x7f1b680c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 shutdown_connections 2024-03-14T09:03:28.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.614+0000 7f1b6f0a1700 1 -- 172.21.15.154:0/3115107695 wait complete. 2024-03-14T09:03:28.625 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:03:28.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.749+0000 7fd2fea42700 1 Processor -- start 2024-03-14T09:03:28.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.749+0000 7fd2fea42700 1 -- start start 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fea42700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd2f814eaf0 0x7fd2f806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fea42700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2f806acb0 0x7fd2f8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fea42700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f80ac8e0 con 0x7fd2f8156330 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fea42700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f806b310 con 0x7fd2f806acb0 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fea42700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f806de30 con 0x7fd2f814eaf0 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2f806acb0 0x7fd2f8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fcfdf700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd2f8156330 0x7fd2f8157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36332/0 (socket says 172.21.15.154:36332) 2024-03-14T09:03:28.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2f806acb0 0x7fd2f8155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35622/0 (socket says 172.21.15.154:35622) 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2fcfdf700 1 -- 172.21.15.154:0/3147475338 learned_addr learned my addr 172.21.15.154:0/3147475338 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 -- 172.21.15.154:0/3147475338 >> v2:172.21.15.191:3300/0 conn(0x7fd2f814eaf0 msgr2=0x7fd2f806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 --2- 172.21.15.154:0/3147475338 >> v2:172.21.15.191:3300/0 conn(0x7fd2f814eaf0 0x7fd2f806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 -- 172.21.15.154:0/3147475338 >> v1:172.21.15.154:6789/0 conn(0x7fd2f8156330 legacy=0x7fd2f8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 -- 172.21.15.154:0/3147475338 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80acb30 con 0x7fd2f806acb0 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f6ffd700 1 -- 172.21.15.154:0/3147475338 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386778029 0 0) 0x7fd2f80ac8e0 con 0x7fd2f8156330 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f77fe700 1 --2- 172.21.15.154:0/3147475338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f806acb0 0x7fd2f8155df0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8070da0 tx=0x7fd2e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33de58e7c94210 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f6ffd700 1 -- 172.21.15.154:0/3147475338 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011830 con 0x7fd2f806acb0 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f6ffd700 1 -- 172.21.15.154:0/3147475338 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8011990 con 0x7fd2f806acb0 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.750+0000 7fd2f6ffd700 1 -- 172.21.15.154:0/3147475338 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011d20 con 0x7fd2f806acb0 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 -- 172.21.15.154:0/3147475338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f806acb0 msgr2=0x7fd2f8155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 --2- 172.21.15.154:0/3147475338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f806acb0 0x7fd2f8155df0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8070da0 tx=0x7fd2e8009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:28.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 -- 172.21.15.154:0/3147475338 shutdown_connections 2024-03-14T09:03:28.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 --2- 172.21.15.154:0/3147475338 >> v2:172.21.15.191:3300/0 conn(0x7fd2f814eaf0 0x7fd2f806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 --2- 172.21.15.154:0/3147475338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f806acb0 0x7fd2f8155df0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 -- 172.21.15.154:0/3147475338 >> 172.21.15.154:0/3147475338 conn(0x7fd2f80c3880 msgr2=0x7fd2f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 -- 172.21.15.154:0/3147475338 shutdown_connections 2024-03-14T09:03:28.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.751+0000 7fd2fea42700 1 -- 172.21.15.154:0/3147475338 wait complete. 2024-03-14T09:03:28.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.752+0000 7fd2fea42700 1 Processor -- start 2024-03-14T09:03:28.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.752+0000 7fd2fea42700 1 -- start start 2024-03-14T09:03:28.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2f806acb0 0x7fd2f8143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2f8156330 0x7fd2f8141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81ed730 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8071570 con 0x7fd2f806acb0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2fea42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81ed980 con 0x7fd2f8156330 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35624/0 (socket says 172.21.15.154:35624) 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 -- 172.21.15.154:0/690461504 learned_addr learned my addr 172.21.15.154:0/690461504 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 -- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2f8156330 msgr2=0x7fd2f8141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2f8156330 0x7fd2f8141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 -- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2f806acb0 msgr2=0x7fd2f8143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2f806acb0 0x7fd2f8143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 -- 172.21.15.154:0/690461504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80094a0 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.753+0000 7fd2f77fe700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.754+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80038a0 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.754+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e801e070 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.754+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003c60 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.754+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f81edc60 con 0x7fd2f814eaf0 2024-03-14T09:03:28.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.754+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f81ee220 con 0x7fd2f814eaf0 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.755+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f8156950 con 0x7fd2f814eaf0 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.755+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd2e802d450 con 0x7fd2f814eaf0 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.755+0000 7fd2f4ff9700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 0x7fd2e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.756+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd2e808a230 con 0x7fd2f814eaf0 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.756+0000 7fd2f7fff700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 0x7fd2e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:28.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.756+0000 7fd2f7fff700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 0x7fd2e406ccf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097b0 tx=0x7fd2ec006d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:28.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.760+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd2e8055380 con 0x7fd2f814eaf0 2024-03-14T09:03:28.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.955+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2f80697d0 con 0x7fd2f814eaf0 2024-03-14T09:03:28.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.955+0000 7fd2f4ff9700 1 -- 172.21.15.154:0/690461504 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v55) v1 ==== 105+0+11 (secure 0 0 0) 0x7fd2e80589a0 con 0x7fd2f814eaf0 2024-03-14T09:03:28.955 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:28.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 msgr2=0x7fd2e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 0x7fd2e406ccf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097b0 tx=0x7fd2ec006d20 comp rx=0 tx=0).stop 2024-03-14T09:03:28.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 msgr2=0x7fd2f8141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:28.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8014040 tx=0x7fd2e8004750 comp rx=0 tx=0).stop 2024-03-14T09:03:28.957 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 shutdown_connections 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2f8156330 0x7fd2f8141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2f806acb0 0x7fd2f8143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd2e406a830 0x7fd2e406ccf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 --2- 172.21.15.154:0/690461504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2f814eaf0 0x7fd2f8141960 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 >> 172.21.15.154:0/690461504 conn(0x7fd2f80c3880 msgr2=0x7fd2f8155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 shutdown_connections 2024-03-14T09:03:28.958 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:28.957+0000 7fd2fea42700 1 -- 172.21.15.154:0/690461504 wait complete. 2024-03-14T09:03:28.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:03:29.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.094+0000 7f2910e85700 1 Processor -- start 2024-03-14T09:03:29.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.094+0000 7f2910e85700 1 -- start start 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f2910e85700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f290c14fbd0 0x7f290c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f2910e85700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0acd10 con 0x7f290c0702f0 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f2910e85700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0ac9c0 con 0x7f290c0c4380 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f2910e85700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c0ac7b0 con 0x7f290c14fbd0 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f290c14fbd0 0x7f290c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f290c14fbd0 0x7f290c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35640/0 (socket says 172.21.15.154:35640) 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290a59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f290c0702f0 0x7f290c0c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36342/0 (socket says 172.21.15.154:36342) 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 -- 172.21.15.154:0/3284186967 learned_addr learned my addr 172.21.15.154:0/3284186967 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:29.095 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 -- 172.21.15.154:0/3284186967 >> v1:172.21.15.191:6789/0 conn(0x7f290c0c4380 legacy=0x7f290c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 -- 172.21.15.154:0/3284186967 >> v1:172.21.15.154:6789/0 conn(0x7f290c0702f0 legacy=0x7f290c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 -- 172.21.15.154:0/3284186967 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0707f0 con 0x7f290c14fbd0 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290ad9d700 1 --2- 172.21.15.154:0/3284186967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c14fbd0 0x7f290c15df60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2904009e50 tx=0x7f290400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc349fbe74f80a85 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290959a700 1 -- 172.21.15.154:0/3284186967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400e070 con 0x7f290c14fbd0 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290959a700 1 -- 172.21.15.154:0/3284186967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290400b9c0 con 0x7f290c14fbd0 2024-03-14T09:03:29.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.095+0000 7f290959a700 1 -- 172.21.15.154:0/3284186967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400bd70 con 0x7f290c14fbd0 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 -- 172.21.15.154:0/3284186967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c14fbd0 msgr2=0x7f290c15df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 --2- 172.21.15.154:0/3284186967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c14fbd0 0x7f290c15df60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2904009e50 tx=0x7f290400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 -- 172.21.15.154:0/3284186967 shutdown_connections 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 --2- 172.21.15.154:0/3284186967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c14fbd0 0x7f290c15df60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 -- 172.21.15.154:0/3284186967 >> 172.21.15.154:0/3284186967 conn(0x7f290c0a9bf0 msgr2=0x7f290c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 -- 172.21.15.154:0/3284186967 shutdown_connections 2024-03-14T09:03:29.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.096+0000 7f2910e85700 1 -- 172.21.15.154:0/3284186967 wait complete. 2024-03-14T09:03:29.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.097+0000 7f2910e85700 1 Processor -- start 2024-03-14T09:03:29.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.097+0000 7f2910e85700 1 -- start start 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c0c4380 0x7f290c0c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c14fbd0 0x7f290c142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1edaf0 con 0x7f290c0702f0 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1eddd0 con 0x7f290c0c4380 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2910e85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c1ee0b0 con 0x7f290c14fbd0 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35650/0 (socket says 172.21.15.154:35650) 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 -- 172.21.15.154:0/2227166916 learned_addr learned my addr 172.21.15.154:0/2227166916 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f2909d9b700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c0c4380 0x7f290c0c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290ad9d700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c14fbd0 0x7f290c142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 -- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c14fbd0 msgr2=0x7f290c142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c14fbd0 0x7f290c142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 -- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c0c4380 msgr2=0x7f290c0c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c0c4380 0x7f290c0c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 -- 172.21.15.154:0/2227166916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009b30 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.098+0000 7f290a59c700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f290000c520 tx=0x7f290000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.099+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900005cb0 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.099+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2900005e10 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.099+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c1ee390 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.099+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000072d0 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.099+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c1ed780 con 0x7f290c0702f0 2024-03-14T09:03:29.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.100+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c156b80 con 0x7f290c0702f0 2024-03-14T09:03:29.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.100+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2900007430 con 0x7f290c0702f0 2024-03-14T09:03:29.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.100+0000 7f28faffd700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 0x7f28ec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.101+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f29000883c0 con 0x7f290c0702f0 2024-03-14T09:03:29.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.101+0000 7f2909d9b700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 0x7f28ec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.101+0000 7f2909d9b700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 0x7f28ec06ccf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f290c1ee840 tx=0x7f28fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.105+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f29000534a0 con 0x7f290c0702f0 2024-03-14T09:03:29.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.296+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f290c1dca70 con 0x7f290c0702f0 2024-03-14T09:03:29.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.297+0000 7f28faffd700 1 -- 172.21.15.154:0/2227166916 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v55) v1 ==== 101+0+11 (secure 0 0 0) 0x7f29000170a0 con 0x7f290c0702f0 2024-03-14T09:03:29.297 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 msgr2=0x7f28ec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 0x7f28ec06ccf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f290c1ee840 tx=0x7f28fc009040 comp rx=0 tx=0).stop 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 msgr2=0x7f290c143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f290000c520 tx=0x7f290000c830 comp rx=0 tx=0).stop 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 shutdown_connections 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f290c14fbd0 0x7f290c142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f28ec06a830 0x7f28ec06ccf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f290c0702f0 0x7f290c143470 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 --2- 172.21.15.154:0/2227166916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c0c4380 0x7f290c0c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 >> 172.21.15.154:0/2227166916 conn(0x7f290c0a9bf0 msgr2=0x7f290c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:29.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 shutdown_connections 2024-03-14T09:03:29.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.299+0000 7f2910e85700 1 -- 172.21.15.154:0/2227166916 wait complete. 2024-03-14T09:03:29.311 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:03:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.433+0000 7fa0ccee6700 1 Processor -- start 2024-03-14T09:03:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 -- start start 2024-03-14T09:03:29.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0c814eaf0 0x7fa0c806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c806acb0 0x7fa0c8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c80ac8e0 con 0x7fa0c8156330 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c806b310 con 0x7fa0c814eaf0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.434+0000 7fa0ccee6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c806de30 con 0x7fa0c806acb0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0c814eaf0 0x7fa0c806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0c814eaf0 0x7fa0c806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53806/0 (socket says 172.21.15.154:53806) 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 -- 172.21.15.154:0/2375565791 learned_addr learned my addr 172.21.15.154:0/2375565791 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c5d9b700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c806acb0 0x7fa0c8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 -- 172.21.15.154:0/2375565791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c806acb0 msgr2=0x7fa0c8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c806acb0 0x7fa0c8155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 -- 172.21.15.154:0/2375565791 >> v1:172.21.15.191:6789/0 conn(0x7fa0c8156330 legacy=0x7fa0c8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 -- 172.21.15.154:0/2375565791 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80acb30 con 0x7fa0c814eaf0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c659c700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c814eaf0 0x7fa0c806a770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098f0 tx=0x7fa0bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c9bedb40fe9392 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c559a700 1 -- 172.21.15.154:0/2375565791 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00b8a0 con 0x7fa0c814eaf0 2024-03-14T09:03:29.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c559a700 1 -- 172.21.15.154:0/2375565791 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00ba00 con 0x7fa0c814eaf0 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.435+0000 7fa0c559a700 1 -- 172.21.15.154:0/2375565791 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00bd90 con 0x7fa0c814eaf0 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 -- 172.21.15.154:0/2375565791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c814eaf0 msgr2=0x7fa0c806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c814eaf0 0x7fa0c806a770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098f0 tx=0x7fa0bc009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 -- 172.21.15.154:0/2375565791 shutdown_connections 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c814eaf0 0x7fa0c806a770 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/2375565791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c806acb0 0x7fa0c8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 -- 172.21.15.154:0/2375565791 >> 172.21.15.154:0/2375565791 conn(0x7fa0c80c3880 msgr2=0x7fa0c80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 -- 172.21.15.154:0/2375565791 shutdown_connections 2024-03-14T09:03:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.436+0000 7fa0ccee6700 1 -- 172.21.15.154:0/2375565791 wait complete. 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.437+0000 7fa0ccee6700 1 Processor -- start 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.437+0000 7fa0ccee6700 1 -- start start 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c814eaf0 0x7fa0c81eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c8156330 0x7fa0c8153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81515e0 con 0x7fa0c806acb0 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8144e20 con 0x7fa0c814eaf0 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0ccee6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c81492d0 con 0x7fa0c8156330 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53810/0 (socket says 172.21.15.154:53810) 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 -- 172.21.15.154:0/795475258 learned_addr learned my addr 172.21.15.154:0/795475258 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c5d9b700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c814eaf0 0x7fa0c81eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c6d9d700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c8156330 0x7fa0c8153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 -- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c8156330 msgr2=0x7fa0c8153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c8156330 0x7fa0c8153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 -- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c814eaf0 msgr2=0x7fa0c81eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c814eaf0 0x7fa0c81eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 -- 172.21.15.154:0/795475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009550 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0c659c700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009c40 tx=0x7fa0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.438+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc004300 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.439+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc01e070 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.439+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0036a0 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.439+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c814a630 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.439+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81ea880 con 0x7fa0c806acb0 2024-03-14T09:03:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.439+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8067970 con 0x7fa0c806acb0 2024-03-14T09:03:29.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.440+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa0bc020030 con 0x7fa0c806acb0 2024-03-14T09:03:29.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.440+0000 7fa0b6ffd700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 0x7fa0a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.440+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa0bc089e10 con 0x7fa0c806acb0 2024-03-14T09:03:29.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.441+0000 7fa0c5d9b700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 0x7fa0a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.441+0000 7fa0c5d9b700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 0x7fa0a806ccf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80c6ac0 tx=0x7fa0b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.445+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa0bc08d030 con 0x7fa0c806acb0 2024-03-14T09:03:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.656+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0c8067dc0 con 0x7fa0c806acb0 2024-03-14T09:03:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.657+0000 7fa0b6ffd700 1 -- 172.21.15.154:0/795475258 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa0bc054e40 con 0x7fa0c806acb0 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 msgr2=0x7fa0a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 0x7fa0a806ccf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80c6ac0 tx=0x7fa0b8008040 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 msgr2=0x7fa0c8148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009c40 tx=0x7fa0bc004750 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 shutdown_connections 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c8156330 0x7fa0c8153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa0a806a830 0x7fa0a806ccf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c806acb0 0x7fa0c8148d90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 --2- 172.21.15.154:0/795475258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c814eaf0 0x7fa0c81eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 >> 172.21.15.154:0/795475258 conn(0x7fa0c80c3880 msgr2=0x7fa0c81574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 shutdown_connections 2024-03-14T09:03:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.659+0000 7fa0ccee6700 1 -- 172.21.15.154:0/795475258 wait complete. 2024-03-14T09:03:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:03:29.671 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T09:03:29.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:29.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:29.714 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:03:29.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:29.715 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T09:03:29.771 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:29.772 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:29.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:03:29.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:03:29.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 Processor -- start 2024-03-14T09:03:29.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 -- start start 2024-03-14T09:03:29.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41b814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80acd10 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80ac9c0 con 0x7f41b80c5d60 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.993+0000 7f41bdf5c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8150110 con 0x7f41b80c4380 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f41b80702f0 0x7f41b80c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54522/0 (socket says 172.21.15.154:54522) 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b77fe700 1 -- 172.21.15.154:0/3478935279 learned_addr learned my addr 172.21.15.154:0/3478935279 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b6ffd700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41b814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561046013 0 0) 0x7f41b80acd10 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a00035b0 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540980722 0 0) 0x7f41a00035b0 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 msgr2=0x7f41b814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41b814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 >> v1:172.21.15.191:6789/0 conn(0x7f41b80c5d60 legacy=0x7f41b815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80707f0 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f41a8001670 con 0x7f41b80702f0 2024-03-14T09:03:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 >> v1:172.21.15.154:6789/0 conn(0x7f41b80702f0 legacy=0x7f41b80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41a0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0003fb0 0x7f41a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a0004d90 0x7f41a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8150110 con 0x7f41a0004d90 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a000baf0 con 0x7f41a0003fb0 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a000bcb0 con 0x7f41b80c4380 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b7fff700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a0004d90 0x7f41a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b6ffd700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0003fb0 0x7f41a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.994+0000 7f41b77fe700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41a0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 -- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 msgr2=0x7f41a0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b80c4380 0x7f41a0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 -- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0003fb0 msgr2=0x7f41a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0003fb0 0x7f41a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 -- 172.21.15.154:0/3478935279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00035b0 con 0x7f41a0004d90 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a0004d90 0x7f41a0005210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f41b000c190 tx=0x7f41a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ebcae446ff13fc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0001db0 con 0x7f41a0004d90 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b000ccd0 con 0x7f41a0004d90 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b67fc700 1 -- 172.21.15.154:0/3478935279 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0003c40 con 0x7f41a0004d90 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 -- 172.21.15.154:0/3478935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a0004d90 msgr2=0x7f41a0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/3478935279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41a0004d90 0x7f41a0005210 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f41b000c190 tx=0x7f41a0005880 comp rx=0 tx=0).stop 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41b7fff700 1 -- 172.21.15.154:0/3478935279 reap_dead start 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 -- 172.21.15.154:0/3478935279 shutdown_connections 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 -- 172.21.15.154:0/3478935279 >> 172.21.15.154:0/3478935279 conn(0x7f41b80a9bf0 msgr2=0x7f41b80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 -- 172.21.15.154:0/3478935279 shutdown_connections 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.995+0000 7f41bdf5c700 1 -- 172.21.15.154:0/3478935279 wait complete. 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 Processor -- start 2024-03-14T09:03:29.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 -- start start 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 0x7f41b81eb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 0x7f41b8144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8150460 con 0x7f41b81532b0 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b814a010 con 0x7f41a0004d90 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.997+0000 7f41bdf5c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b81e7f40 con 0x7f41b81528e0 2024-03-14T09:03:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53824/0 (socket says 172.21.15.154:53824) 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 -- 172.21.15.154:0/2780513861 learned_addr learned my addr 172.21.15.154:0/2780513861 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b77fe700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 0x7f41b81eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b7fff700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 0x7f41b8144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 -- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 msgr2=0x7f41b8144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 0x7f41b8144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 -- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 msgr2=0x7f41b81eb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 0x7f41b81eb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 -- 172.21.15.154:0/2780513861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b81e8220 con 0x7f41b81532b0 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b7fff700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 0x7f41b8144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b77fe700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 0x7f41b81eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41b6ffd700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41ac011aa0 tx=0x7f41ac011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00f980 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac00fae0 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b81e8c80 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00fe90 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.998+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b81e59e0 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:29.999+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b81e8590 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.000+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f41ac014440 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.000+0000 7f419ffff700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 0x7f41a406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.000+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f41ac0898b0 con 0x7f41b81532b0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.000+0000 7f41b77fe700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 0x7f41a406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.001+0000 7f41b77fe700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 0x7f41a406ccf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41a8001eb0 tx=0x7f41a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:30.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.005+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f41ac054a30 con 0x7f41b81532b0 2024-03-14T09:03:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.239+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f41b81e83b0 con 0x7f41b81532b0 2024-03-14T09:03:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.239+0000 7f419ffff700 1 -- 172.21.15.154:0/2780513861 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+149 (secure 0 0 0) 0x7f41ac058050 con 0x7f41b81532b0 2024-03-14T09:03:30.239 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:03:30.239 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:03:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 msgr2=0x7f41a406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 0x7f41a406ccf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41a8001eb0 tx=0x7f41a8009040 comp rx=0 tx=0).stop 2024-03-14T09:03:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 msgr2=0x7f41b81e8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41ac011aa0 tx=0x7f41ac011e60 comp rx=0 tx=0).stop 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 shutdown_connections 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41b81528e0 0x7f41b8144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0004d90 0x7f41b81eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f41a406a830 0x7f41a406ccf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 --2- 172.21.15.154:0/2780513861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41b81532b0 0x7f41b81e8740 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 >> 172.21.15.154:0/2780513861 conn(0x7f41b80a9bf0 msgr2=0x7f41b814ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 shutdown_connections 2024-03-14T09:03:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:30.241+0000 7f41bdf5c700 1 -- 172.21.15.154:0/2780513861 wait complete. 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:30.253 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:03:30.254 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:03:30.254 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.DmznG5J3zd 2024-03-14T09:03:30.254 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:03:30.271 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.4PyMFYKYSt 2024-03-14T09:03:30.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /tmp/tmp.DmznG5J3zd /tmp/tmp.4PyMFYKYSt 2024-03-14T09:03:30.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.4PyMFYKYSt 2024-03-14T09:03:30.467 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.4PyMFYKYSt is 245B 2024-03-14T09:03:30.515 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.4PyMFYKYSt 2024-03-14T09:03:30.532 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA== caps mds = "allow rw fsname=cephfs" caps mon = "allow rw fsname=cephfs, allow rw fsname=cephfs2" caps osd = "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-03-14T09:03:30.532 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:03:30.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.532 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:03:30.589 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42249sec preferred_lft 42249sec 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:03:30.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:03:30.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.592 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:03:30.655 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:03:30.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.657 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:03:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:03:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:03:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:03:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:03:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:03:30.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:03:30.717 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:03:30.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.822 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:03:30.838 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:03:30.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:03:30.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:30.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:03:30.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:30.906 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:30.962 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:03:30.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:03:30.963 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:03:31.019 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:31.019 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/tmp/tmp.DmznG5J3zd of=/dev/stdout 2024-03-14T09:03:31.076 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-03-14T09:03:31.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:31.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:31.164 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:31.164 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:31.165 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:31.185 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:31.186 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:31.186 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:03:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:mount error 13 = Permission denied 2024-03-14T09:03:31.206 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:03:31.206 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:03:31.206 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:03:31.206 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:03:31.207 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,key=testuser,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:03:31.208 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-14T09:03:31.209 INFO:tasks.cephfs.kernel_mount:mount command failed 2024-03-14T09:03:31.209 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:03:31.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:03:31.256 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:03:31.304 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:03:31.304 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:31.304 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:31.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:03:31.322 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:31.322 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:31.343 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:31.343 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:31.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:31.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.525+0000 7fa369baf700 1 Processor -- start 2024-03-14T09:03:31.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.525+0000 7fa369baf700 1 -- start start 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa369baf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 0x7fa3640c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa369baf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa369baf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3640acd10 con 0x7fa3640c78c0 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa369baf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640ac7b0 con 0x7fa3640702f0 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa369baf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa364150050 con 0x7fa36414fbd0 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa362ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa3637fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 0x7fa3640c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa362ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:36268/0 (socket says 172.21.15.154:36268) 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.526+0000 7fa362ffd700 1 -- 172.21.15.154:0/2189195939 learned_addr learned my addr 172.21.15.154:0/2189195939 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa362ffd700 1 -- 172.21.15.154:0/2189195939 >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 msgr2=0x7fa3640c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa362ffd700 1 --2- 172.21.15.154:0/2189195939 >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 0x7fa3640c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa362ffd700 1 -- 172.21.15.154:0/2189195939 >> v1:172.21.15.191:6789/0 conn(0x7fa3640c78c0 legacy=0x7fa3640c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa362ffd700 1 -- 172.21.15.154:0/2189195939 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640ac9c0 con 0x7fa36414fbd0 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa3637fe700 1 --2- 172.21.15.154:0/2189195939 >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 0x7fa3640c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa362ffd700 1 --2- 172.21.15.154:0/2189195939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098f0 tx=0x7fa34c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a042980e672c7b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa3627fc700 1 -- 172.21.15.154:0/2189195939 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c011830 con 0x7fa36414fbd0 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa3627fc700 1 -- 172.21.15.154:0/2189195939 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c011990 con 0x7fa36414fbd0 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.527+0000 7fa3627fc700 1 -- 172.21.15.154:0/2189195939 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c011d20 con 0x7fa36414fbd0 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 -- 172.21.15.154:0/2189195939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 msgr2=0x7fa3640c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 --2- 172.21.15.154:0/2189195939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0098f0 tx=0x7fa34c009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 -- 172.21.15.154:0/2189195939 shutdown_connections 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 --2- 172.21.15.154:0/2189195939 >> v2:172.21.15.191:3300/0 conn(0x7fa3640702f0 0x7fa3640c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 --2- 172.21.15.154:0/2189195939 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa36414fbd0 0x7fa3640c7380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 -- 172.21.15.154:0/2189195939 >> 172.21.15.154:0/2189195939 conn(0x7fa3640a9bf0 msgr2=0x7fa3640a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 -- 172.21.15.154:0/2189195939 shutdown_connections 2024-03-14T09:03:31.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.528+0000 7fa369baf700 1 -- 172.21.15.154:0/2189195939 wait complete. 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.529+0000 7fa369baf700 1 Processor -- start 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.529+0000 7fa369baf700 1 -- start start 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3640702f0 0x7fa3641550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3640c78c0 0x7fa364152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641528e0 con 0x7fa36414fbd0 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641ec7b0 con 0x7fa3640c78c0 2024-03-14T09:03:31.532 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa364150c40 con 0x7fa3640702f0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53838/0 (socket says 172.21.15.154:53838) 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 -- 172.21.15.154:0/553361012 learned_addr learned my addr 172.21.15.154:0/553361012 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa3637fe700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3640702f0 0x7fa3641550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa362ffd700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3640c78c0 0x7fa364152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 -- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3640702f0 msgr2=0x7fa3641550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3640702f0 0x7fa3641550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 -- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3640c78c0 msgr2=0x7fa364152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3640c78c0 0x7fa364152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 -- 172.21.15.154:0/553361012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540094a0 con 0x7fa36414fbd0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa363fff700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00cb50 tx=0x7fa35c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c003c00 con 0x7fa36414fbd0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c003d60 con 0x7fa36414fbd0 2024-03-14T09:03:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00aa20 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.530+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c009550 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.531+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa364151200 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.531+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36414b8b0 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.532+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa35c00ab80 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.532+0000 7fa35bfff700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 0x7fa34406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.532+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fa35c089060 con 0x7fa36414fbd0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.532+0000 7fa3637fe700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 0x7fa34406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.534 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.533+0000 7fa3637fe700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 0x7fa34406ccf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa354009cd0 tx=0x7fa354009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.537+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa35c054130 con 0x7fa36414fbd0 2024-03-14T09:03:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.736+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa364152a20 con 0x7fa36414fbd0 2024-03-14T09:03:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.737+0000 7fa35bfff700 1 -- 172.21.15.154:0/553361012 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7fa35c057750 con 0x7fa36414fbd0 2024-03-14T09:03:31.737 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:31.738 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.739+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 msgr2=0x7fa34406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.739+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 0x7fa34406ccf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa354009cd0 tx=0x7fa354009210 comp rx=0 tx=0).stop 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.739+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 msgr2=0x7fa364144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.739+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00cb50 tx=0x7fa35c00ce60 comp rx=0 tx=0).stop 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 shutdown_connections 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa34406a830 0x7fa34406ccf0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa36414fbd0 0x7fa364144e20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa3640702f0 0x7fa3641550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 --2- 172.21.15.154:0/553361012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3640c78c0 0x7fa364152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 >> 172.21.15.154:0/553361012 conn(0x7fa3640a9bf0 msgr2=0x7fa3640a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 shutdown_connections 2024-03-14T09:03:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.740+0000 7fa369baf700 1 -- 172.21.15.154:0/553361012 wait complete. 2024-03-14T09:03:31.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:57.920434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:58.934033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:03:11.026192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:03:11.722376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:31.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:31.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.877+0000 7f7b8c3f4700 1 Processor -- start 2024-03-14T09:03:31.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.877+0000 7f7b8c3f4700 1 -- start start 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8c3f4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 0x7f7b8406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8c3f4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b8406acb0 0x7f7b84155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8c3f4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b840ac8e0 con 0x7f7b84156330 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8c3f4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8406b310 con 0x7f7b8406acb0 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8c3f4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8406de30 con 0x7f7b8414eaf0 2024-03-14T09:03:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b8406acb0 0x7f7b84155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b8406acb0 0x7f7b84155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53850/0 (socket says 172.21.15.154:53850) 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 -- 172.21.15.154:0/3253101680 learned_addr learned my addr 172.21.15.154:0/3253101680 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8a190700 1 --2- 172.21.15.154:0/3253101680 >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 0x7f7b8406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 -- 172.21.15.154:0/3253101680 >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 msgr2=0x7f7b8406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 --2- 172.21.15.154:0/3253101680 >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 0x7f7b8406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 -- 172.21.15.154:0/3253101680 >> v1:172.21.15.191:6789/0 conn(0x7f7b84156330 legacy=0x7f7b84157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 -- 172.21.15.154:0/3253101680 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840acb30 con 0x7f7b8406acb0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8a190700 1 --2- 172.21.15.154:0/3253101680 >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 0x7f7b8406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8998f700 1 --2- 172.21.15.154:0/3253101680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8406acb0 0x7f7b84155df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b84070da0 tx=0x7f7b74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb32695be8805c65 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8918e700 1 -- 172.21.15.154:0/3253101680 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74004230 con 0x7f7b8406acb0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8918e700 1 -- 172.21.15.154:0/3253101680 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74004390 con 0x7f7b8406acb0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.878+0000 7f7b8918e700 1 -- 172.21.15.154:0/3253101680 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740036a0 con 0x7f7b8406acb0 2024-03-14T09:03:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/3253101680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8406acb0 msgr2=0x7f7b84155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/3253101680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8406acb0 0x7f7b84155df0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7b84070da0 tx=0x7f7b74009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/3253101680 shutdown_connections 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/3253101680 >> v2:172.21.15.191:3300/0 conn(0x7f7b8414eaf0 0x7f7b8406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/3253101680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8406acb0 0x7f7b84155df0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/3253101680 >> 172.21.15.154:0/3253101680 conn(0x7f7b840c3880 msgr2=0x7f7b840c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/3253101680 shutdown_connections 2024-03-14T09:03:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.879+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/3253101680 wait complete. 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.880+0000 7f7b8c3f4700 1 Processor -- start 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.880+0000 7f7b8c3f4700 1 -- start start 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8406acb0 0x7f7b840c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b84156330 0x7f7b84140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8413eda0 con 0x7f7b8414eaf0 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841498b0 con 0x7f7b84156330 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8c3f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b841eaf30 con 0x7f7b8406acb0 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53856/0 (socket says 172.21.15.154:53856) 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 -- 172.21.15.154:0/1681024243 learned_addr learned my addr 172.21.15.154:0/1681024243 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8a190700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8406acb0 0x7f7b840c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8a991700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b84156330 0x7f7b84140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 -- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8406acb0 msgr2=0x7f7b840c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8406acb0 0x7f7b840c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 -- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b84156330 msgr2=0x7f7b84140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b84156330 0x7f7b84140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 -- 172.21.15.154:0/1681024243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74009550 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.881+0000 7f7b8998f700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7b840c43c0 tx=0x7f7b74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.882+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74003ea0 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.882+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74011630 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.882+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8413e750 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.882+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740119e0 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.882+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b841eb1d0 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.883+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84072460 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.883+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7b74011b40 con 0x7f7b8414eaf0 2024-03-14T09:03:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.883+0000 7f7b7affd700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 0x7f7b6806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.884+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f7b7408a610 con 0x7f7b8414eaf0 2024-03-14T09:03:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.884+0000 7f7b8a190700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 0x7f7b6806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.884+0000 7f7b8a190700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 0x7f7b6806ccf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:31.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:31.888+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7b74055760 con 0x7f7b8414eaf0 2024-03-14T09:03:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.083+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b841eb360 con 0x7f7b8414eaf0 2024-03-14T09:03:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.083+0000 7f7b7affd700 1 -- 172.21.15.154:0/1681024243 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b7405f0a0 con 0x7f7b8414eaf0 2024-03-14T09:03:32.083 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.085+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 msgr2=0x7f7b6806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.085+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 0x7f7b6806ccf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.085+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 msgr2=0x7f7b840c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.085+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7b840c43c0 tx=0x7f7b74004750 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 shutdown_connections 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b84156330 0x7f7b84140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7b8406acb0 0x7f7b840c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7b6806a830 0x7f7b6806ccf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 --2- 172.21.15.154:0/1681024243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b8414eaf0 0x7f7b840c6400 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 >> 172.21.15.154:0/1681024243 conn(0x7f7b840c3880 msgr2=0x7f7b84155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 shutdown_connections 2024-03-14T09:03:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.086+0000 7f7b8c3f4700 1 -- 172.21.15.154:0/1681024243 wait complete. 2024-03-14T09:03:32.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:03:32.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.224+0000 7f774a5fe700 1 Processor -- start 2024-03-14T09:03:32.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.224+0000 7f774a5fe700 1 -- start start 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f774a5fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440a8540 0x7f774413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f774a5fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77440c3250 0x7f7744154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f774a5fe700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774413ade0 con 0x7f77441550b0 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f774a5fe700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774406fb30 con 0x7f77440c3250 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f774a5fe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774406ec30 con 0x7f77440a8540 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77440c3250 0x7f7744154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77440c3250 0x7f7744154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53864/0 (socket says 172.21.15.154:53864) 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 -- 172.21.15.154:0/188325155 learned_addr learned my addr 172.21.15.154:0/188325155 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f7743fff700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440a8540 0x7f774413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 -- 172.21.15.154:0/188325155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440a8540 msgr2=0x7f774413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440a8540 0x7f774413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 -- 172.21.15.154:0/188325155 >> v1:172.21.15.191:6789/0 conn(0x7f77441550b0 legacy=0x7f77440c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.225+0000 7f77437fe700 1 -- 172.21.15.154:0/188325155 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774406ef10 con 0x7f77440c3250 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f77437fe700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440c3250 0x7f7744154b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77380098f0 tx=0x7f7738009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1151ae37cdf767 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f7742ffd700 1 -- 172.21.15.154:0/188325155 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738011830 con 0x7f77440c3250 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f7742ffd700 1 -- 172.21.15.154:0/188325155 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7738011990 con 0x7f77440c3250 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f7742ffd700 1 -- 172.21.15.154:0/188325155 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738011d20 con 0x7f77440c3250 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 -- 172.21.15.154:0/188325155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440c3250 msgr2=0x7f7744154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440c3250 0x7f7744154b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f77380098f0 tx=0x7f7738009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 -- 172.21.15.154:0/188325155 shutdown_connections 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440a8540 0x7f774413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 --2- 172.21.15.154:0/188325155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440c3250 0x7f7744154b70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 -- 172.21.15.154:0/188325155 >> 172.21.15.154:0/188325155 conn(0x7f77440c3880 msgr2=0x7f77440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 -- 172.21.15.154:0/188325155 shutdown_connections 2024-03-14T09:03:32.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.226+0000 7f774a5fe700 1 -- 172.21.15.154:0/188325155 wait complete. 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.227+0000 7f774a5fe700 1 Processor -- start 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 -- start start 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440c3250 0x7f77440c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 0x7f77441dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77441edb10 con 0x7f77440a8540 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77441ead60 con 0x7f77441550b0 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f774a5fe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77441ea800 con 0x7f77440c3250 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f7743fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f7743fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53872/0 (socket says 172.21.15.154:53872) 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.228+0000 7f7743fff700 1 -- 172.21.15.154:0/1954197094 learned_addr learned my addr 172.21.15.154:0/1954197094 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f77437fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440c3250 0x7f77440c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7748b9b700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 0x7f77441dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 -- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440c3250 msgr2=0x7f77440c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440c3250 0x7f77440c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 -- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 msgr2=0x7f77441dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 0x7f77441dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 -- 172.21.15.154:0/1954197094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009550 con 0x7f77440a8540 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7748b9b700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 0x7f77441dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7743fff700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f773400e9e0 tx=0x7f773400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734009710 con 0x7f77440a8540 2024-03-14T09:03:32.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773400c850 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77441eaa80 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400cbc0 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.229+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77441ed730 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.230+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77441edd80 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.230+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f773400cd20 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.231+0000 7f7740ff9700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 0x7f773006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.231+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f7734089510 con 0x7f77440a8540 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.231+0000 7f77437fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 0x7f773006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.231+0000 7f77437fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 0x7f773006ccf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7738009c40 tx=0x7f7738008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.235+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7734054660 con 0x7f77440a8540 2024-03-14T09:03:32.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.431+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77440697d0 con 0x7f77440a8540 2024-03-14T09:03:32.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.431+0000 7f7740ff9700 1 -- 172.21.15.154:0/1954197094 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7734057c80 con 0x7f77440a8540 2024-03-14T09:03:32.431 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:32.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 msgr2=0x7f773006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 0x7f773006ccf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7738009c40 tx=0x7f7738008030 comp rx=0 tx=0).stop 2024-03-14T09:03:32.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 msgr2=0x7f77441edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f773400e9e0 tx=0x7f773400eda0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 shutdown_connections 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77441550b0 0x7f77441dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f773006a830 0x7f773006ccf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77440a8540 0x7f77441edf30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 --2- 172.21.15.154:0/1954197094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77440c3250 0x7f77440c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 >> 172.21.15.154:0/1954197094 conn(0x7f77440c3880 msgr2=0x7f7744138780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 shutdown_connections 2024-03-14T09:03:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.433+0000 7f774a5fe700 1 -- 172.21.15.154:0/1954197094 wait complete. 2024-03-14T09:03:32.445 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:03:32.570 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 Processor -- start 2024-03-14T09:03:32.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 -- start start 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f62ec0c4a70 0x7f62ec0c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0c7550 con 0x7f62ec0c6b00 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec06c140 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62f136e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec14b780 con 0x7f62ec0c4a70 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62eaffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f62ec0c6b00 0x7f62ec155190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54536/0 (socket says 172.21.15.154:54536) 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.571+0000 7f62eaffd700 1 -- 172.21.15.154:0/469321213 learned_addr learned my addr 172.21.15.154:0/469321213 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62eb7fe700 1 --2- 172.21.15.154:0/469321213 >> v2:172.21.15.191:3300/0 conn(0x7f62ec0c4a70 0x7f62ec0c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62e9ffb700 1 -- 172.21.15.154:0/469321213 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010169719 0 0) 0x7f62ec0c7550 con 0x7f62ec0c6b00 2024-03-14T09:03:32.573 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62e9ffb700 1 -- 172.21.15.154:0/469321213 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d80035b0 con 0x7f62ec0c6b00 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62eb7fe700 1 -- 172.21.15.154:0/469321213 >> v1:172.21.15.191:6789/0 conn(0x7f62ec0c7ff0 legacy=0x7f62ec06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62eb7fe700 1 -- 172.21.15.154:0/469321213 >> v1:172.21.15.154:6789/0 conn(0x7f62ec0c6b00 legacy=0x7f62ec155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62eb7fe700 1 -- 172.21.15.154:0/469321213 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0c8450 con 0x7f62ec0c4a70 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62eb7fe700 1 --2- 172.21.15.154:0/469321213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c4eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009d90 tx=0x7f62e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439ffb2be9c72909 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62e9ffb700 1 -- 172.21.15.154:0/469321213 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400e070 con 0x7f62ec0c4a70 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62e9ffb700 1 -- 172.21.15.154:0/469321213 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e400bb20 con 0x7f62ec0c4a70 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.572+0000 7f62e9ffb700 1 -- 172.21.15.154:0/469321213 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4012430 con 0x7f62ec0c4a70 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 -- 172.21.15.154:0/469321213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 msgr2=0x7f62ec0c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 --2- 172.21.15.154:0/469321213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c4eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009d90 tx=0x7f62e400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 -- 172.21.15.154:0/469321213 shutdown_connections 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 --2- 172.21.15.154:0/469321213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c4eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 -- 172.21.15.154:0/469321213 >> 172.21.15.154:0/469321213 conn(0x7f62ec0abc80 msgr2=0x7f62ec0ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.574 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 -- 172.21.15.154:0/469321213 shutdown_connections 2024-03-14T09:03:32.575 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.573+0000 7f62f136e700 1 -- 172.21.15.154:0/469321213 wait complete. 2024-03-14T09:03:32.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.574+0000 7f62f136e700 1 Processor -- start 2024-03-14T09:03:32.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.574+0000 7f62f136e700 1 -- start start 2024-03-14T09:03:32.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.577 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62ec1543d0 0x7f62ec1edab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0c2cb0 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1e9fa0 con 0x7f62ec0c4a70 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62f136e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0c3e70 con 0x7f62ec1543d0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53878/0 (socket says 172.21.15.154:53878) 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 -- 172.21.15.154:0/1983202341 learned_addr learned my addr 172.21.15.154:0/1983202341 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62eaffd700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62eb7fe700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62ec1543d0 0x7f62ec1edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 -- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62ec1543d0 msgr2=0x7f62ec1edab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62ec1543d0 0x7f62ec1edab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 -- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 msgr2=0x7f62ec0c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 -- 172.21.15.154:0/1983202341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4009a70 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.578 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.575+0000 7f62ea7fc700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f62d400c4e0 tx=0x7f62d400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40052d0 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4005430 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1ea260 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40038c0 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1ea820 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.576+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec067bd0 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.577+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f62d4003a20 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.577+0000 7f62e37fe700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 0x7f62cc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.577+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f62d4088450 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.578+0000 7f62eaffd700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 0x7f62cc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.579 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.578+0000 7f62eaffd700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 0x7f62cc06ccf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f62dc007fd0 tx=0x7f62dc007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.582+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f62d4053520 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.774+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62ec0697d0 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.775+0000 7f62e37fe700 1 -- 172.21.15.154:0/1983202341 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62d4056b40 con 0x7f62ec0c7ff0 2024-03-14T09:03:32.775 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 msgr2=0x7f62cc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 0x7f62cc06ccf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f62dc007fd0 tx=0x7f62dc007dc0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 msgr2=0x7f62ec1ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f62d400c4e0 tx=0x7f62d400c7f0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 shutdown_connections 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62ec1543d0 0x7f62ec1edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62ec0c4a70 0x7f62ec0c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62cc06a830 0x7f62cc06ccf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 --2- 172.21.15.154:0/1983202341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62ec0c7ff0 0x7f62ec1ed570 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 >> 172.21.15.154:0/1983202341 conn(0x7f62ec0abc80 msgr2=0x7f62ec0ab3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 shutdown_connections 2024-03-14T09:03:32.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.777+0000 7f62f136e700 1 -- 172.21.15.154:0/1983202341 wait complete. 2024-03-14T09:03:32.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:03:32.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.914+0000 7fb22c2a8700 1 Processor -- start 2024-03-14T09:03:32.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.914+0000 7fb22c2a8700 1 -- start start 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22c2a8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb22415e280 0x7fb224160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22c2a8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240ac8e0 con 0x7fb22414eaf0 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22c2a8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22406de30 con 0x7fb224155df0 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22c2a8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22406b310 con 0x7fb22415e280 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a845700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb22415e280 0x7fb224160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a044700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb22414eaf0 0x7fb22406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54552/0 (socket says 172.21.15.154:54552) 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a044700 1 -- 172.21.15.154:0/50113559 learned_addr learned my addr 172.21.15.154:0/50113559 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a845700 1 -- 172.21.15.154:0/50113559 >> v1:172.21.15.191:6789/0 conn(0x7fb224155df0 legacy=0x7fb224156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a845700 1 -- 172.21.15.154:0/50113559 >> v1:172.21.15.154:6789/0 conn(0x7fb22414eaf0 legacy=0x7fb22406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a845700 1 -- 172.21.15.154:0/50113559 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224156960 con 0x7fb22415e280 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb229042700 1 -- 172.21.15.154:0/50113559 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843420268 0 0) 0x7fb2240ac8e0 con 0x7fb22414eaf0 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb22a845700 1 --2- 172.21.15.154:0/50113559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb22415e280 0x7fb224160670 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009d90 tx=0x7fb21c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29181bc73766ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb229042700 1 -- 172.21.15.154:0/50113559 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00e070 con 0x7fb22415e280 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb229042700 1 -- 172.21.15.154:0/50113559 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00bbd0 con 0x7fb22415e280 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.915+0000 7fb229042700 1 -- 172.21.15.154:0/50113559 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c012430 con 0x7fb22415e280 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 -- 172.21.15.154:0/50113559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb22415e280 msgr2=0x7fb224160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/50113559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb22415e280 0x7fb224160670 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009d90 tx=0x7fb21c00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 -- 172.21.15.154:0/50113559 shutdown_connections 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/50113559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb22415e280 0x7fb224160670 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 -- 172.21.15.154:0/50113559 >> 172.21.15.154:0/50113559 conn(0x7fb2240c3880 msgr2=0x7fb2240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:32.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 -- 172.21.15.154:0/50113559 shutdown_connections 2024-03-14T09:03:32.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.916+0000 7fb22c2a8700 1 -- 172.21.15.154:0/50113559 wait complete. 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.917+0000 7fb22c2a8700 1 Processor -- start 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 -- start start 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22414eaf0 0x7fb2241ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb22415e280 0x7fb2241eae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241eec90 con 0x7fb224155df0 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241ea970 con 0x7fb22414eaf0 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.918+0000 7fb22c2a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c2b10 con 0x7fb22415e280 2024-03-14T09:03:32.920 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53908/0 (socket says 172.21.15.154:53908) 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 -- 172.21.15.154:0/2874367331 learned_addr learned my addr 172.21.15.154:0/2874367331 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb22a845700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb22415e280 0x7fb2241eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb22a044700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22414eaf0 0x7fb2241ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 -- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb22415e280 msgr2=0x7fb2241eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb22415e280 0x7fb2241eae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 -- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22414eaf0 msgr2=0x7fb2241ee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22414eaf0 0x7fb2241ee750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 -- 172.21.15.154:0/2874367331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009a70 con 0x7fb224155df0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb229843700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c4e0 tx=0x7fb21400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140052d0 con 0x7fb224155df0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb214005430 con 0x7fb224155df0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2240c2df0 con 0x7fb224155df0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140038c0 con 0x7fb224155df0 2024-03-14T09:03:32.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.919+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240c33b0 con 0x7fb224155df0 2024-03-14T09:03:32.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.920+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224067dc0 con 0x7fb224155df0 2024-03-14T09:03:32.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.921+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb214003a20 con 0x7fb224155df0 2024-03-14T09:03:32.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.921+0000 7fb21affd700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 0x7fb20806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:32.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.921+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb214088580 con 0x7fb224155df0 2024-03-14T09:03:32.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.921+0000 7fb22a044700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 0x7fb20806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:32.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.922+0000 7fb22a044700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 0x7fb20806ccf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb220000f60 tx=0x7fb220009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:32.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:32.925+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb2140536d0 con 0x7fb224155df0 2024-03-14T09:03:33.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.117+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2240697d0 con 0x7fb224155df0 2024-03-14T09:03:33.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.118+0000 7fb21affd700 1 -- 172.21.15.154:0/2874367331 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v55) v1 ==== 105+0+11 (secure 0 0 0) 0x7fb214056cf0 con 0x7fb224155df0 2024-03-14T09:03:33.117 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.119+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 msgr2=0x7fb20806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.119+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 0x7fb20806ccf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb220000f60 tx=0x7fb220009040 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.119+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 msgr2=0x7fb224149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.119+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c4e0 tx=0x7fb21400c8a0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 shutdown_connections 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb22415e280 0x7fb2241eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22414eaf0 0x7fb2241ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb20806a830 0x7fb20806ccf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 --2- 172.21.15.154:0/2874367331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb224155df0 0x7fb224149ee0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 >> 172.21.15.154:0/2874367331 conn(0x7fb2240c3880 msgr2=0x7fb224154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 shutdown_connections 2024-03-14T09:03:33.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.120+0000 7fb22c2a8700 1 -- 172.21.15.154:0/2874367331 wait complete. 2024-03-14T09:03:33.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:03:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 Processor -- start 2024-03-14T09:03:33.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 -- start start 2024-03-14T09:03:33.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 0x7f5c8006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800c5f10 con 0x7f5c8006d600 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c800c7d50 con 0x7f5c80070a10 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.257+0000 7f5c85e8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800c8030 con 0x7f5c8006c6e0 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5c8006d600 0x7f5c800c6ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54554/0 (socket says 172.21.15.154:54554) 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7ffff700 1 -- 172.21.15.154:0/3017711513 learned_addr learned my addr 172.21.15.154:0/3017711513 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7f7fe700 1 --2- 172.21.15.154:0/3017711513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 0x7f5c8006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271307474 0 0) 0x7f5c800c5f10 con 0x7f5c8006d600 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c6c0035b0 con 0x7f5c8006d600 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221424377 0 0) 0x7f5c800c7d50 con 0x7f5c80070a10 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c800c5f10 con 0x7f5c80070a10 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7f7fe700 1 -- 172.21.15.154:0/3017711513 >> v1:172.21.15.191:6789/0 conn(0x7f5c80070a10 legacy=0x7f5c800c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7f7fe700 1 -- 172.21.15.154:0/3017711513 >> v1:172.21.15.154:6789/0 conn(0x7f5c8006d600 legacy=0x7f5c800c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7f7fe700 1 -- 172.21.15.154:0/3017711513 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800c8310 con 0x7f5c8006c6e0 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7f7fe700 1 --2- 172.21.15.154:0/3017711513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 0x7f5c8006d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009870 tx=0x7f5c70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec99e31b23cc8b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011830 con 0x7f5c8006c6e0 2024-03-14T09:03:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70011990 con 0x7f5c8006c6e0 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.258+0000 7f5c7e7fc700 1 -- 172.21.15.154:0/3017711513 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011d00 con 0x7f5c8006c6e0 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3017711513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 msgr2=0x7f5c8006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3017711513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 0x7f5c8006d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009870 tx=0x7f5c70009b80 comp rx=0 tx=0).stop 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3017711513 shutdown_connections 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3017711513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006c6e0 0x7f5c8006d0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3017711513 >> 172.21.15.154:0/3017711513 conn(0x7f5c800a9bf0 msgr2=0x7f5c800a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3017711513 shutdown_connections 2024-03-14T09:03:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.259+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3017711513 wait complete. 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.260+0000 7f5c85e8a700 1 Processor -- start 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.260+0000 7f5c85e8a700 1 -- start start 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c8006c6e0 0x7f5c801dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006d600 0x7f5c801da840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800abcc0 con 0x7f5c80070a10 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800ac4d0 con 0x7f5c8006c6e0 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c85e8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800abf10 con 0x7f5c8006d600 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53920/0 (socket says 172.21.15.154:53920) 2024-03-14T09:03:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 -- 172.21.15.154:0/3780360764 learned_addr learned my addr 172.21.15.154:0/3780360764 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7f7fe700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c8006c6e0 0x7f5c801dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7effd700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006d600 0x7f5c801da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 -- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006d600 msgr2=0x7f5c801da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006d600 0x7f5c801da840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 -- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c8006c6e0 msgr2=0x7f5c801dc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.261+0000 7f5c7ffff700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c8006c6e0 0x7f5c801dc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c7ffff700 1 -- 172.21.15.154:0/3780360764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700094a0 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c7ffff700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800cba0 tx=0x7f5c7800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800aa40 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7800aba0 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c800ac1f0 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78012400 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.262+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c800aafa0 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.263+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8009aad0 con 0x7f5c80070a10 2024-03-14T09:03:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.263+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5c7801b3b0 con 0x7f5c80070a10 2024-03-14T09:03:33.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.264+0000 7f5c77fff700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 0x7f5c6006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.264+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f5c78088ce0 con 0x7f5c80070a10 2024-03-14T09:03:33.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.264+0000 7f5c7f7fe700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 0x7f5c6006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.264+0000 7f5c7f7fe700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 0x7f5c6006ccf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c700090b0 tx=0x7f5c70008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.268+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5c78053e30 con 0x7f5c80070a10 2024-03-14T09:03:33.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.464+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f5c800697d0 con 0x7f5c80070a10 2024-03-14T09:03:33.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.464+0000 7f5c77fff700 1 -- 172.21.15.154:0/3780360764 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v55) v1 ==== 101+0+11 (secure 0 0 0) 0x7f5c780170a0 con 0x7f5c80070a10 2024-03-14T09:03:33.464 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:33.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 msgr2=0x7f5c6006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 0x7f5c6006ccf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5c700090b0 tx=0x7f5c70008030 comp rx=0 tx=0).stop 2024-03-14T09:03:33.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 msgr2=0x7f5c800a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800cba0 tx=0x7f5c7800ceb0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 shutdown_connections 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5c6006a830 0x7f5c6006ccf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c80070a10 0x7f5c800a66d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c8006c6e0 0x7f5c801dc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 --2- 172.21.15.154:0/3780360764 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c8006d600 0x7f5c801da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 >> 172.21.15.154:0/3780360764 conn(0x7f5c800a9bf0 msgr2=0x7f5c800a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 shutdown_connections 2024-03-14T09:03:33.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.466+0000 7f5c85e8a700 1 -- 172.21.15.154:0/3780360764 wait complete. 2024-03-14T09:03:33.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:03:33.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.601+0000 7f1bd7e03700 1 Processor -- start 2024-03-14T09:03:33.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 -- start start 2024-03-14T09:03:33.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1bd00c4380 0x7f1bd014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bd00c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd00acd10 con 0x7f1bd00702f0 2024-03-14T09:03:33.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd0150110 con 0x7f1bd00c4380 2024-03-14T09:03:33.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd7e03700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00ac7b0 con 0x7f1bd00c5d60 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd5b9f700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1bd00702f0 0x7f1bd00c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54568/0 (socket says 172.21.15.154:54568) 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.602+0000 7f1bd5b9f700 1 -- 172.21.15.154:0/11586740 learned_addr learned my addr 172.21.15.154:0/11586740 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 --2- 172.21.15.154:0/11586740 >> v2:172.21.15.191:3300/0 conn(0x7f1bd00c4380 0x7f1bd014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bd00c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239671272 0 0) 0x7f1bd00acd10 con 0x7f1bd00702f0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bbc0035b0 con 0x7f1bd00702f0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298480803 0 0) 0x7f1bbc0035b0 con 0x7f1bd00702f0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 msgr2=0x7f1bd00c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bd00c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 >> v2:172.21.15.191:3300/0 conn(0x7f1bd00c4380 msgr2=0x7f1bd014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 --2- 172.21.15.154:0/11586740 >> v2:172.21.15.191:3300/0 conn(0x7f1bd00c4380 0x7f1bd014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd00ac9c0 con 0x7f1bd00702f0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f1bc0001670 con 0x7f1bd00702f0 2024-03-14T09:03:33.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 >> v1:172.21.15.154:6789/0 conn(0x7f1bd00702f0 legacy=0x7f1bd00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc003a00 0x7f1bbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bbc004850 0x7f1bbc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bbc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0150110 con 0x7f1bbc004850 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00ac7b0 con 0x7f1bbc003a00 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc00ba10 con 0x7f1bd00c5d60 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bbc004850 0x7f1bbc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd5b9f700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc003a00 0x7f1bbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bbc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 -- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 msgr2=0x7f1bbc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bbc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 -- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc003a00 msgr2=0x7f1bbc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc003a00 0x7f1bbc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd539e700 1 -- 172.21.15.154:0/11586740 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0035b0 con 0x7f1bbc004850 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.603+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd00c5d60 0x7f1bbc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd539e700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bbc004850 0x7f1bbc004cd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009b60 tx=0x7f1bbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93df22cdedb7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00e6c0 con 0x7f1bbc004850 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc00e820 con 0x7f1bbc004850 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd4b9d700 1 -- 172.21.15.154:0/11586740 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00eb90 con 0x7f1bbc004850 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 -- 172.21.15.154:0/11586740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bbc004850 msgr2=0x7f1bbc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/11586740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bbc004850 0x7f1bbc004cd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009b60 tx=0x7f1bbc005880 comp rx=0 tx=0).stop 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd63a0700 1 -- 172.21.15.154:0/11586740 reap_dead start 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 -- 172.21.15.154:0/11586740 shutdown_connections 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 -- 172.21.15.154:0/11586740 >> 172.21.15.154:0/11586740 conn(0x7f1bd00a9bf0 msgr2=0x7f1bd00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 -- 172.21.15.154:0/11586740 shutdown_connections 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.604+0000 7f1bd7e03700 1 -- 172.21.15.154:0/11586740 wait complete. 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.605+0000 7f1bd7e03700 1 Processor -- start 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.605+0000 7f1bd7e03700 1 -- start start 2024-03-14T09:03:33.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 0x7f1bd014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd0147360 0x7f1bd01eb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0150cd0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd014ac70 con 0x7f1bbc004850 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd7e03700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0152900 con 0x7f1bd0147360 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd63a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd63a0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53932/0 (socket says 172.21.15.154:53932) 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd63a0700 1 -- 172.21.15.154:0/1883897530 learned_addr learned my addr 172.21.15.154:0/1883897530 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd539e700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd0147360 0x7f1bd01eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.606+0000 7f1bd5b9f700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 0x7f1bd014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 -- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd0147360 msgr2=0x7f1bd01eb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd0147360 0x7f1bd01eb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 -- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 msgr2=0x7f1bd014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 0x7f1bd014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 -- 172.21.15.154:0/1883897530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd01ec2e0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd5b9f700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 0x7f1bd014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd63a0700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0153700 tx=0x7f1bc80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800e050 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc800f040 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd0152ba0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8010e50 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.607+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd0153360 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.608+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd0152d90 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.608+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1bc8015440 con 0x7f1bd01e81c0 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.609+0000 7f1bc67fc700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 0x7f1bb406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:33.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.609+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1bc80891a0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.609+0000 7f1bd5b9f700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 0x7f1bb406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:33.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.609+0000 7f1bd5b9f700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 0x7f1bb406ccf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00015f0 tx=0x7f1bc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:33.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.613+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1bc80542f0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.823+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1bd00697d0 con 0x7f1bd01e81c0 2024-03-14T09:03:33.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.824+0000 7f1bc67fc700 1 -- 172.21.15.154:0/1883897530 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1bc8057910 con 0x7f1bd01e81c0 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 msgr2=0x7f1bb406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 0x7f1bb406ccf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00015f0 tx=0x7f1bc0009040 comp rx=0 tx=0).stop 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 msgr2=0x7f1bd01e8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0153700 tx=0x7f1bc80104a0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 shutdown_connections 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1bb406a830 0x7f1bb406ccf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bd01e81c0 0x7f1bd01e8670 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bbc004850 0x7f1bd014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 --2- 172.21.15.154:0/1883897530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1bd0147360 0x7f1bd01eb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:33.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 >> 172.21.15.154:0/1883897530 conn(0x7f1bd00a9bf0 msgr2=0x7f1bd00c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:33.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 shutdown_connections 2024-03-14T09:03:33.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:33.826+0000 7f1bd7e03700 1 -- 172.21.15.154:0/1883897530 wait complete. 2024-03-14T09:03:33.827 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:03:33.837 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:03:33.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:33.838 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:03:33.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:03:33.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:03:33.882 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:03:33.943 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:34.066 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 Processor -- start 2024-03-14T09:03:34.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 -- start start 2024-03-14T09:03:34.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f30e80a8540 0x7f30e813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80c3250 0x7f30e8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30e81550b0 0x7f30e80c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e813ade0 con 0x7f30e81550b0 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e806fb30 con 0x7f30e80a8540 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.067+0000 7f30edec7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e806ec30 con 0x7f30e80c3250 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30e81550b0 0x7f30e80c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30e81550b0 0x7f30e80c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53934/0 (socket says 172.21.15.154:53934) 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 -- 172.21.15.154:0/4247864246 learned_addr learned my addr 172.21.15.154:0/4247864246 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e6ffd700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80c3250 0x7f30e8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e77fe700 1 --2- 172.21.15.154:0/4247864246 >> v2:172.21.15.191:3300/0 conn(0x7f30e80a8540 0x7f30e813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 -- 172.21.15.154:0/4247864246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80c3250 msgr2=0x7f30e8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80c3250 0x7f30e8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 -- 172.21.15.154:0/4247864246 >> v2:172.21.15.191:3300/0 conn(0x7f30e80a8540 msgr2=0x7f30e813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 --2- 172.21.15.154:0/4247864246 >> v2:172.21.15.191:3300/0 conn(0x7f30e80a8540 0x7f30e813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 -- 172.21.15.154:0/4247864246 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e806ef10 con 0x7f30e81550b0 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e7fff700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e81550b0 0x7f30e80c1780 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f30e8070da0 tx=0x7f30e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f29473381920da server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e67fc700 1 -- 172.21.15.154:0/4247864246 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e000e070 con 0x7f30e81550b0 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e67fc700 1 -- 172.21.15.154:0/4247864246 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e000b9c0 con 0x7f30e81550b0 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.068+0000 7f30e67fc700 1 -- 172.21.15.154:0/4247864246 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e000bd70 con 0x7f30e81550b0 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 -- 172.21.15.154:0/4247864246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e81550b0 msgr2=0x7f30e80c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e81550b0 0x7f30e80c1780 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f30e8070da0 tx=0x7f30e000b330 comp rx=0 tx=0).stop 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 -- 172.21.15.154:0/4247864246 shutdown_connections 2024-03-14T09:03:34.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e81550b0 0x7f30e80c1780 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 --2- 172.21.15.154:0/4247864246 >> v2:172.21.15.191:3300/0 conn(0x7f30e80a8540 0x7f30e813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 --2- 172.21.15.154:0/4247864246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80c3250 0x7f30e8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 -- 172.21.15.154:0/4247864246 >> 172.21.15.154:0/4247864246 conn(0x7f30e80c3880 msgr2=0x7f30e80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 -- 172.21.15.154:0/4247864246 shutdown_connections 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.069+0000 7f30edec7700 1 -- 172.21.15.154:0/4247864246 wait complete. 2024-03-14T09:03:34.072 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.070+0000 7f30edec7700 1 Processor -- start 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.070+0000 7f30edec7700 1 -- start start 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80a8540 0x7f30e80a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e81550b0 0x7f30e8133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e814f3f0 con 0x7f30e80c3250 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81edcf0 con 0x7f30e81550b0 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30edec7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81edfd0 con 0x7f30e80a8540 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53942/0 (socket says 172.21.15.154:53942) 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 -- 172.21.15.154:0/321922264 learned_addr learned my addr 172.21.15.154:0/321922264 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e77fe700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80a8540 0x7f30e80a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e7fff700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e81550b0 0x7f30e8133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 -- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80a8540 msgr2=0x7f30e80a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80a8540 0x7f30e80a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 -- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e81550b0 msgr2=0x7f30e8133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e81550b0 0x7f30e8133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 -- 172.21.15.154:0/321922264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e0009b30 con 0x7f30e80c3250 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30e6ffd700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e810 tx=0x7f30d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.071+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000c070 con 0x7f30e80c3250 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.072+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0013610 con 0x7f30e80c3250 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.072+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e80ace60 con 0x7f30e80c3250 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.072+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00139c0 con 0x7f30e80c3250 2024-03-14T09:03:34.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.072+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e814f640 con 0x7f30e80c3250 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.072+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e80697d0 con 0x7f30e80c3250 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.073+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f30d0013b20 con 0x7f30e80c3250 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.073+0000 7f30dffff700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 0x7f30c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.073+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f30d0088d70 con 0x7f30e80c3250 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.074+0000 7f30e77fe700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 0x7f30c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.074+0000 7f30e77fe700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 0x7f30c806ccf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006b20 tx=0x7f30d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.078+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f30d0053e40 con 0x7f30e80c3250 2024-03-14T09:03:34.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.313+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30e806d420 con 0x7f30e80c3250 2024-03-14T09:03:34.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.313+0000 7f30dffff700 1 -- 172.21.15.154:0/321922264 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5956 (secure 0 0 0) 0x7f30d0057460 con 0x7f30e80c3250 2024-03-14T09:03:34.313 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:34.313 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 msgr2=0x7f30c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 0x7f30c806ccf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006b20 tx=0x7f30d8008040 comp rx=0 tx=0).stop 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 msgr2=0x7f30e80c22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e810 tx=0x7f30d000eb20 comp rx=0 tx=0).stop 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 shutdown_connections 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30e81550b0 0x7f30e8133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30e80a8540 0x7f30e80a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30c806a830 0x7f30c806ccf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 --2- 172.21.15.154:0/321922264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30e80c3250 0x7f30e80c22f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 >> 172.21.15.154:0/321922264 conn(0x7f30e80c3880 msgr2=0x7f30e81396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:34.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 shutdown_connections 2024-03-14T09:03:34.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.316+0000 7f30edec7700 1 -- 172.21.15.154:0/321922264 wait complete. 2024-03-14T09:03:34.317 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 35 2024-03-14T09:03:34.328 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:34.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.450+0000 7f27c3a68700 1 Processor -- start 2024-03-14T09:03:34.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.450+0000 7f27c3a68700 1 -- start start 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f27bc14eaf0 0x7f27bc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27bc06acb0 0x7f27bc155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc156330 0x7f27bc157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27bc06b310 con 0x7f27bc06acb0 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27bc0ac8e0 con 0x7f27bc14eaf0 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.451+0000 7f27c3a68700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc06de30 con 0x7f27bc156330 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27bc06acb0 0x7f27bc155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27bc06acb0 0x7f27bc155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53948/0 (socket says 172.21.15.154:53948) 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 -- 172.21.15.154:0/252043148 learned_addr learned my addr 172.21.15.154:0/252043148 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1804700 1 --2- 172.21.15.154:0/252043148 >> v2:172.21.15.191:3300/0 conn(0x7f27bc14eaf0 0x7f27bc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c2005700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc156330 0x7f27bc157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 -- 172.21.15.154:0/252043148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc156330 msgr2=0x7f27bc157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc156330 0x7f27bc157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 -- 172.21.15.154:0/252043148 >> v2:172.21.15.191:3300/0 conn(0x7f27bc14eaf0 msgr2=0x7f27bc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 --2- 172.21.15.154:0/252043148 >> v2:172.21.15.191:3300/0 conn(0x7f27bc14eaf0 0x7f27bc06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 -- 172.21.15.154:0/252043148 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc0acb30 con 0x7f27bc06acb0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c1003700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc155df0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f27bc070da0 tx=0x7f27b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d234adb7b822c74c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c0802700 1 -- 172.21.15.154:0/252043148 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0011830 con 0x7f27bc06acb0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c0802700 1 -- 172.21.15.154:0/252043148 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b0011990 con 0x7f27bc06acb0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.452+0000 7f27c0802700 1 -- 172.21.15.154:0/252043148 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0011d20 con 0x7f27bc06acb0 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 -- 172.21.15.154:0/252043148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 msgr2=0x7f27bc155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc155df0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f27bc070da0 tx=0x7f27b0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 -- 172.21.15.154:0/252043148 shutdown_connections 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc156330 0x7f27bc157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 --2- 172.21.15.154:0/252043148 >> v2:172.21.15.191:3300/0 conn(0x7f27bc14eaf0 0x7f27bc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 --2- 172.21.15.154:0/252043148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc155df0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 -- 172.21.15.154:0/252043148 >> 172.21.15.154:0/252043148 conn(0x7f27bc0c3880 msgr2=0x7f27bc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 -- 172.21.15.154:0/252043148 shutdown_connections 2024-03-14T09:03:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.453+0000 7f27c3a68700 1 -- 172.21.15.154:0/252043148 wait complete. 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.454+0000 7f27c3a68700 1 Processor -- start 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.454+0000 7f27c3a68700 1 -- start start 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc14eaf0 0x7f27bc141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27bc156330 0x7f27bc153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc154610 con 0x7f27bc06acb0 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc152730 con 0x7f27bc156330 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c3a68700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc152cf0 con 0x7f27bc14eaf0 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53962/0 (socket says 172.21.15.154:53962) 2024-03-14T09:03:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 -- 172.21.15.154:0/1064158083 learned_addr learned my addr 172.21.15.154:0/1064158083 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c2005700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27bc156330 0x7f27bc153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1003700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc14eaf0 0x7f27bc141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 -- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc14eaf0 msgr2=0x7f27bc141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc14eaf0 0x7f27bc141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 -- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27bc156330 msgr2=0x7f27bc153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27bc156330 0x7f27bc153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.455+0000 7f27c1804700 1 -- 172.21.15.154:0/1064158083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009550 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27c1804700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f27b4007e60 tx=0x7f27b400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400ed30 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b400ee90 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc151a20 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4014660 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.456+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc1eb060 con 0x7f27bc06acb0 2024-03-14T09:03:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.457+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc072200 con 0x7f27bc06acb0 2024-03-14T09:03:34.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.457+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f27b40147c0 con 0x7f27bc06acb0 2024-03-14T09:03:34.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.457+0000 7f27ba7fc700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 0x7f27a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.458+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f27b4089670 con 0x7f27bc06acb0 2024-03-14T09:03:34.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.458+0000 7f27c1003700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 0x7f27a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.458+0000 7f27c1003700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 0x7f27a806ccf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27b0014040 tx=0x7f27b0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.462+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f27b40547c0 con 0x7f27bc06acb0 2024-03-14T09:03:34.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.658+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27bc0697d0 con 0x7f27bc06acb0 2024-03-14T09:03:34.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.660+0000 7f27ba7fc700 1 -- 172.21.15.154:0/1064158083 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7f27b4009a20 con 0x7f27bc06acb0 2024-03-14T09:03:34.660 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:34.660 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 msgr2=0x7f27a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 0x7f27a806ccf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27b0014040 tx=0x7f27b0008030 comp rx=0 tx=0).stop 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 msgr2=0x7f27bc153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f27b4007e60 tx=0x7f27b400e3c0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 shutdown_connections 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27bc156330 0x7f27bc153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f27a806a830 0x7f27a806ccf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27bc06acb0 0x7f27bc153180 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 --2- 172.21.15.154:0/1064158083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f27bc14eaf0 0x7f27bc141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 >> 172.21.15.154:0/1064158083 conn(0x7f27bc0c3880 msgr2=0x7f27bc14c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:34.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 shutdown_connections 2024-03-14T09:03:34.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.662+0000 7f27c3a68700 1 -- 172.21.15.154:0/1064158083 wait complete. 2024-03-14T09:03:34.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:02:57.920434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:02:58.934033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:03:11.026192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:03:11.722376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:34.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:34.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 Processor -- start 2024-03-14T09:03:34.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 -- start start 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fced8070a10 0x7fced80c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced80c31c0 0x7fced80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced80c5f10 con 0x7fced806c6e0 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced80c7d50 con 0x7fced8070a10 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.800+0000 7fcede465700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80c8030 con 0x7fced80c31c0 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fced8070a10 0x7fced80c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced7fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fced806c6e0 0x7fced806d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:51204/0 (socket says 172.21.15.154:51204) 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced7fff700 1 -- 172.21.15.154:0/235439117 learned_addr learned my addr 172.21.15.154:0/235439117 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fcedca02700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced80c31c0 0x7fced80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 -- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced80c31c0 msgr2=0x7fced80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced80c31c0 0x7fced80c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 -- 172.21.15.154:0/235439117 >> v1:172.21.15.191:6789/0 conn(0x7fced806c6e0 legacy=0x7fced806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 -- 172.21.15.154:0/235439117 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80c8310 con 0x7fced8070a10 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced77fe700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced8070a10 0x7fced80c63a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009840 tx=0x7fcecc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faafd843f28322ef server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.801+0000 7fced6ffd700 1 -- 172.21.15.154:0/235439117 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc011830 con 0x7fced8070a10 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fced6ffd700 1 -- 172.21.15.154:0/235439117 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc011990 con 0x7fced8070a10 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fced6ffd700 1 -- 172.21.15.154:0/235439117 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc011d20 con 0x7fced8070a10 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 -- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced8070a10 msgr2=0x7fced80c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced8070a10 0x7fced80c63a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009840 tx=0x7fcecc009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 -- 172.21.15.154:0/235439117 shutdown_connections 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced80c31c0 0x7fced80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 --2- 172.21.15.154:0/235439117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced8070a10 0x7fced80c63a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 -- 172.21.15.154:0/235439117 >> 172.21.15.154:0/235439117 conn(0x7fced80a9bf0 msgr2=0x7fced80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 -- 172.21.15.154:0/235439117 shutdown_connections 2024-03-14T09:03:34.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.802+0000 7fcede465700 1 -- 172.21.15.154:0/235439117 wait complete. 2024-03-14T09:03:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.803+0000 7fcede465700 1 Processor -- start 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.803+0000 7fcede465700 1 -- start start 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806c6e0 0x7fced81dc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 0x7fced81ea750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81e9dc0 con 0x7fced8070a10 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8099c80 con 0x7fced80c31c0 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fcede465700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced80acbe0 con 0x7fced806c6e0 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fced77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fced77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53968/0 (socket says 172.21.15.154:53968) 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.804+0000 7fced77fe700 1 -- 172.21.15.154:0/36241802 learned_addr learned my addr 172.21.15.154:0/36241802 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced7fff700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806c6e0 0x7fced81dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fcedca02700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 0x7fced81ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 -- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806c6e0 msgr2=0x7fced81dc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806c6e0 0x7fced81dc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 -- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 msgr2=0x7fced81ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 0x7fced81ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 -- 172.21.15.154:0/36241802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0094a0 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fcedca02700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 0x7fced81ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced77fe700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0090b0 tx=0x7fcecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0036a0 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc01f070 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc003a60 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced81ee0b0 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.805+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced80ad110 con 0x7fced8070a10 2024-03-14T09:03:34.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.806+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced80ad2a0 con 0x7fced8070a10 2024-03-14T09:03:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.806+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcecc003bc0 con 0x7fced8070a10 2024-03-14T09:03:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.807+0000 7fced4ff9700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 0x7fcec406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.807+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fcecc021070 con 0x7fced8070a10 2024-03-14T09:03:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.807+0000 7fced7fff700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 0x7fcec406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.807+0000 7fced7fff700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 0x7fcec406ccf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcec8005910 tx=0x7fcec8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:34.811+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcecc055960 con 0x7fced8070a10 2024-03-14T09:03:35.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.004+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fced81e9f00 con 0x7fced8070a10 2024-03-14T09:03:35.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.005+0000 7fced4ff9700 1 -- 172.21.15.154:0/36241802 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcecc058f80 con 0x7fced8070a10 2024-03-14T09:03:35.004 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 msgr2=0x7fcec406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 0x7fcec406ccf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcec8005910 tx=0x7fcec8009040 comp rx=0 tx=0).stop 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 msgr2=0x7fced81ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0090b0 tx=0x7fcecc004750 comp rx=0 tx=0).stop 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 shutdown_connections 2024-03-14T09:03:35.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80c31c0 0x7fced81ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced806c6e0 0x7fced81dc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcec406a830 0x7fcec406ccf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 --2- 172.21.15.154:0/36241802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fced8070a10 0x7fced81ea210 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 >> 172.21.15.154:0/36241802 conn(0x7fced80a9bf0 msgr2=0x7fced80c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 shutdown_connections 2024-03-14T09:03:35.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.007+0000 7fcede465700 1 -- 172.21.15.154:0/36241802 wait complete. 2024-03-14T09:03:35.019 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:03:35.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.144+0000 7fd86142f700 1 Processor -- start 2024-03-14T09:03:35.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.144+0000 7fd86142f700 1 -- start start 2024-03-14T09:03:35.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.144+0000 7fd86142f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd85c0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.144+0000 7fd86142f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd85c0c5150 0x7fd85c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd86142f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0c6a20 con 0x7fd85c155350 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd86142f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0707b0 con 0x7fd85c0c5150 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd86142f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0c6460 con 0x7fd85c0c87e0 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85b7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd85c155350 0x7fd85c0c3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54572/0 (socket says 172.21.15.154:54572) 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2089073224 learned_addr learned my addr 172.21.15.154:0/2089073224 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85affd700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd85c0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85a7fc700 1 --2- 172.21.15.154:0/2089073224 >> v2:172.21.15.191:3300/0 conn(0x7fd85c0c5150 0x7fd85c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373117980 0 0) 0x7fd854001490 con 0x7fd85c155350 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8440035b0 con 0x7fd85c155350 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2721691679 0 0) 0x7fd8440035b0 con 0x7fd85c155350 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 msgr2=0x7fd85c0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd85c0c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 >> v2:172.21.15.191:3300/0 conn(0x7fd85c0c5150 msgr2=0x7fd85c154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 --2- 172.21.15.154:0/2089073224 >> v2:172.21.15.191:3300/0 conn(0x7fd85c0c5150 0x7fd85c154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0c6740 con 0x7fd85c155350 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fd854001b80 con 0x7fd85c155350 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 >> v1:172.21.15.154:6789/0 conn(0x7fd85c155350 legacy=0x7fd85c0c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844003a00 0x7fd844003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd844004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844004d90 0x7fd844005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0707b0 con 0x7fd844003a00 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0c6460 con 0x7fd844004d90 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84400ba10 con 0x7fd85c0c87e0 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85affd700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844003a00 0x7fd844003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.145+0000 7fd85a7fc700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd844004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85b7fe700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844004d90 0x7fd844005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 -- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 msgr2=0x7fd844004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c0c87e0 0x7fd844004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 -- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844004d90 msgr2=0x7fd844005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844004d90 0x7fd844005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 -- 172.21.15.154:0/2089073224 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440035b0 con 0x7fd844003a00 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85affd700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844003a00 0x7fd844003e20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009b60 tx=0x7fd844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbcad78855024ab server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00e6c0 con 0x7fd844003a00 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00e820 con 0x7fd844003a00 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd859ffb700 1 -- 172.21.15.154:0/2089073224 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00eb90 con 0x7fd844003a00 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd86142f700 1 -- 172.21.15.154:0/2089073224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844003a00 msgr2=0x7fd844003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd86142f700 1 --2- 172.21.15.154:0/2089073224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844003a00 0x7fd844003e20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009b60 tx=0x7fd844005880 comp rx=0 tx=0).stop 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2089073224 reap_dead start 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.146+0000 7fd86142f700 1 -- 172.21.15.154:0/2089073224 shutdown_connections 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.147+0000 7fd86142f700 1 -- 172.21.15.154:0/2089073224 >> 172.21.15.154:0/2089073224 conn(0x7fd85c06e4a0 msgr2=0x7fd85c06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.147+0000 7fd86142f700 1 -- 172.21.15.154:0/2089073224 shutdown_connections 2024-03-14T09:03:35.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.147+0000 7fd86142f700 1 -- 172.21.15.154:0/2089073224 wait complete. 2024-03-14T09:03:35.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 Processor -- start 2024-03-14T09:03:35.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 -- start start 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 0x7fd85c0a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 0x7fd85c1e1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1496a0 con 0x7fd85c149ac0 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14a8e0 con 0x7fd844003a00 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.148+0000 7fd86142f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14b100 con 0x7fd85c1e1ba0 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53986/0 (socket says 172.21.15.154:53986) 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2668722718 learned_addr learned my addr 172.21.15.154:0/2668722718 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85a7fc700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 0x7fd85c1e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85affd700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 0x7fd85c0a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 msgr2=0x7fd85c1e1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 0x7fd85c1e1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 msgr2=0x7fd85c0a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 0x7fd85c0a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 -- 172.21.15.154:0/2668722718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c14ae20 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85a7fc700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 0x7fd85c1e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85affd700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 0x7fd85c0a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd85b7fe700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0c55d0 tx=0x7fd854007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85400ece0 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85400ee60 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c1492c0 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854013420 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.149+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0c3570 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.150+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c1497e0 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.151+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd854013580 con 0x7fd85c149ac0 2024-03-14T09:03:35.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.151+0000 7fd8437fe700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 0x7fd84806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.151+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd854089110 con 0x7fd85c149ac0 2024-03-14T09:03:35.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.151+0000 7fd85affd700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 0x7fd84806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.152+0000 7fd85affd700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 0x7fd84806ccf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0090b0 tx=0x7fd84c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.156 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.155+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd85c1497e0 con 0x7fd85c149ac0 2024-03-14T09:03:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.348+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd85c0c6a20 con 0x7fd85c149ac0 2024-03-14T09:03:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.349+0000 7fd8437fe700 1 -- 172.21.15.154:0/2668722718 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd85c0c6a20 con 0x7fd85c149ac0 2024-03-14T09:03:35.349 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 msgr2=0x7fd84806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 0x7fd84806ccf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0090b0 tx=0x7fd84c008030 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 msgr2=0x7fd85c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0c55d0 tx=0x7fd854007c90 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 shutdown_connections 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd84806a830 0x7fd84806ccf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd85c149ac0 0x7fd85c149f50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd844003a00 0x7fd85c0a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 --2- 172.21.15.154:0/2668722718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd85c1e1ba0 0x7fd85c1e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 >> 172.21.15.154:0/2668722718 conn(0x7fd85c06e4a0 msgr2=0x7fd85c0c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 shutdown_connections 2024-03-14T09:03:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.351+0000 7fd86142f700 1 -- 172.21.15.154:0/2668722718 wait complete. 2024-03-14T09:03:35.363 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:03:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.487+0000 7fcd64903700 1 Processor -- start 2024-03-14T09:03:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.487+0000 7fcd64903700 1 -- start start 2024-03-14T09:03:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd64903700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd60155df0 0x7fcd60156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd64903700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 0x7fcd6015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd64903700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd600ac8e0 con 0x7fcd6014eaf0 2024-03-14T09:03:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd64903700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6006b310 con 0x7fcd60155df0 2024-03-14T09:03:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd64903700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6006de30 con 0x7fcd6015d270 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd60155df0 0x7fcd60156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5e59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcd6014eaf0 0x7fcd6006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54582/0 (socket says 172.21.15.154:54582) 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd60155df0 0x7fcd60156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54000/0 (socket says 172.21.15.154:54000) 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5e59c700 1 -- 172.21.15.154:0/1779137392 learned_addr learned my addr 172.21.15.154:0/1779137392 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5ed9d700 1 --2- 172.21.15.154:0/1779137392 >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 0x7fcd6015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 -- 172.21.15.154:0/1779137392 >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 msgr2=0x7fcd6015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 --2- 172.21.15.154:0/1779137392 >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 0x7fcd6015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 -- 172.21.15.154:0/1779137392 >> v1:172.21.15.154:6789/0 conn(0x7fcd6014eaf0 legacy=0x7fcd6006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5dd9b700 1 -- 172.21.15.154:0/1779137392 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600acb30 con 0x7fcd60155df0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5d59a700 1 -- 172.21.15.154:0/1779137392 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509560332 0 0) 0x7fcd600ac8e0 con 0x7fcd6014eaf0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.488+0000 7fcd5ed9d700 1 --2- 172.21.15.154:0/1779137392 >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 0x7fcd6015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd5dd9b700 1 --2- 172.21.15.154:0/1779137392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd60155df0 0x7fcd60156250 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd600c4920 tx=0x7fcd50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaf5636b9d082bb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd5d59a700 1 -- 172.21.15.154:0/1779137392 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50011830 con 0x7fcd60155df0 2024-03-14T09:03:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd5d59a700 1 -- 172.21.15.154:0/1779137392 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd50011990 con 0x7fcd60155df0 2024-03-14T09:03:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd5d59a700 1 -- 172.21.15.154:0/1779137392 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50011d20 con 0x7fcd60155df0 2024-03-14T09:03:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 -- 172.21.15.154:0/1779137392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd60155df0 msgr2=0x7fcd60156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 --2- 172.21.15.154:0/1779137392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd60155df0 0x7fcd60156250 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcd600c4920 tx=0x7fcd50009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 -- 172.21.15.154:0/1779137392 shutdown_connections 2024-03-14T09:03:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 --2- 172.21.15.154:0/1779137392 >> v2:172.21.15.191:3300/0 conn(0x7fcd6015d270 0x7fcd6015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 --2- 172.21.15.154:0/1779137392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd60155df0 0x7fcd60156250 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 -- 172.21.15.154:0/1779137392 >> 172.21.15.154:0/1779137392 conn(0x7fcd600c3880 msgr2=0x7fcd600c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 -- 172.21.15.154:0/1779137392 shutdown_connections 2024-03-14T09:03:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.489+0000 7fcd64903700 1 -- 172.21.15.154:0/1779137392 wait complete. 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.491+0000 7fcd64903700 1 Processor -- start 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.491+0000 7fcd64903700 1 -- start start 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6014eaf0 0x7fcd601eae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd60155df0 0x7fcd600c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6015ae20 con 0x7fcd6015d270 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd601eb3a0 con 0x7fcd6014eaf0 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd64903700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600c33f0 con 0x7fcd60155df0 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54016/0 (socket says 172.21.15.154:54016) 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 -- 172.21.15.154:0/3292875235 learned_addr learned my addr 172.21.15.154:0/3292875235 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5e59c700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6014eaf0 0x7fcd601eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 -- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd60155df0 msgr2=0x7fcd600c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd60155df0 0x7fcd600c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 -- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6014eaf0 msgr2=0x7fcd601eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6014eaf0 0x7fcd601eae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 -- 172.21.15.154:0/3292875235 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500094a0 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd5ed9d700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcd58007df0 tx=0x7fcd5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800ddd0 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.492+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5800f040 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.493+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6015bd50 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.493+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58003c60 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.493+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd600c1d80 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.494+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd600697d0 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.494+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fcd58004030 con 0x7fcd6015d270 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.494+0000 7fcd4effd700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 0x7fcd4006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.495+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fcd580892a0 con 0x7fcd6015d270 2024-03-14T09:03:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.495+0000 7fcd5e59c700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 0x7fcd4006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.495+0000 7fcd5e59c700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 0x7fcd4006ccf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcd54000f60 tx=0x7fcd54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.499+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fcd580543f0 con 0x7fcd6015d270 2024-03-14T09:03:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.694+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd6015d050 con 0x7fcd6015d270 2024-03-14T09:03:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.694+0000 7fcd4effd700 1 -- 172.21.15.154:0/3292875235 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd58057a10 con 0x7fcd6015d270 2024-03-14T09:03:35.694 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 msgr2=0x7fcd4006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 0x7fcd4006ccf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcd54000f60 tx=0x7fcd54009040 comp rx=0 tx=0).stop 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 msgr2=0x7fcd601445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcd58007df0 tx=0x7fcd5800d3b0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 shutdown_connections 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fcd4006a830 0x7fcd4006ccf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd6015d270 0x7fcd601445d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd6014eaf0 0x7fcd601eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 --2- 172.21.15.154:0/3292875235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd60155df0 0x7fcd600c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 >> 172.21.15.154:0/3292875235 conn(0x7fcd600c3880 msgr2=0x7fcd601561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 shutdown_connections 2024-03-14T09:03:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.696+0000 7fcd64903700 1 -- 172.21.15.154:0/3292875235 wait complete. 2024-03-14T09:03:35.708 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.832+0000 7f814e619700 1 Processor -- start 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.832+0000 7f814e619700 1 -- start start 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814e619700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 0x7f814813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814e619700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814e619700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814813ade0 con 0x7f81480c3250 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814e619700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814806fb30 con 0x7f81480a8540 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814e619700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814806ec30 con 0x7f81480c1780 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f8147fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 0x7f814813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:36468/0 (socket says 172.21.15.154:36468) 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 -- 172.21.15.154:0/1422305484 learned_addr learned my addr 172.21.15.154:0/1422305484 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 -- 172.21.15.154:0/1422305484 >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 msgr2=0x7f814813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 --2- 172.21.15.154:0/1422305484 >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 0x7f814813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 -- 172.21.15.154:0/1422305484 >> v1:172.21.15.191:6789/0 conn(0x7f81480c3250 legacy=0x7f8148154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f814cbb6700 1 -- 172.21.15.154:0/1422305484 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814806ef10 con 0x7f81480c1780 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.833+0000 7f8147fff700 1 --2- 172.21.15.154:0/1422305484 >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 0x7f814813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814cbb6700 1 --2- 172.21.15.154:0/1422305484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8140009e50 tx=0x7f814000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319b943273cf6448 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f8146ffd700 1 -- 172.21.15.154:0/1422305484 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000e070 con 0x7f81480c1780 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f8146ffd700 1 -- 172.21.15.154:0/1422305484 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814000b910 con 0x7f81480c1780 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f8146ffd700 1 -- 172.21.15.154:0/1422305484 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814000bcc0 con 0x7f81480c1780 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 -- 172.21.15.154:0/1422305484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 msgr2=0x7f81480c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 --2- 172.21.15.154:0/1422305484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8140009e50 tx=0x7f814000b330 comp rx=0 tx=0).stop 2024-03-14T09:03:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 -- 172.21.15.154:0/1422305484 shutdown_connections 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 --2- 172.21.15.154:0/1422305484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f81480c1c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 --2- 172.21.15.154:0/1422305484 >> v2:172.21.15.191:3300/0 conn(0x7f81480a8540 0x7f814813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.834+0000 7f814e619700 1 -- 172.21.15.154:0/1422305484 >> 172.21.15.154:0/1422305484 conn(0x7f81480c3880 msgr2=0x7f81480c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.835+0000 7f814e619700 1 -- 172.21.15.154:0/1422305484 shutdown_connections 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.835+0000 7f814e619700 1 -- 172.21.15.154:0/1422305484 wait complete. 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.836+0000 7f814e619700 1 Processor -- start 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.836+0000 7f814e619700 1 -- start start 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f8148152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81480c3250 0x7f81481ed680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148138800 con 0x7f81480a8540 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148138dc0 con 0x7f81480c3250 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81481542a0 con 0x7f81480c1780 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54024/0 (socket says 172.21.15.154:54024) 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 -- 172.21.15.154:0/4217552119 learned_addr learned my addr 172.21.15.154:0/4217552119 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f81477fe700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f8148152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814cbb6700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81480c3250 0x7f81481ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 -- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 msgr2=0x7f8148152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f8148152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 -- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81480c3250 msgr2=0x7f81481ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81480c3250 0x7f81481ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 -- 172.21.15.154:0/4217552119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0094a0 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8147fff700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f813c00ec10 tx=0x7f813c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c009760 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c00c850 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8140009b30 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00cbe0 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.837+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148130370 con 0x7f81480a8540 2024-03-14T09:03:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.838+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81480697d0 con 0x7f81480a8540 2024-03-14T09:03:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.839+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f813c00cd40 con 0x7f81480a8540 2024-03-14T09:03:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.839+0000 7f8144ff9700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 0x7f813406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.839+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f813c089380 con 0x7f81480a8540 2024-03-14T09:03:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.840+0000 7f81477fe700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 0x7f813406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.840+0000 7f81477fe700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 0x7f813406ccf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f814812e730 tx=0x7f8138009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:35.844+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f813c0544d0 con 0x7f81480a8540 2024-03-14T09:03:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.035+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8148152a00 con 0x7f81480a8540 2024-03-14T09:03:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.035+0000 7f8144ff9700 1 -- 172.21.15.154:0/4217552119 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v55) v1 ==== 105+0+11 (secure 0 0 0) 0x7f813c057af0 con 0x7f81480a8540 2024-03-14T09:03:36.035 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:36.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 msgr2=0x7f813406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 0x7f813406ccf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f814812e730 tx=0x7f8138009040 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 msgr2=0x7f81481546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f813c00ec10 tx=0x7f813c00ef20 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 shutdown_connections 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81480c3250 0x7f81481ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813406a830 0x7f813406ccf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81480a8540 0x7f81481546c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 --2- 172.21.15.154:0/4217552119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81480c1780 0x7f8148152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 >> 172.21.15.154:0/4217552119 conn(0x7f81480c3880 msgr2=0x7f81481392e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.037+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 shutdown_connections 2024-03-14T09:03:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.038+0000 7f814e619700 1 -- 172.21.15.154:0/4217552119 wait complete. 2024-03-14T09:03:36.049 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:03:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.174+0000 7faef3340700 1 Processor -- start 2024-03-14T09:03:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.174+0000 7faef3340700 1 -- start start 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef3340700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faeec15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef3340700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0c7bb0 con 0x7faeec0c31c0 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef3340700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0c7e90 con 0x7faeec06c670 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef3340700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0c3e00 con 0x7faeec0c6850 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef08db700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faeec0c31c0 0x7faeec0c6140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54594/0 (socket says 172.21.15.154:54594) 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef08db700 1 -- 172.21.15.154:0/230128883 learned_addr learned my addr 172.21.15.154:0/230128883 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499290951 0 0) 0x7faeec0c7bb0 con 0x7faeec0c31c0 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faef18dd700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faeec15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed80035b0 con 0x7faeec0c31c0 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3886873886 0 0) 0x7faed80035b0 con 0x7faeec0c31c0 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 msgr2=0x7faeec15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faeec15fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 >> v1:172.21.15.191:6789/0 conn(0x7faeec06c670 legacy=0x7faeec06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0c8170 con 0x7faeec0c31c0 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7faedc001670 con 0x7faeec0c31c0 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 >> v1:172.21.15.154:6789/0 conn(0x7faeec0c31c0 legacy=0x7faeec0c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed8003a30 0x7faed8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0c3e00 con 0x7faed8003a30 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed800bb40 con 0x7faed8004d90 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.175+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed800bd00 con 0x7faeec0c6850 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed8003a30 0x7faed8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef08db700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef18dd700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 -- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 msgr2=0x7faed8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec0c6850 0x7faed8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 -- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8004d90 msgr2=0x7faed8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 -- 172.21.15.154:0/230128883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80035b0 con 0x7faed8003a30 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef18dd700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8004d90 0x7faed8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faef10dc700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed8003a30 0x7faed8003e50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faee80017c0 tx=0x7faed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8330f1cf77e56698 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80052d0 con 0x7faed8003a30 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee8005430 con 0x7faed8003a30 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.176+0000 7faee3fff700 1 -- 172.21.15.154:0/230128883 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80038c0 con 0x7faed8003a30 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 -- 172.21.15.154:0/230128883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed8003a30 msgr2=0x7faed8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 --2- 172.21.15.154:0/230128883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faed8003a30 0x7faed8003e50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faee80017c0 tx=0x7faed8005880 comp rx=0 tx=0).stop 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef18dd700 1 -- 172.21.15.154:0/230128883 reap_dead start 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 -- 172.21.15.154:0/230128883 shutdown_connections 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 -- 172.21.15.154:0/230128883 >> 172.21.15.154:0/230128883 conn(0x7faeec0a9bf0 msgr2=0x7faeec0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 -- 172.21.15.154:0/230128883 shutdown_connections 2024-03-14T09:03:36.180 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.177+0000 7faef3340700 1 -- 172.21.15.154:0/230128883 wait complete. 2024-03-14T09:03:36.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.178+0000 7faef3340700 1 Processor -- start 2024-03-14T09:03:36.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.178+0000 7faef3340700 1 -- start start 2024-03-14T09:03:36.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 0x7faeec1dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.183 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec1d6d20 0x7faeec1d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1deef0 con 0x7faeec0ac940 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1ee560 con 0x7faed8003a30 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef3340700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1ed9e0 con 0x7faeec1d6d20 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef08db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef08db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54036/0 (socket says 172.21.15.154:54036) 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef08db700 1 -- 172.21.15.154:0/492943895 learned_addr learned my addr 172.21.15.154:0/492943895 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef10dc700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 0x7faeec1dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef18dd700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec1d6d20 0x7faeec1d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef08db700 1 -- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec1d6d20 msgr2=0x7faeec1d71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.179+0000 7faef08db700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec1d6d20 0x7faeec1d71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef08db700 1 -- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 msgr2=0x7faeec1dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.184 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef08db700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 0x7faeec1dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef08db700 1 -- 172.21.15.154:0/492943895 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec1edcc0 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef10dc700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 0x7faeec1dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef08db700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7faedc007290 tx=0x7faedc007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc0014c0 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc00f8d0 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec1edfa0 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00fc80 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.180+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec1e0080 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.181+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec073c70 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.181+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7faedc01d460 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.182+0000 7faee1ffb700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 0x7faed006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.182+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7faedc0891d0 con 0x7faeec0ac940 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.182+0000 7faef10dc700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 0x7faed006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.185 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.182+0000 7faef10dc700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 0x7faed006ccf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faee8015040 tx=0x7faee8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.186 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.186+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7faedc054280 con 0x7faeec0ac940 2024-03-14T09:03:36.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.383+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7faeec1dd590 con 0x7faeec0ac940 2024-03-14T09:03:36.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.383+0000 7faee1ffb700 1 -- 172.21.15.154:0/492943895 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v55) v1 ==== 101+0+11 (secure 0 0 0) 0x7faeec1dd590 con 0x7faeec0ac940 2024-03-14T09:03:36.383 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:36.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 msgr2=0x7faed006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 0x7faed006ccf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faee8015040 tx=0x7faee8009030 comp rx=0 tx=0).stop 2024-03-14T09:03:36.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 msgr2=0x7faeec0acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7faedc007290 tx=0x7faedc007650 comp rx=0 tx=0).stop 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 shutdown_connections 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeec1d6d20 0x7faeec1d71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faed8003a30 0x7faeec1dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7faed006a830 0x7faed006ccf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 --2- 172.21.15.154:0/492943895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeec0ac940 0x7faeec0acdc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 >> 172.21.15.154:0/492943895 conn(0x7faeec0a9bf0 msgr2=0x7faeec0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 shutdown_connections 2024-03-14T09:03:36.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.385+0000 7faef3340700 1 -- 172.21.15.154:0/492943895 wait complete. 2024-03-14T09:03:36.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.518+0000 7f706c028700 1 Processor -- start 2024-03-14T09:03:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.518+0000 7f706c028700 1 -- start start 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706c028700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f706414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706c028700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70640c5a00 0x7f7064151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706c028700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7064155a40 con 0x7f70640c55a0 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706c028700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70640c5e70 con 0x7f70640c5a00 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706c028700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640c2f10 con 0x7f70640c26c0 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f70695c3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f70640c55a0 0x7f70641518c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54602/0 (socket says 172.21.15.154:54602) 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f706a5c5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f70640c5a00 0x7f7064151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.519+0000 7f70695c3700 1 -- 172.21.15.154:0/1038549863 learned_addr learned my addr 172.21.15.154:0/1038549863 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f7069dc4700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f706414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f706a5c5700 1 -- 172.21.15.154:0/1038549863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 msgr2=0x7f706414edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f706a5c5700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f706414edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f706a5c5700 1 -- 172.21.15.154:0/1038549863 >> v1:172.21.15.154:6789/0 conn(0x7f70640c55a0 legacy=0x7f70641518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f7068dc2700 1 -- 172.21.15.154:0/1038549863 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978710095 0 0) 0x7f7064155a40 con 0x7f70640c55a0 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f706a5c5700 1 -- 172.21.15.154:0/1038549863 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064072a90 con 0x7f70640c5a00 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f706a5c5700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c5a00 0x7f7064151fd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7054009840 tx=0x7f7054009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b803eb256bbb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f7068dc2700 1 -- 172.21.15.154:0/1038549863 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054011830 con 0x7f70640c5a00 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f7068dc2700 1 -- 172.21.15.154:0/1038549863 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054011990 con 0x7f70640c5a00 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.520+0000 7f7068dc2700 1 -- 172.21.15.154:0/1038549863 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054011d20 con 0x7f70640c5a00 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 -- 172.21.15.154:0/1038549863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c5a00 msgr2=0x7f7064151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c5a00 0x7f7064151fd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7054009840 tx=0x7f7054009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 -- 172.21.15.154:0/1038549863 shutdown_connections 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c5a00 0x7f7064151fd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 --2- 172.21.15.154:0/1038549863 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f706414edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 -- 172.21.15.154:0/1038549863 >> 172.21.15.154:0/1038549863 conn(0x7f706406a770 msgr2=0x7f706406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 -- 172.21.15.154:0/1038549863 shutdown_connections 2024-03-14T09:03:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.521+0000 7f706c028700 1 -- 172.21.15.154:0/1038549863 wait complete. 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 Processor -- start 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 -- start start 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f70640c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70640c5a00 0x7f706414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641e7b00 con 0x7f70640c55a0 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706414c210 con 0x7f70640c5a00 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.522+0000 7f706c028700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640ab8c0 con 0x7f70640c26c0 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54048/0 (socket says 172.21.15.154:54048) 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 -- 172.21.15.154:0/101521688 learned_addr learned my addr 172.21.15.154:0/101521688 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f706a5c5700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70640c5a00 0x7f706414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f7069dc4700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f70640c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 -- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 msgr2=0x7f70640c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f70640c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 -- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70640c5a00 msgr2=0x7f706414bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70640c5a00 0x7f706414bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 -- 172.21.15.154:0/101521688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70540094a0 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f7069dc4700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f70640c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f70695c3700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70600011e0 tx=0x7f706000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060004c80 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060004e00 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007310 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70640ab5e0 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.523+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70641ebd20 con 0x7f70640c55a0 2024-03-14T09:03:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.524+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70641e7c40 con 0x7f70640c55a0 2024-03-14T09:03:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.525+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7060007470 con 0x7f70640c55a0 2024-03-14T09:03:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.525+0000 7f705a7fc700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 0x7f704806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.525+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f7060089270 con 0x7f70640c55a0 2024-03-14T09:03:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.526+0000 7f7069dc4700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 0x7f704806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.526+0000 7f7069dc4700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 0x7f704806ccf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f705c009bc0 tx=0x7f705c003a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.529+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f70600544a0 con 0x7f70640c55a0 2024-03-14T09:03:36.763 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.763+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70641e7c40 con 0x7f70640c55a0 2024-03-14T09:03:36.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.764+0000 7f705a7fc700 1 -- 172.21.15.154:0/101521688 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5956 (secure 0 0 0) 0x7f706005b760 con 0x7f70640c55a0 2024-03-14T09:03:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:36.764 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 msgr2=0x7f704806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 0x7f704806ccf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f705c009bc0 tx=0x7f705c003a60 comp rx=0 tx=0).stop 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 msgr2=0x7f706414b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70600011e0 tx=0x7f706000cae0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 shutdown_connections 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70640c5a00 0x7f706414bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70640c26c0 0x7f70640c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f704806a830 0x7f704806ccf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 --2- 172.21.15.154:0/101521688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70640c55a0 0x7f706414b790 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 >> 172.21.15.154:0/101521688 conn(0x7f706406a770 msgr2=0x7f706406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 shutdown_connections 2024-03-14T09:03:36.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.766+0000 7f706c028700 1 -- 172.21.15.154:0/101521688 wait complete. 2024-03-14T09:03:36.767 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 35 2024-03-14T09:03:36.778 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:36.899 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.899+0000 7eff50eb1700 1 Processor -- start 2024-03-14T09:03:36.899 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.899+0000 7eff50eb1700 1 -- start start 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff50eb1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7eff4c155df0 0x7eff4c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff50eb1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff4c15d270 0x7eff4c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff50eb1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4c0ac8e0 con 0x7eff4c14eaf0 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff50eb1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c06b310 con 0x7eff4c15d270 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff50eb1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c06de30 con 0x7eff4c155df0 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff4ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff4c15d270 0x7eff4c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff4ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff4c15d270 0x7eff4c15f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54060/0 (socket says 172.21.15.154:54060) 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.900+0000 7eff4ad9d700 1 -- 172.21.15.154:0/2209653208 learned_addr learned my addr 172.21.15.154:0/2209653208 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4ad9d700 1 -- 172.21.15.154:0/2209653208 >> v2:172.21.15.191:3300/0 conn(0x7eff4c155df0 msgr2=0x7eff4c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff49d9b700 1 --2- 172.21.15.154:0/2209653208 >> v2:172.21.15.191:3300/0 conn(0x7eff4c155df0 0x7eff4c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4ad9d700 1 --2- 172.21.15.154:0/2209653208 >> v2:172.21.15.191:3300/0 conn(0x7eff4c155df0 0x7eff4c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4ad9d700 1 -- 172.21.15.154:0/2209653208 >> v1:172.21.15.191:6789/0 conn(0x7eff4c14eaf0 legacy=0x7eff4c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4ad9d700 1 -- 172.21.15.154:0/2209653208 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0acb30 con 0x7eff4c15d270 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4ad9d700 1 --2- 172.21.15.154:0/2209653208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c15d270 0x7eff4c15f660 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff4c14f1c0 tx=0x7eff4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a979a4b9bc774526 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4959a700 1 -- 172.21.15.154:0/2209653208 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400e070 con 0x7eff4c15d270 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4959a700 1 -- 172.21.15.154:0/2209653208 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff44003a20 con 0x7eff4c15d270 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.901+0000 7eff4959a700 1 -- 172.21.15.154:0/2209653208 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff44003dd0 con 0x7eff4c15d270 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 -- 172.21.15.154:0/2209653208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c15d270 msgr2=0x7eff4c15f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 --2- 172.21.15.154:0/2209653208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c15d270 0x7eff4c15f660 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff4c14f1c0 tx=0x7eff4400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 -- 172.21.15.154:0/2209653208 shutdown_connections 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 --2- 172.21.15.154:0/2209653208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c15d270 0x7eff4c15f660 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 --2- 172.21.15.154:0/2209653208 >> v2:172.21.15.191:3300/0 conn(0x7eff4c155df0 0x7eff4c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 -- 172.21.15.154:0/2209653208 >> 172.21.15.154:0/2209653208 conn(0x7eff4c0c3880 msgr2=0x7eff4c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 -- 172.21.15.154:0/2209653208 shutdown_connections 2024-03-14T09:03:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.902+0000 7eff50eb1700 1 -- 172.21.15.154:0/2209653208 wait complete. 2024-03-14T09:03:36.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.903+0000 7eff50eb1700 1 Processor -- start 2024-03-14T09:03:36.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.903+0000 7eff50eb1700 1 -- start start 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff4c14eaf0 0x7eff4c14f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff4c15d270 0x7eff4c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c150e20 con 0x7eff4c155df0 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c152260 con 0x7eff4c14eaf0 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c151f10 con 0x7eff4c15d270 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54066/0 (socket says 172.21.15.154:54066) 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 -- 172.21.15.154:0/1939849480 learned_addr learned my addr 172.21.15.154:0/1939849480 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff4a59c700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff4c14eaf0 0x7eff4c14f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff4ad9d700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff4c15d270 0x7eff4c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 -- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff4c15d270 msgr2=0x7eff4c0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff4c15d270 0x7eff4c0c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 -- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff4c14eaf0 msgr2=0x7eff4c14f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff4c14eaf0 0x7eff4c14f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 -- 172.21.15.154:0/1939849480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44009b30 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff49d9b700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c7f0 tx=0x7eff3c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c004220 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c004380 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c14f540 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.904+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0036a0 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.905+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c1eb3f0 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.905+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c1523f0 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.906+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7eff3c003800 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.906+0000 7eff42ffd700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 0x7eff2c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.906+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7eff3c088bd0 con 0x7eff4c155df0 2024-03-14T09:03:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.907+0000 7eff4a59c700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 0x7eff2c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.907+0000 7eff4a59c700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 0x7eff2c06ccf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff34007fd0 tx=0x7eff34007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:36.911+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7eff3c053c90 con 0x7eff4c155df0 2024-03-14T09:03:37.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.106+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff4c0697d0 con 0x7eff4c155df0 2024-03-14T09:03:37.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.107+0000 7eff42ffd700 1 -- 172.21.15.154:0/1939849480 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7eff3c015090 con 0x7eff4c155df0 2024-03-14T09:03:37.107 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:37.107 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:37.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 msgr2=0x7eff2c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 0x7eff2c06ccf0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff34007fd0 tx=0x7eff34007ec0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 msgr2=0x7eff4c150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.109 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c7f0 tx=0x7eff3c00cb00 comp rx=0 tx=0).stop 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 shutdown_connections 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff4c15d270 0x7eff4c0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff4c14eaf0 0x7eff4c14f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff2c06a830 0x7eff2c06ccf0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 --2- 172.21.15.154:0/1939849480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4c155df0 0x7eff4c150350 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 >> 172.21.15.154:0/1939849480 conn(0x7eff4c0c3880 msgr2=0x7eff4c1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 shutdown_connections 2024-03-14T09:03:37.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.109+0000 7eff50eb1700 1 -- 172.21.15.154:0/1939849480 wait complete. 2024-03-14T09:03:37.121 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T09:03:37.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:37.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.243+0000 7f3a025db700 1 Processor -- start 2024-03-14T09:03:37.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.243+0000 7f3a025db700 1 -- start start 2024-03-14T09:03:37.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39fc0a8540 0x7f39fc13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f39fc1550b0 0x7f39fc0c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc13ade0 con 0x7f39fc0a8540 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc06fb30 con 0x7f39fc1550b0 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a025db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc06ec30 con 0x7f39fc0c3250 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39fc0a8540 0x7f39fc13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39fc0a8540 0x7f39fc13a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54076/0 (socket says 172.21.15.154:54076) 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 -- 172.21.15.154:0/1773061340 learned_addr learned my addr 172.21.15.154:0/1773061340 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fb7fe700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f3a00b78700 1 --2- 172.21.15.154:0/1773061340 >> v2:172.21.15.191:3300/0 conn(0x7f39fc1550b0 0x7f39fc0c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 -- 172.21.15.154:0/1773061340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 msgr2=0x7f39fc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 -- 172.21.15.154:0/1773061340 >> v2:172.21.15.191:3300/0 conn(0x7f39fc1550b0 msgr2=0x7f39fc0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 --2- 172.21.15.154:0/1773061340 >> v2:172.21.15.191:3300/0 conn(0x7f39fc1550b0 0x7f39fc0c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.244+0000 7f39fbfff700 1 -- 172.21.15.154:0/1773061340 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc06ef10 con 0x7f39fc0a8540 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f39fbfff700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc13a8a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c1440 tx=0x7f39f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3e9eee0b7077a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f39faffd700 1 -- 172.21.15.154:0/1773061340 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0011830 con 0x7f39fc0a8540 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f39faffd700 1 -- 172.21.15.154:0/1773061340 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f0011990 con 0x7f39fc0a8540 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f39faffd700 1 -- 172.21.15.154:0/1773061340 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0011d20 con 0x7f39fc0a8540 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 -- 172.21.15.154:0/1773061340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 msgr2=0x7f39fc13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc13a8a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c1440 tx=0x7f39f0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 -- 172.21.15.154:0/1773061340 shutdown_connections 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 --2- 172.21.15.154:0/1773061340 >> v2:172.21.15.191:3300/0 conn(0x7f39fc1550b0 0x7f39fc0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc13a8a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 --2- 172.21.15.154:0/1773061340 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 -- 172.21.15.154:0/1773061340 >> 172.21.15.154:0/1773061340 conn(0x7f39fc0c3880 msgr2=0x7f39fc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 -- 172.21.15.154:0/1773061340 shutdown_connections 2024-03-14T09:03:37.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.245+0000 7f3a025db700 1 -- 172.21.15.154:0/1773061340 wait complete. 2024-03-14T09:03:37.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.246+0000 7f3a025db700 1 Processor -- start 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 -- start start 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc1db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39fc1550b0 0x7f39fc153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc152360 con 0x7f39fc0a8540 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc1dd670 con 0x7f39fc1550b0 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.247+0000 7f3a025db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc151d50 con 0x7f39fc0c3250 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54080/0 (socket says 172.21.15.154:54080) 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 -- 172.21.15.154:0/3373797131 learned_addr learned my addr 172.21.15.154:0/3373797131 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fb7fe700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc1db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f3a00b78700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39fc1550b0 0x7f39fc153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 -- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 msgr2=0x7f39fc1db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc1db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 -- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39fc1550b0 msgr2=0x7f39fc153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39fc1550b0 0x7f39fc153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 -- 172.21.15.154:0/3373797131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0009550 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39fbfff700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009c10 tx=0x7f39f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0004300 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f001f070 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc1dd0b0 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f00036a0 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.248+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc151f80 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.249+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc152110 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.250+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f39f0021030 con 0x7f39fc0a8540 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.250+0000 7f39f8ff9700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 0x7f39e806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.250+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f39f008a3f0 con 0x7f39fc0a8540 2024-03-14T09:03:37.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.250+0000 7f39fb7fe700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 0x7f39e806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.251+0000 7f39fb7fe700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 0x7f39e806ccf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c6360 tx=0x7f39ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.255+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f39f0055660 con 0x7f39fc0a8540 2024-03-14T09:03:37.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.492+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39fc0697d0 con 0x7f39fc0a8540 2024-03-14T09:03:37.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.493+0000 7f39f8ff9700 1 -- 172.21.15.154:0/3373797131 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5956 (secure 0 0 0) 0x7f39f0058c80 con 0x7f39fc0a8540 2024-03-14T09:03:37.493 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:37.493 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 msgr2=0x7f39e806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 0x7f39e806ccf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0c6360 tx=0x7f39ec008040 comp rx=0 tx=0).stop 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 msgr2=0x7f39fc14f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009c10 tx=0x7f39f0004750 comp rx=0 tx=0).stop 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 shutdown_connections 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39fc1550b0 0x7f39fc153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f39e806a830 0x7f39e806ccf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39fc0a8540 0x7f39fc14f3f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 --2- 172.21.15.154:0/3373797131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39fc0c3250 0x7f39fc1db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 >> 172.21.15.154:0/3373797131 conn(0x7f39fc0c3880 msgr2=0x7f39fc1397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 shutdown_connections 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.495+0000 7f3a025db700 1 -- 172.21.15.154:0/3373797131 wait complete. 2024-03-14T09:03:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 35 2024-03-14T09:03:37.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:37.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.630+0000 7fd8c56eb700 1 Processor -- start 2024-03-14T09:03:37.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.630+0000 7fd8c56eb700 1 -- start start 2024-03-14T09:03:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8c56eb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c0155df0 0x7fd8c0156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8c56eb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8c015d270 0x7fd8c015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8c56eb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c00ac8e0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8c56eb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c006b310 con 0x7fd8c015d270 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8c56eb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c006de30 con 0x7fd8c0155df0 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8c015d270 0x7fd8c015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8c015d270 0x7fd8c015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54094/0 (socket says 172.21.15.154:54094) 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 -- 172.21.15.154:0/2453016127 learned_addr learned my addr 172.21.15.154:0/2453016127 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8be7fc700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c0155df0 0x7fd8c0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 -- 172.21.15.154:0/2453016127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c0155df0 msgr2=0x7fd8c0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c0155df0 0x7fd8c0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 -- 172.21.15.154:0/2453016127 >> v1:172.21.15.191:6789/0 conn(0x7fd8c014eaf0 legacy=0x7fd8c006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 -- 172.21.15.154:0/2453016127 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00acb30 con 0x7fd8c015d270 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.631+0000 7fd8bf7fe700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c015d270 0x7fd8c015f660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009e50 tx=0x7fd8b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b7f97be1d94b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8bdffb700 1 -- 172.21.15.154:0/2453016127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800e070 con 0x7fd8c015d270 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8bdffb700 1 -- 172.21.15.154:0/2453016127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b800b910 con 0x7fd8c015d270 2024-03-14T09:03:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8bdffb700 1 -- 172.21.15.154:0/2453016127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800bcc0 con 0x7fd8c015d270 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 -- 172.21.15.154:0/2453016127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c015d270 msgr2=0x7fd8c015f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c015d270 0x7fd8c015f660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009e50 tx=0x7fd8b800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 -- 172.21.15.154:0/2453016127 shutdown_connections 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c015d270 0x7fd8c015f660 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/2453016127 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c0155df0 0x7fd8c0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 -- 172.21.15.154:0/2453016127 >> 172.21.15.154:0/2453016127 conn(0x7fd8c00c3880 msgr2=0x7fd8c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:37.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 -- 172.21.15.154:0/2453016127 shutdown_connections 2024-03-14T09:03:37.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.632+0000 7fd8c56eb700 1 -- 172.21.15.154:0/2453016127 wait complete. 2024-03-14T09:03:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.633+0000 7fd8c56eb700 1 Processor -- start 2024-03-14T09:03:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 -- start start 2024-03-14T09:03:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c0155df0 0x7fd8c0149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c015d270 0x7fd8c0144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ed690 con 0x7fd8c014eaf0 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ed970 con 0x7fd8c0155df0 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8c56eb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0152660 con 0x7fd8c015d270 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54104/0 (socket says 172.21.15.154:54104) 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8beffd700 1 -- 172.21.15.154:0/3111380041 learned_addr learned my addr 172.21.15.154:0/3111380041 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8bf7fe700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c015d270 0x7fd8c0144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.634+0000 7fd8be7fc700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c0155df0 0x7fd8c0149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 -- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c015d270 msgr2=0x7fd8c0144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c015d270 0x7fd8c0144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 -- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c0155df0 msgr2=0x7fd8c0149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c0155df0 0x7fd8c0149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 -- 172.21.15.154:0/3111380041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009b30 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8beffd700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c4e0 tx=0x7fd8b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00052d0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0005430 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c013ec00 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00038c0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.635+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c01497d0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.636+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c00697d0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.636+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd8b0003a20 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.637+0000 7fd8b77fe700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 0x7fd8a006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.637+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fd8b0088420 con 0x7fd8c014eaf0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.637+0000 7fd8be7fc700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 0x7fd8a006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.638+0000 7fd8be7fc700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 0x7fd8a006ccf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8c013f7f0 tx=0x7fd8a8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:37.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.641+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd8b00534d0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.878+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c006b470 con 0x7fd8c014eaf0 2024-03-14T09:03:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.878+0000 7fd8b77fe700 1 -- 172.21.15.154:0/3111380041 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5956 (secure 0 0 0) 0x7fd8b0056af0 con 0x7fd8c014eaf0 2024-03-14T09:03:37.879 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:37.879 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:02:59.971830+0000","modified":"2024-03-14T09:03:03.317866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"b","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6837/1333349284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1333349284},{"type":"v1","addr":"172.21.15.191:6837","nonce":1333349284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4},{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 msgr2=0x7fd8a006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 0x7fd8a006ccf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8c013f7f0 tx=0x7fd8a8009210 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 msgr2=0x7fd8c01eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c4e0 tx=0x7fd8b000c7f0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 shutdown_connections 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8c015d270 0x7fd8c0144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd8a006a830 0x7fd8a006ccf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8c014eaf0 0x7fd8c01eb600 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 --2- 172.21.15.154:0/3111380041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8c0155df0 0x7fd8c0149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 >> 172.21.15.154:0/3111380041 conn(0x7fd8c00c3880 msgr2=0x7fd8c006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 shutdown_connections 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:37.881+0000 7fd8c56eb700 1 -- 172.21.15.154:0/3111380041 wait complete. 2024-03-14T09:03:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 35 2024-03-14T09:03:37.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:38.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.014+0000 7f1ea7d25700 1 Processor -- start 2024-03-14T09:03:38.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.015+0000 7f1ea7d25700 1 -- start start 2024-03-14T09:03:38.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea7d25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1ea00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea7d25700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1ea0158370 0x7f1ea01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea7d25700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea014ed60 con 0x7f1ea014e980 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea7d25700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00c6b10 con 0x7f1ea0158370 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea7d25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c6cf0 con 0x7f1ea00c4a80 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea5ac1700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1ea014e980 0x7f1ea01552e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54614/0 (socket says 172.21.15.154:54614) 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea5ac1700 1 -- 172.21.15.154:0/3648682229 learned_addr learned my addr 172.21.15.154:0/3648682229 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea62c2700 1 --2- 172.21.15.154:0/3648682229 >> v2:172.21.15.191:3300/0 conn(0x7f1ea0158370 0x7f1ea01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea52c0700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1ea00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636541828 0 0) 0x7f1ea014ed60 con 0x7f1ea014e980 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8c0035b0 con 0x7f1ea014e980 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2922931643 0 0) 0x7f1e8c0035b0 con 0x7f1ea014e980 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 msgr2=0x7f1ea00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1ea00c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 >> v2:172.21.15.191:3300/0 conn(0x7f1ea0158370 msgr2=0x7f1ea01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 --2- 172.21.15.154:0/3648682229 >> v2:172.21.15.191:3300/0 conn(0x7f1ea0158370 0x7f1ea01587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00c6fd0 con 0x7f1ea014e980 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f1e9c001670 con 0x7f1ea014e980 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 >> v1:172.21.15.154:6789/0 conn(0x7f1ea014e980 legacy=0x7f1ea01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e8c003a00 0x7f1e8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1e8c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c004d90 0x7f1e8c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c6b10 con 0x7f1e8c003a00 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c6cf0 con 0x7f1e8c004d90 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00ba40 con 0x7f1ea00c4a80 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.016+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e8c003a00 0x7f1e8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea52c0700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1e8c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea62c2700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c004d90 0x7f1e8c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 -- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 msgr2=0x7f1e8c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00c4a80 0x7f1e8c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 -- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c004d90 msgr2=0x7f1e8c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c004d90 0x7f1e8c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 -- 172.21.15.154:0/3648682229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0035b0 con 0x7f1e8c003a00 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e8c003a00 0x7f1e8c003e20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c370 tx=0x7f1e8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80353c88cd5fa802 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c007c60 con 0x7f1e8c003a00 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c005960 con 0x7f1e8c003a00 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea4abf700 1 -- 172.21.15.154:0/3648682229 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c005d10 con 0x7f1e8c003a00 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea7d25700 1 -- 172.21.15.154:0/3648682229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e8c003a00 msgr2=0x7f1e8c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/3648682229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e8c003a00 0x7f1e8c003e20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c370 tx=0x7f1e8c005880 comp rx=0 tx=0).stop 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.017+0000 7f1ea62c2700 1 -- 172.21.15.154:0/3648682229 reap_dead start 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.018+0000 7f1ea7d25700 1 -- 172.21.15.154:0/3648682229 shutdown_connections 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.018+0000 7f1ea7d25700 1 -- 172.21.15.154:0/3648682229 >> 172.21.15.154:0/3648682229 conn(0x7f1ea00abcf0 msgr2=0x7f1ea00ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.018+0000 7f1ea7d25700 1 -- 172.21.15.154:0/3648682229 shutdown_connections 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.018+0000 7f1ea7d25700 1 -- 172.21.15.154:0/3648682229 wait complete. 2024-03-14T09:03:38.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 Processor -- start 2024-03-14T09:03:38.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 -- start start 2024-03-14T09:03:38.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 0x7f1ea00bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 0x7f1ea00be630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00aee80 con 0x7f1ea00accc0 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c0ca0 con 0x7f1e8c003a00 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.019+0000 7f1ea7d25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00bdca0 con 0x7f1ea00be180 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54126/0 (socket says 172.21.15.154:54126) 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 -- 172.21.15.154:0/746674113 learned_addr learned my addr 172.21.15.154:0/746674113 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea62c2700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 0x7f1ea00be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 0x7f1ea00bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 -- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 msgr2=0x7f1ea00be630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 0x7f1ea00be630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 -- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 msgr2=0x7f1ea00bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 0x7f1ea00bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 -- 172.21.15.154:0/746674113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00c09c0 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea62c2700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 0x7f1ea00be630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 0x7f1ea00bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea52c0700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1e940119a0 tx=0x7f1e94011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400f8e0 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9400fa40 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea01eb130 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400fdd0 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.020+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea00ab870 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.021+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea01eaf60 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.021+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1e94014440 con 0x7f1ea00accc0 2024-03-14T09:03:38.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.022+0000 7f1e927fc700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 0x7f1e8406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.022+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7f1e94089790 con 0x7f1ea00accc0 2024-03-14T09:03:38.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.022+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 0x7f1e8406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.023+0000 7f1ea5ac1700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 0x7f1e8406ccf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c4670 tx=0x7f1e9c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.027+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1e940548e0 con 0x7f1ea00accc0 2024-03-14T09:03:38.221 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.222+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea00bdde0 con 0x7f1ea00accc0 2024-03-14T09:03:38.223 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.223+0000 7f1e927fc700 1 -- 172.21.15.154:0/746674113 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+19860 (secure 0 0 0) 0x7f1e94057f00 con 0x7f1ea00accc0 2024-03-14T09:03:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:38.223 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":55,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:13.762182+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:02:57.920434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-14T09:02:58.934033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 msgr2=0x7f1e8406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 0x7f1e8406ccf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00c4670 tx=0x7f1e9c00a000 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 msgr2=0x7f1ea00c0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1e940119a0 tx=0x7f1e94011d60 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 shutdown_connections 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ea00be180 0x7f1ea00be630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e8c003a00 0x7f1ea00bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1e8406a830 0x7f1e8406ccf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 --2- 172.21.15.154:0/746674113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ea00accc0 0x7f1ea00c0350 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 >> 172.21.15.154:0/746674113 conn(0x7f1ea00abcf0 msgr2=0x7f1ea0156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:38.226 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 shutdown_connections 2024-03-14T09:03:38.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.226+0000 7f1ea7d25700 1 -- 172.21.15.154:0/746674113 wait complete. 2024-03-14T09:03:38.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T09:03:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.362+0000 7fb207a9a700 1 Processor -- start 2024-03-14T09:03:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.362+0000 7fb207a9a700 1 -- start start 2024-03-14T09:03:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb207a9a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb207a9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 0x7fb20015f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb207a9a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2000ac8e0 con 0x7fb20006acb0 2024-03-14T09:03:38.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb207a9a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20006b310 con 0x7fb20014eaf0 2024-03-14T09:03:38.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb207a9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20006de30 con 0x7fb200156500 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb205836700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb206037700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 0x7fb20015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb205836700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:37748/0 (socket says 172.21.15.154:37748) 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb205836700 1 -- 172.21.15.154:0/1119191811 learned_addr learned my addr 172.21.15.154:0/1119191811 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.363+0000 7fb205035700 1 --1- 172.21.15.154:0/1119191811 >> v1:172.21.15.191:6789/0 conn(0x7fb20006acb0 0x7fb200155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:51252/0 (socket says 172.21.15.154:51252) 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb206037700 1 -- 172.21.15.154:0/1119191811 >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 msgr2=0x7fb20006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb206037700 1 --2- 172.21.15.154:0/1119191811 >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb206037700 1 -- 172.21.15.154:0/1119191811 >> v1:172.21.15.191:6789/0 conn(0x7fb20006acb0 legacy=0x7fb200155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb206037700 1 -- 172.21.15.154:0/1119191811 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200156930 con 0x7fb200156500 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb205836700 1 --2- 172.21.15.154:0/1119191811 >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb206037700 1 --2- 172.21.15.154:0/1119191811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 0x7fb20015f280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009d90 tx=0x7fb1f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373b7b6ae75a29e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb204834700 1 -- 172.21.15.154:0/1119191811 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800e070 con 0x7fb200156500 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb204834700 1 -- 172.21.15.154:0/1119191811 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f800bb20 con 0x7fb200156500 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.364+0000 7fb204834700 1 -- 172.21.15.154:0/1119191811 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8012430 con 0x7fb200156500 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 -- 172.21.15.154:0/1119191811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 msgr2=0x7fb20015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 --2- 172.21.15.154:0/1119191811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 0x7fb20015f280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009d90 tx=0x7fb1f800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 -- 172.21.15.154:0/1119191811 shutdown_connections 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 --2- 172.21.15.154:0/1119191811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb200156500 0x7fb20015f280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 --2- 172.21.15.154:0/1119191811 >> v2:172.21.15.191:3300/0 conn(0x7fb20014eaf0 0x7fb20006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 -- 172.21.15.154:0/1119191811 >> 172.21.15.154:0/1119191811 conn(0x7fb2000c3880 msgr2=0x7fb2000c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 -- 172.21.15.154:0/1119191811 shutdown_connections 2024-03-14T09:03:38.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.365+0000 7fb207a9a700 1 -- 172.21.15.154:0/1119191811 wait complete. 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.366+0000 7fb207a9a700 1 Processor -- start 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.366+0000 7fb207a9a700 1 -- start start 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20006acb0 0x7fb2001ea970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20014eaf0 0x7fb2000c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200152820 con 0x7fb200156500 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb200151760 con 0x7fb20014eaf0 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb200144fb0 con 0x7fb20006acb0 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54132/0 (socket says 172.21.15.154:54132) 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 -- 172.21.15.154:0/3563664249 learned_addr learned my addr 172.21.15.154:0/3563664249 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb205035700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20014eaf0 0x7fb2000c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb205836700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20006acb0 0x7fb2001ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 -- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20006acb0 msgr2=0x7fb2001ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20006acb0 0x7fb2001ea970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 -- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20014eaf0 msgr2=0x7fb2000c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20014eaf0 0x7fb2000c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 -- 172.21.15.154:0/3563664249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0009550 con 0x7fb200156500 2024-03-14T09:03:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb206037700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8014040 tx=0x7fb1f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8012430 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f8009a70 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8020070 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8003ea0 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.367+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb200153ae0 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.368+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200152960 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.369+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb1f8004040 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.369+0000 7fb1f67fc700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 0x7fb1e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.369+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6565+0+0 (secure 0 0 0) 0x7fb1f8022070 con 0x7fb200156500 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.369+0000 7fb205836700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 0x7fb1e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.370+0000 7fb205836700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 0x7fb1e406ccf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0006d30 tx=0x7fb1f00092c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.374+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb200152960 con 0x7fb200156500 2024-03-14T09:03:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.611+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb200152960 con 0x7fb200156500 2024-03-14T09:03:38.611 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:03:38.612+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:03:38.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.624+0000 7fb1f67fc700 1 -- 172.21.15.154:0/3563664249 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb200152960 con 0x7fb200156500 2024-03-14T09:03:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 msgr2=0x7fb1e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 0x7fb1e406ccf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0006d30 tx=0x7fb1f00092c0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 msgr2=0x7fb200152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8014040 tx=0x7fb1f8004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 shutdown_connections 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1e406a830 0x7fb1e406ccf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb200156500 0x7fb200152c40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20006acb0 0x7fb2001ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 --2- 172.21.15.154:0/3563664249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20014eaf0 0x7fb2000c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 >> 172.21.15.154:0/3563664249 conn(0x7fb2000c3880 msgr2=0x7fb200158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 shutdown_connections 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.626+0000 7fb207a9a700 1 -- 172.21.15.154:0/3563664249 wait complete. 2024-03-14T09:03:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:03:38.628 INFO:tasks.ceph.mds.b.smithi191.stderr: -4099> 2024-03-14T09:03:05.548+0000 7fe683d2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:03:38.638 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T09:03:38.663 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:03:38.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.764+0000 7f35a574d700 1 Processor -- start 2024-03-14T09:03:38.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.764+0000 7f35a574d700 1 -- start start 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f35a574d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f35a014e980 0x7f35a01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f35a574d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a014ed60 con 0x7f35a0158a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f35a574d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a0155820 con 0x7f35a00c4a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f35a574d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a00c6b10 con 0x7f35a014e980 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359f7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f35a0158a80 0x7f35a015f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54626/0 (socket says 172.21.15.154:54626) 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359f7fe700 1 -- 172.21.15.154:0/241017186 learned_addr learned my addr 172.21.15.154:0/241017186 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359effd700 1 --2- 172.21.15.154:0/241017186 >> v2:172.21.15.191:3300/0 conn(0x7f35a014e980 0x7f35a01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080450742 0 0) 0x7f35a014ed60 con 0x7f35a0158a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35880035b0 con 0x7f35a0158a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134308234 0 0) 0x7f35a0155820 con 0x7f35a00c4a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35a014ed60 con 0x7f35a00c4a80 2024-03-14T09:03:38.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359effd700 1 -- 172.21.15.154:0/241017186 >> v1:172.21.15.191:6789/0 conn(0x7f35a00c4a80 legacy=0x7f35a0158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359effd700 1 -- 172.21.15.154:0/241017186 >> v1:172.21.15.154:6789/0 conn(0x7f35a0158a80 legacy=0x7f35a015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359effd700 1 -- 172.21.15.154:0/241017186 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a00c6cf0 con 0x7f35a014e980 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.765+0000 7f359effd700 1 --2- 172.21.15.154:0/241017186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a014e980 0x7f35a01552e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3594009840 tx=0x7f3594009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4a951b97dd2f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594011830 con 0x7f35a014e980 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594011990 con 0x7f35a014e980 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f359dffb700 1 -- 172.21.15.154:0/241017186 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594011d20 con 0x7f35a014e980 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 -- 172.21.15.154:0/241017186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a014e980 msgr2=0x7f35a01552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 --2- 172.21.15.154:0/241017186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a014e980 0x7f35a01552e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3594009840 tx=0x7f3594009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 -- 172.21.15.154:0/241017186 shutdown_connections 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 --2- 172.21.15.154:0/241017186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a014e980 0x7f35a01552e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 -- 172.21.15.154:0/241017186 >> 172.21.15.154:0/241017186 conn(0x7f35a00abcf0 msgr2=0x7f35a00ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 -- 172.21.15.154:0/241017186 shutdown_connections 2024-03-14T09:03:38.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.766+0000 7f35a574d700 1 -- 172.21.15.154:0/241017186 wait complete. 2024-03-14T09:03:38.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 Processor -- start 2024-03-14T09:03:38.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 -- start start 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 0x7f35a00aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35a0158a80 0x7f35a01449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0155160 con 0x7f35a014e980 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a01ede10 con 0x7f35a00c4a80 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f35a574d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0145b30 con 0x7f35a0158a80 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f359e7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54142/0 (socket says 172.21.15.154:54142) 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.768+0000 7f359e7fc700 1 -- 172.21.15.154:0/56699108 learned_addr learned my addr 172.21.15.154:0/56699108 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359effd700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 0x7f35a00aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359f7fe700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35a0158a80 0x7f35a01449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 -- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35a0158a80 msgr2=0x7f35a01449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35a0158a80 0x7f35a01449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 -- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 msgr2=0x7f35a00aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 0x7f35a00aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 -- 172.21.15.154:0/56699108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940094a0 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359effd700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 0x7f35a00aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f359e7fc700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f359000c500 tx=0x7f359000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900052d0 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3590005450 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900038c0 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a00aba40 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.769+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a0145850 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.770+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a01edfc0 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.770+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3590003a20 con 0x7f35a014e980 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.771+0000 7f35877fe700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 0x7f358c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:38.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.771+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f3590088530 con 0x7f35a014e980 2024-03-14T09:03:38.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.771+0000 7f359effd700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 0x7f358c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:38.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.771+0000 7f359effd700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 0x7f358c06ccf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3594000c00 tx=0x7f359401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:38.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:38.775+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f35a01edfc0 con 0x7f35a014e980 2024-03-14T09:03:39.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.017+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f35a01edf80 con 0x7f35a014e980 2024-03-14T09:03:39.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.625+0000 7f35877fe700 1 -- 172.21.15.154:0/56699108 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3590053630 con 0x7f35a014e980 2024-03-14T09:03:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 msgr2=0x7f358c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 0x7f358c06ccf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3594000c00 tx=0x7f359401c040 comp rx=0 tx=0).stop 2024-03-14T09:03:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 msgr2=0x7f35a01534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f359000c500 tx=0x7f359000c8c0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 shutdown_connections 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35a0158a80 0x7f35a01449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35a00c4a80 0x7f35a00aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f358c06a830 0x7f358c06ccf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 --2- 172.21.15.154:0/56699108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35a014e980 0x7f35a01534a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 >> 172.21.15.154:0/56699108 conn(0x7f35a00abcf0 msgr2=0x7f35a014d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 shutdown_connections 2024-03-14T09:03:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.628+0000 7f35a574d700 1 -- 172.21.15.154:0/56699108 wait complete. 2024-03-14T09:03:39.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T09:03:39.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.769+0000 7f91586cd700 1 Processor -- start 2024-03-14T09:03:39.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.769+0000 7f91586cd700 1 -- start start 2024-03-14T09:03:39.775 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f91586cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9150155060 0x7f91500c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f91586cd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915006b110 con 0x7f915006d050 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f91586cd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915006ab50 con 0x7f91501554e0 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f91586cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915006ae30 con 0x7f9150155060 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f9156469700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f915006d050 0x7f9150158750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37842/0 (socket says 172.21.15.154:37842) 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.770+0000 7f9156469700 1 -- 172.21.15.154:0/1157712598 learned_addr learned my addr 172.21.15.154:0/1157712598 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9150155060 0x7f91500c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610600674 0 0) 0x7f915006b110 con 0x7f915006d050 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f913c0035b0 con 0x7f915006d050 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175354935 0 0) 0x7f915006ab50 con 0x7f91501554e0 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915006b110 con 0x7f91501554e0 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619265473 0 0) 0x7f913c0035b0 con 0x7f915006d050 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9150155060 msgr2=0x7f91500c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:39.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9150155060 0x7f91500c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 >> v1:172.21.15.191:6789/0 conn(0x7f91501554e0 legacy=0x7f915015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150154860 con 0x7f915006d050 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9140001670 con 0x7f915006d050 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 >> v1:172.21.15.154:6789/0 conn(0x7f915006d050 legacy=0x7f9150158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c003bc0 0x7f913c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f913c004850 0x7f913c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f913c005210 0x7f913c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915006ae30 con 0x7f913c004850 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c00c0e0 con 0x7f913c005210 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c00c2a0 con 0x7f913c003bc0 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9150155060 0x7f91500c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9156469700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c003bc0 0x7f913c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f913c004850 0x7f913c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9156c6a700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f913c005210 0x7f913c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 -- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c003bc0 msgr2=0x7f913c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c003bc0 0x7f913c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.771+0000 7f9155c68700 1 -- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f913c005210 msgr2=0x7f913c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f913c005210 0x7f913c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155c68700 1 -- 172.21.15.154:0/1157712598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0035b0 con 0x7f913c004850 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155c68700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f913c004850 0x7f913c004cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9150158e60 tx=0x7f913c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55fafbf849b26cd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c004030 con 0x7f913c004850 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c004190 con 0x7f913c004850 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9155467700 1 -- 172.21.15.154:0/1157712598 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c004540 con 0x7f913c004850 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 -- 172.21.15.154:0/1157712598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f913c004850 msgr2=0x7f913c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 --2- 172.21.15.154:0/1157712598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f913c004850 0x7f913c004cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9150158e60 tx=0x7f913c005880 comp rx=0 tx=0).stop 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f9156c6a700 1 -- 172.21.15.154:0/1157712598 reap_dead start 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 -- 172.21.15.154:0/1157712598 shutdown_connections 2024-03-14T09:03:39.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 -- 172.21.15.154:0/1157712598 >> 172.21.15.154:0/1157712598 conn(0x7f91500c39f0 msgr2=0x7f91500c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 -- 172.21.15.154:0/1157712598 shutdown_connections 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.772+0000 7f91586cd700 1 -- 172.21.15.154:0/1157712598 wait complete. 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 Processor -- start 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 -- start start 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c004850 0x7f91500c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9150147aa0 0x7f91501e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501df220 con 0x7f91501e06e0 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91501dfd50 con 0x7f9150147aa0 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f91586cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9150147570 con 0x7f913c004850 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f9156c6a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f9156c6a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54414/0 (socket says 172.21.15.154:54414) 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.774+0000 7f9156c6a700 1 -- 172.21.15.154:0/1714056581 learned_addr learned my addr 172.21.15.154:0/1714056581 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9155c68700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9150147aa0 0x7f91501e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156469700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c004850 0x7f91500c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 -- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c004850 msgr2=0x7f91500c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c004850 0x7f91500c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 -- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9150147aa0 msgr2=0x7f91501e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9150147aa0 0x7f91501e01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 -- 172.21.15.154:0/1714056581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91501de860 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9156c6a700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f91500acd40 tx=0x7f914800d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800db10 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800dc70 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91501e0b60 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148003a00 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.775+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91501ddfc0 con 0x7f91501e06e0 2024-03-14T09:03:39.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.776+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91501dfec0 con 0x7f91501e06e0 2024-03-14T09:03:39.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.777+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9148003b60 con 0x7f91501e06e0 2024-03-14T09:03:39.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.777+0000 7f9146ffd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 0x7f913406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:39.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.777+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6651+0+0 (secure 0 0 0) 0x7f9148089410 con 0x7f91501e06e0 2024-03-14T09:03:39.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.777+0000 7f9156469700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 0x7f913406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:39.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.778+0000 7f9156469700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 0x7f913406ccf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9140001140 tx=0x7f9140001170 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:39.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.781+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f91501dfec0 con 0x7f91501e06e0 2024-03-14T09:03:39.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:39.976+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91501dfec0 con 0x7f91501e06e0 2024-03-14T09:03:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.627+0000 7f9146ffd700 1 -- 172.21.15.154:0/1714056581 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f91501dfec0 con 0x7f91501e06e0 2024-03-14T09:03:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 msgr2=0x7f913406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 0x7f913406ccf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9140001140 tx=0x7f9140001170 comp rx=0 tx=0).stop 2024-03-14T09:03:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 msgr2=0x7f9150146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f91500acd40 tx=0x7f914800d660 comp rx=0 tx=0).stop 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 shutdown_connections 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f913406a830 0x7f913406ccf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91501e06e0 0x7f9150146850 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f913c004850 0x7f91500c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 --2- 172.21.15.154:0/1714056581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9150147aa0 0x7f91501e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 >> 172.21.15.154:0/1714056581 conn(0x7f91500c39f0 msgr2=0x7f915014fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 shutdown_connections 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.630+0000 7f91586cd700 1 -- 172.21.15.154:0/1714056581 wait complete. 2024-03-14T09:03:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2024-03-14T09:03:40.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T09:03:40.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.774+0000 7fa6f7adc700 1 Processor -- start 2024-03-14T09:03:40.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.774+0000 7fa6f7adc700 1 -- start start 2024-03-14T09:03:40.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.774+0000 7fa6f7adc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6f00c4440 0x7fa6f00705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f7adc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa6f00ada40 0x7fa6f006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f7adc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f0156950 con 0x7fa6f00c2d20 2024-03-14T09:03:40.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f7adc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00c4820 con 0x7fa6f00c4440 2024-03-14T09:03:40.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f7adc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00c7800 con 0x7fa6f00ada40 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6f00c4440 0x7fa6f00705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6f00c4440 0x7fa6f00705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54418/0 (socket says 172.21.15.154:54418) 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 -- 172.21.15.154:0/1486668938 learned_addr learned my addr 172.21.15.154:0/1486668938 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 -- 172.21.15.154:0/1486668938 >> v2:172.21.15.191:3300/0 conn(0x7fa6f00ada40 msgr2=0x7fa6f006e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 --2- 172.21.15.154:0/1486668938 >> v2:172.21.15.191:3300/0 conn(0x7fa6f00ada40 0x7fa6f006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 -- 172.21.15.154:0/1486668938 >> v1:172.21.15.191:6789/0 conn(0x7fa6f00c2d20 legacy=0x7fa6f00c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 -- 172.21.15.154:0/1486668938 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0070af0 con 0x7fa6f00c4440 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f5878700 1 --2- 172.21.15.154:0/1486668938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f00705b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0070f60 tx=0x7fa6ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6cc392698648bf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f4876700 1 -- 172.21.15.154:0/1486668938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004230 con 0x7fa6f00c4440 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f4876700 1 -- 172.21.15.154:0/1486668938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec004390 con 0x7fa6f00c4440 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.775+0000 7fa6f4876700 1 -- 172.21.15.154:0/1486668938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0036a0 con 0x7fa6f00c4440 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 -- 172.21.15.154:0/1486668938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 msgr2=0x7fa6f00705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:40.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/1486668938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f00705b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0070f60 tx=0x7fa6ec009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 -- 172.21.15.154:0/1486668938 shutdown_connections 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/1486668938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f00705b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/1486668938 >> v2:172.21.15.191:3300/0 conn(0x7fa6f00ada40 0x7fa6f006e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 -- 172.21.15.154:0/1486668938 >> 172.21.15.154:0/1486668938 conn(0x7fa6f00a9d00 msgr2=0x7fa6f00aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 -- 172.21.15.154:0/1486668938 shutdown_connections 2024-03-14T09:03:40.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.776+0000 7fa6f7adc700 1 -- 172.21.15.154:0/1486668938 wait complete. 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.777+0000 7fa6f7adc700 1 Processor -- start 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 -- start start 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f00ada40 0x7fa6f01dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 0x7fa6f0154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01536c0 con 0x7fa6f00c4440 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01dc360 con 0x7fa6f00c2d20 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f7adc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01ec3f0 con 0x7fa6f00ada40 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.778+0000 7fa6f6079700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54430/0 (socket says 172.21.15.154:54430) 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 -- 172.21.15.154:0/2178896319 learned_addr learned my addr 172.21.15.154:0/2178896319 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:40.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 -- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f00ada40 msgr2=0x7fa6f01dccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f5878700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f00ada40 0x7fa6f01dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f5077700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 0x7fa6f0154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f00ada40 0x7fa6f01dccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 -- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 msgr2=0x7fa6f0154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 0x7fa6f0154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 -- 172.21.15.154:0/2178896319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009550 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f5077700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 0x7fa6f0154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f6079700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c7a0 tx=0x7fa6e800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003e10 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8005070 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f01e8450 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8005420 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.779+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f0154e60 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.780+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f01e8970 con 0x7fa6f00c4440 2024-03-14T09:03:40.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.780+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa6e8005580 con 0x7fa6f00c4440 2024-03-14T09:03:40.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.781+0000 7fa6e67fc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 0x7fa6d406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:40.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.781+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6213+0+0 (secure 0 0 0) 0x7fa6e80885e0 con 0x7fa6f00c4440 2024-03-14T09:03:40.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.781+0000 7fa6f5878700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 0x7fa6d406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:40.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.782+0000 7fa6f5878700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 0x7fa6d406ccf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00c5a70 tx=0x7fa6ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:40.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.785+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa6e8053890 con 0x7fa6f00c4440 2024-03-14T09:03:40.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:40.987+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa6f006eb30 con 0x7fa6f00c4440 2024-03-14T09:03:41.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.633+0000 7fa6e67fc700 1 -- 172.21.15.154:0/2178896319 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa6e80170d0 con 0x7fa6f00c4440 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 msgr2=0x7fa6d406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 0x7fa6d406ccf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00c5a70 tx=0x7fa6ec008030 comp rx=0 tx=0).stop 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 msgr2=0x7fa6f01e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c7a0 tx=0x7fa6e800cb60 comp rx=0 tx=0).stop 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 shutdown_connections 2024-03-14T09:03:41.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6d406a830 0x7fa6d406ccf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f00c4440 0x7fa6f01e8c50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f00ada40 0x7fa6f01dccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 --2- 172.21.15.154:0/2178896319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f00c2d20 0x7fa6f0154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 >> 172.21.15.154:0/2178896319 conn(0x7fa6f00a9d00 msgr2=0x7fa6f0155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 shutdown_connections 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.640+0000 7fa6f7adc700 1 -- 172.21.15.154:0/2178896319 wait complete. 2024-03-14T09:03:41.641 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2024-03-14T09:03:41.656 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.781+0000 7fc32a441700 1 Processor -- start 2024-03-14T09:03:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.781+0000 7fc32a441700 1 -- start start 2024-03-14T09:03:41.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc32a441700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc3240c1e90 0x7fc32413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc32a441700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32413ade0 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc32a441700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32406fb30 con 0x7fc3240c3250 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc32a441700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32406ec30 con 0x7fc3240c1e90 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc323fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc3240a8540 0x7fc324154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37858/0 (socket says 172.21.15.154:37858) 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.782+0000 7fc323fff700 1 -- 172.21.15.154:0/536204496 learned_addr learned my addr 172.21.15.154:0/536204496 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 --2- 172.21.15.154:0/536204496 >> v2:172.21.15.191:3300/0 conn(0x7fc3240c1e90 0x7fc32413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556534941 0 0) 0x7fc318001110 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30c0057c0 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4181758861 0 0) 0x7fc30c0057c0 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 >> v2:172.21.15.191:3300/0 conn(0x7fc3240c1e90 msgr2=0x7fc32413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 --2- 172.21.15.154:0/536204496 >> v2:172.21.15.191:3300/0 conn(0x7fc3240c1e90 0x7fc32413a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 >> v1:172.21.15.191:6789/0 conn(0x7fc3240c3250 legacy=0x7fc3240c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32406ef10 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fc318001800 con 0x7fc3240a8540 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 >> v1:172.21.15.154:6789/0 conn(0x7fc3240a8540 legacy=0x7fc324154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c005c10 0x7fc30c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c006a60 0x7fc30c006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 0x7fc30c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32406ec30 con 0x7fc30c007540 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c00e410 con 0x7fc30c006a60 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c00e5d0 con 0x7fc30c005c10 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 0x7fc30c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3237fe700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c006a60 0x7fc30c006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc323fff700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c005c10 0x7fc30c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 -- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c005c10 msgr2=0x7fc30c006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c005c10 0x7fc30c006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 -- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c006a60 msgr2=0x7fc30c006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc30c006a60 0x7fc30c006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 -- 172.21.15.154:0/536204496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0057c0 con 0x7fc30c007540 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.783+0000 7fc3289de700 1 -- 172.21.15.154:0/536204496 reap_dead start 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc3289de700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 0x7fc30c00dc20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009f90 tx=0x7fc30c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed1942a47684db4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003a20 con 0x7fc30c007540 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00f040 con 0x7fc30c007540 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc322ffd700 1 -- 172.21.15.154:0/536204496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003dc0 con 0x7fc30c007540 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 -- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 msgr2=0x7fc30c00dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 0x7fc30c00dc20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009f90 tx=0x7fc30c007a90 comp rx=0 tx=0).stop 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 -- 172.21.15.154:0/536204496 shutdown_connections 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 --2- 172.21.15.154:0/536204496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc30c007540 0x7fc30c00dc20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 -- 172.21.15.154:0/536204496 >> 172.21.15.154:0/536204496 conn(0x7fc3240c3880 msgr2=0x7fc3240c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 -- 172.21.15.154:0/536204496 shutdown_connections 2024-03-14T09:03:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.784+0000 7fc32a441700 1 -- 172.21.15.154:0/536204496 wait complete. 2024-03-14T09:03:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.786+0000 7fc32a441700 1 Processor -- start 2024-03-14T09:03:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.786+0000 7fc32a441700 1 -- start start 2024-03-14T09:03:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c007540 0x7fc324134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3241362f0 0x7fc324135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240aac40 con 0x7fc32414e600 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3241e6010 con 0x7fc3241362f0 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc32a441700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32414efc0 con 0x7fc30c007540 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54442/0 (socket says 172.21.15.154:54442) 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 -- 172.21.15.154:0/3020333079 learned_addr learned my addr 172.21.15.154:0/3020333079 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3237fe700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3241362f0 0x7fc324135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc323fff700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c007540 0x7fc324134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 -- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c007540 msgr2=0x7fc324134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c007540 0x7fc324134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 -- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3241362f0 msgr2=0x7fc324135ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3241362f0 0x7fc324135ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 -- 172.21.15.154:0/3020333079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3240aaee0 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc3289de700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc3240728f0 tx=0x7fc31c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.787+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c003e90 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.788+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c004230 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.788+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3240ab1c0 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.788+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c004600 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.788+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc324136000 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.789+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3240697d0 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.789+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc31c004760 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.789+0000 7fc320ff9700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 0x7fc31006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.789+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fc31c08a0a0 con 0x7fc32414e600 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.790+0000 7fc323fff700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 0x7fc31006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:41.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.790+0000 7fc323fff700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 0x7fc31006ccf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc318000ff0 tx=0x7fc318009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.794+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc31c0554f0 con 0x7fc32414e600 2024-03-14T09:03:41.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.987+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc32406d420 con 0x7fc32414e600 2024-03-14T09:03:41.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.988+0000 7fc320ff9700 1 -- 172.21.15.154:0/3020333079 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16283 (secure 0 0 0) 0x7fc31c058b10 con 0x7fc32414e600 2024-03-14T09:03:41.988 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:41.988 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":58,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:41.626557+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/1333349284":"2024-03-15T09:03:38.613073+0000","172.21.15.191:6836/1333349284":"2024-03-15T09:03:38.613073+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 msgr2=0x7fc31006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 0x7fc31006ccf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc318000ff0 tx=0x7fc318009040 comp rx=0 tx=0).stop 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 msgr2=0x7fc32414ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc3240728f0 tx=0x7fc31c004b40 comp rx=0 tx=0).stop 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 shutdown_connections 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc31006a830 0x7fc31006ccf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc32414e600 0x7fc32414ea80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc30c007540 0x7fc324134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 --2- 172.21.15.154:0/3020333079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3241362f0 0x7fc324135ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:41.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 >> 172.21.15.154:0/3020333079 conn(0x7fc3240c3880 msgr2=0x7fc3240c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:41.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 shutdown_connections 2024-03-14T09:03:41.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:41.990+0000 7fc32a441700 1 -- 172.21.15.154:0/3020333079 wait complete. 2024-03-14T09:03:42.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:03:11.026192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:03:11.722376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:42.003 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.133+0000 7fda8d5a3700 1 Processor -- start 2024-03-14T09:03:42.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.133+0000 7fda8d5a3700 1 -- start start 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda8d5a3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda880a9650 0x7fda880c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda8d5a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda8d5a3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda88156050 con 0x7fda881564e0 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda8d5a3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880a86f0 con 0x7fda880a9650 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda8d5a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c34b0 con 0x7fda88156960 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda877fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda880a9650 0x7fda880c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda880a9650 0x7fda880c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:34934/0 (socket says 172.21.15.154:34934) 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 -- 172.21.15.154:0/596130694 learned_addr learned my addr 172.21.15.154:0/596130694 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda877fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:58238/0 (socket says 172.21.15.154:58238) 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda867fc700 1 --1- 172.21.15.154:0/596130694 >> v1:172.21.15.191:6789/0 conn(0x7fda881564e0 0x7fda8806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:37404/0 (socket says 172.21.15.154:37404) 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 -- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 msgr2=0x7fda8815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 -- 172.21.15.154:0/596130694 >> v1:172.21.15.191:6789/0 conn(0x7fda881564e0 legacy=0x7fda8806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 -- 172.21.15.154:0/596130694 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda880a8990 con 0x7fda880a9650 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda877fe700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.134+0000 7fda86ffd700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda880c1780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda780098f0 tx=0x7fda78009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b061e43b1435218 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda85ffb700 1 -- 172.21.15.154:0/596130694 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78011830 con 0x7fda880a9650 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda85ffb700 1 -- 172.21.15.154:0/596130694 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78011990 con 0x7fda880a9650 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda85ffb700 1 -- 172.21.15.154:0/596130694 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78011d20 con 0x7fda880a9650 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 -- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 msgr2=0x7fda880c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda880c1780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda780098f0 tx=0x7fda78009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:42.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 -- 172.21.15.154:0/596130694 shutdown_connections 2024-03-14T09:03:42.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda88156960 0x7fda8815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/596130694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda880c1780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 -- 172.21.15.154:0/596130694 >> 172.21.15.154:0/596130694 conn(0x7fda880c3880 msgr2=0x7fda880c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:42.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 -- 172.21.15.154:0/596130694 shutdown_connections 2024-03-14T09:03:42.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.135+0000 7fda8d5a3700 1 -- 172.21.15.154:0/596130694 wait complete. 2024-03-14T09:03:42.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 Processor -- start 2024-03-14T09:03:42.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 -- start start 2024-03-14T09:03:42.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda881e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda881564e0 0x7fda881ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88154100 con 0x7fda88156960 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88154350 con 0x7fda880a9650 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda8d5a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda881eb8a0 con 0x7fda881564e0 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54458/0 (socket says 172.21.15.154:54458) 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.137+0000 7fda877fe700 1 -- 172.21.15.154:0/3433843789 learned_addr learned my addr 172.21.15.154:0/3433843789 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda867fc700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda881564e0 0x7fda881ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda86ffd700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda881e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 -- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda881564e0 msgr2=0x7fda881ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda881564e0 0x7fda881ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 -- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 msgr2=0x7fda881e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda881e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 -- 172.21.15.154:0/3433843789 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80009a70 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda877fe700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda8000efc0 tx=0x7fda8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ca90 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8000f040 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda78009550 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ce50 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.138+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda881e87d0 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.139+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda880697d0 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.140+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fda80014480 con 0x7fda88156960 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.140+0000 7fda6f7fe700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 0x7fda7406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.140+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fda800892f0 con 0x7fda88156960 2024-03-14T09:03:42.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.140+0000 7fda86ffd700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 0x7fda7406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.141+0000 7fda86ffd700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 0x7fda7406ccf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda78014040 tx=0x7fda78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.144+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fda80054740 con 0x7fda88156960 2024-03-14T09:03:42.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.335+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda88072a20 con 0x7fda88156960 2024-03-14T09:03:42.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.335+0000 7fda6f7fe700 1 -- 172.21.15.154:0/3433843789 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda80057d60 con 0x7fda88156960 2024-03-14T09:03:42.335 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 msgr2=0x7fda7406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 0x7fda7406ccf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda78014040 tx=0x7fda78008030 comp rx=0 tx=0).stop 2024-03-14T09:03:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 msgr2=0x7fda8814eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.337 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda8000efc0 tx=0x7fda8000c3a0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 shutdown_connections 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda7406a830 0x7fda7406ccf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda88156960 0x7fda8814eb80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda880a9650 0x7fda881e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 --2- 172.21.15.154:0/3433843789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda881564e0 0x7fda881ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 >> 172.21.15.154:0/3433843789 conn(0x7fda880c3880 msgr2=0x7fda88148a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 shutdown_connections 2024-03-14T09:03:42.338 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.337+0000 7fda8d5a3700 1 -- 172.21.15.154:0/3433843789 wait complete. 2024-03-14T09:03:42.349 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:03:42.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 Processor -- start 2024-03-14T09:03:42.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 -- start start 2024-03-14T09:03:42.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffba41553d0 0x7ffba4158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba4154f40 con 0x7ffba40bf9c0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba4158d80 con 0x7ffba40c1d50 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffbaac9d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40c16a0 con 0x7ffba41553d0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffba8a39700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffba41553d0 0x7ffba4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffba8a39700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffba41553d0 0x7ffba4158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54460/0 (socket says 172.21.15.154:54460) 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffba923a700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffba40bf9c0 0x7ffba415f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37868/0 (socket says 172.21.15.154:37868) 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.475+0000 7ffba8a39700 1 -- 172.21.15.154:0/1868302462 learned_addr learned my addr 172.21.15.154:0/1868302462 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba8a39700 1 -- 172.21.15.154:0/1868302462 >> v1:172.21.15.191:6789/0 conn(0x7ffba40c1d50 legacy=0x7ffba40c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba8a39700 1 -- 172.21.15.154:0/1868302462 >> v1:172.21.15.154:6789/0 conn(0x7ffba40bf9c0 legacy=0x7ffba415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba8a39700 1 -- 172.21.15.154:0/1868302462 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4154930 con 0x7ffba41553d0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba37fe700 1 -- 172.21.15.154:0/1868302462 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051003922 0 0) 0x7ffba4154f40 con 0x7ffba40bf9c0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba8a39700 1 --2- 172.21.15.154:0/1868302462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba41553d0 0x7ffba4158840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098f0 tx=0x7ffb98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfe5b1ac027925c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba37fe700 1 -- 172.21.15.154:0/1868302462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98004230 con 0x7ffba41553d0 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba37fe700 1 -- 172.21.15.154:0/1868302462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98004390 con 0x7ffba41553d0 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.476+0000 7ffba37fe700 1 -- 172.21.15.154:0/1868302462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980036a0 con 0x7ffba41553d0 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 -- 172.21.15.154:0/1868302462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba41553d0 msgr2=0x7ffba4158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/1868302462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba41553d0 0x7ffba4158840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098f0 tx=0x7ffb98009c00 comp rx=0 tx=0).stop 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 -- 172.21.15.154:0/1868302462 shutdown_connections 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/1868302462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba41553d0 0x7ffba4158840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 -- 172.21.15.154:0/1868302462 >> 172.21.15.154:0/1868302462 conn(0x7ffba406a770 msgr2=0x7ffba406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 -- 172.21.15.154:0/1868302462 shutdown_connections 2024-03-14T09:03:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.477+0000 7ffbaac9d700 1 -- 172.21.15.154:0/1868302462 wait complete. 2024-03-14T09:03:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.478+0000 7ffbaac9d700 1 Processor -- start 2024-03-14T09:03:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.478+0000 7ffbaac9d700 1 -- start start 2024-03-14T09:03:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 0x7ffba41de780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffba41553d0 0x7ffba414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41df040 con 0x7ffba40c1d50 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba414f820 con 0x7ffba40bf9c0 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffbaac9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41ed5f0 con 0x7ffba41553d0 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54470/0 (socket says 172.21.15.154:54470) 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffba3fff700 1 -- 172.21.15.154:0/432139437 learned_addr learned my addr 172.21.15.154:0/432139437 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffba8a39700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 0x7ffba41de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.479+0000 7ffba923a700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffba41553d0 0x7ffba414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 -- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffba41553d0 msgr2=0x7ffba414f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffba41553d0 0x7ffba414f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 -- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 msgr2=0x7ffba41de780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 0x7ffba41de780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 -- 172.21.15.154:0/432139437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb98009550 con 0x7ffba40c1d50 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba8a39700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 0x7ffba41de780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba3fff700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c4d0 tx=0x7ffb9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011790 con 0x7ffba40c1d50 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb900118f0 con 0x7ffba40c1d50 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4151560 con 0x7ffba40c1d50 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011ca0 con 0x7ffba40c1d50 2024-03-14T09:03:42.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.480+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba41edb20 con 0x7ffba40c1d50 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.481+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba40697d0 con 0x7ffba40c1d50 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.481+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffb9001c460 con 0x7ffba40c1d50 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.482+0000 7ffba17fa700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 0x7ffb8806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.482+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7ffb90088b70 con 0x7ffba40c1d50 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.482+0000 7ffba8a39700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 0x7ffb8806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.482+0000 7ffba8a39700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 0x7ffb8806ccf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009130 tx=0x7ffb98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.486+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffb90053fc0 con 0x7ffba40c1d50 2024-03-14T09:03:42.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.680+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffba4066550 con 0x7ffba40c1d50 2024-03-14T09:03:42.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.681+0000 7ffba17fa700 1 -- 172.21.15.154:0/432139437 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v58) v1 ==== 105+0+11 (secure 0 0 0) 0x7ffb900575e0 con 0x7ffba40c1d50 2024-03-14T09:03:42.681 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 msgr2=0x7ffb8806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 0x7ffb8806ccf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009130 tx=0x7ffb98008030 comp rx=0 tx=0).stop 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 msgr2=0x7ffba414eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c4d0 tx=0x7ffb9000c890 comp rx=0 tx=0).stop 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 shutdown_connections 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffba41553d0 0x7ffba414f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffba40bf9c0 0x7ffba41de780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb8806a830 0x7ffb8806ccf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 --2- 172.21.15.154:0/432139437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffba40c1d50 0x7ffba414eda0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 >> 172.21.15.154:0/432139437 conn(0x7ffba406a770 msgr2=0x7ffba40c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:42.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 shutdown_connections 2024-03-14T09:03:42.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.683+0000 7ffbaac9d700 1 -- 172.21.15.154:0/432139437 wait complete. 2024-03-14T09:03:42.695 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:03:42.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.820+0000 7fb214218700 1 Processor -- start 2024-03-14T09:03:42.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.820+0000 7fb214218700 1 -- start start 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb214218700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 0x7fb20c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb214218700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb20c15d270 0x7fb20c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb214218700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c0ac8e0 con 0x7fb20c14eaf0 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb214218700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c06b310 con 0x7fb20c15d270 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb214218700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c06de30 con 0x7fb20c155df0 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb20c15d270 0x7fb20c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb20c15d270 0x7fb20c15f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54472/0 (socket says 172.21.15.154:54472) 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 -- 172.21.15.154:0/2789644791 learned_addr learned my addr 172.21.15.154:0/2789644791 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2117b3700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 0x7fb20c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 -- 172.21.15.154:0/2789644791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 msgr2=0x7fb20c156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 0x7fb20c156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 -- 172.21.15.154:0/2789644791 >> v1:172.21.15.191:6789/0 conn(0x7fb20c14eaf0 legacy=0x7fb20c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 -- 172.21.15.154:0/2789644791 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0acb30 con 0x7fb20c15d270 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2117b3700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 0x7fb20c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb2127b5700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c15d270 0x7fb20c15f660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb20400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fe8cad5193ea84 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.821+0000 7fb210fb2700 1 -- 172.21.15.154:0/2789644791 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400e070 con 0x7fb20c15d270 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb210fb2700 1 -- 172.21.15.154:0/2789644791 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204003a20 con 0x7fb20c15d270 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb210fb2700 1 -- 172.21.15.154:0/2789644791 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204003dd0 con 0x7fb20c15d270 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 -- 172.21.15.154:0/2789644791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c15d270 msgr2=0x7fb20c15f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c15d270 0x7fb20c15f660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb20400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 -- 172.21.15.154:0/2789644791 shutdown_connections 2024-03-14T09:03:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c15d270 0x7fb20c15f660 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 --2- 172.21.15.154:0/2789644791 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c155df0 0x7fb20c156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 -- 172.21.15.154:0/2789644791 >> 172.21.15.154:0/2789644791 conn(0x7fb20c0c3880 msgr2=0x7fb20c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 -- 172.21.15.154:0/2789644791 shutdown_connections 2024-03-14T09:03:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.822+0000 7fb214218700 1 -- 172.21.15.154:0/2789644791 wait complete. 2024-03-14T09:03:42.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.823+0000 7fb214218700 1 Processor -- start 2024-03-14T09:03:42.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.823+0000 7fb214218700 1 -- start start 2024-03-14T09:03:42.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20c14eaf0 0x7fb20c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c15d270 0x7fb20c1ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c14c5f0 con 0x7fb20c155df0 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c13f3a0 con 0x7fb20c14eaf0 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb214218700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c14c310 con 0x7fb20c15d270 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2127b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c15d270 0x7fb20c1ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54478/0 (socket says 172.21.15.154:54478) 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 -- 172.21.15.154:0/195055601 learned_addr learned my addr 172.21.15.154:0/195055601 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:42.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb211fb4700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20c14eaf0 0x7fb20c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 -- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c15d270 msgr2=0x7fb20c1ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c15d270 0x7fb20c1ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 -- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20c14eaf0 msgr2=0x7fb20c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20c14eaf0 0x7fb20c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.824+0000 7fb2117b3700 1 -- 172.21.15.154:0/195055601 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009b30 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb2117b3700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c750 tx=0x7fb20800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208011800 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb208011960 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c14a2f0 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208011cd0 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.825+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c14dc70 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.826+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c14a530 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.826+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb208021350 con 0x7fb20c155df0 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.826+0000 7fb2027fc700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 0x7fb1f006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.827+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fb208088c00 con 0x7fb20c155df0 2024-03-14T09:03:42.829 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.827+0000 7fb211fb4700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 0x7fb1f006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:42.829 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.827+0000 7fb211fb4700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 0x7fb1f006ccf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc007fd0 tx=0x7fb1fc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:42.830 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:42.831+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb20c14a530 con 0x7fb20c155df0 2024-03-14T09:03:43.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.022+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fb20c14a530 con 0x7fb20c155df0 2024-03-14T09:03:43.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.023+0000 7fb2027fc700 1 -- 172.21.15.154:0/195055601 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v58) v1 ==== 101+0+11 (secure 0 0 0) 0x7fb20c14a530 con 0x7fb20c155df0 2024-03-14T09:03:43.023 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 msgr2=0x7fb1f006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 0x7fb1f006ccf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc007fd0 tx=0x7fb1fc007ec0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 msgr2=0x7fb20c14a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c750 tx=0x7fb20800ca60 comp rx=0 tx=0).stop 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 shutdown_connections 2024-03-14T09:03:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb20c15d270 0x7fb20c1ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb20c14eaf0 0x7fb20c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb1f006a830 0x7fb1f006ccf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 --2- 172.21.15.154:0/195055601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c155df0 0x7fb20c14a710 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 >> 172.21.15.154:0/195055601 conn(0x7fb20c0c3880 msgr2=0x7fb20c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 shutdown_connections 2024-03-14T09:03:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.025+0000 7fb214218700 1 -- 172.21.15.154:0/195055601 wait complete. 2024-03-14T09:03:43.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.159+0000 7fabdc62b700 1 Processor -- start 2024-03-14T09:03:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.159+0000 7fabdc62b700 1 -- start start 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fabd414c5c0 0x7fabd4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd4072290 0x7fabd4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fabd40c1fd0 0x7fabd40c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40c1bb0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40c6170 con 0x7fabd414c5c0 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdc62b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd4070e10 con 0x7fabd4072290 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fabd40c1fd0 0x7fabd40c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fabd40c1fd0 0x7fabd40c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54490/0 (socket says 172.21.15.154:54490) 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 -- 172.21.15.154:0/630286473 learned_addr learned my addr 172.21.15.154:0/630286473 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabd9bc6700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd4072290 0x7fabd4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 -- 172.21.15.154:0/630286473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd4072290 msgr2=0x7fabd4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabda3c7700 1 --2- 172.21.15.154:0/630286473 >> v2:172.21.15.191:3300/0 conn(0x7fabd414c5c0 0x7fabd4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd4072290 0x7fabd4072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 -- 172.21.15.154:0/630286473 >> v2:172.21.15.191:3300/0 conn(0x7fabd414c5c0 msgr2=0x7fabd4152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 --2- 172.21.15.154:0/630286473 >> v2:172.21.15.191:3300/0 conn(0x7fabd414c5c0 0x7fabd4152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.160+0000 7fabdabc8700 1 -- 172.21.15.154:0/630286473 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd414c9a0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdabc8700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd40c2460 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fabd40a9340 tx=0x7fabcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb0476abd828337 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabd93c5700 1 -- 172.21.15.154:0/630286473 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00e070 con 0x7fabd40c1fd0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabd93c5700 1 -- 172.21.15.154:0/630286473 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc00b9c0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabd93c5700 1 -- 172.21.15.154:0/630286473 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00bd70 con 0x7fabd40c1fd0 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 -- 172.21.15.154:0/630286473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 msgr2=0x7fabd40c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd40c2460 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fabd40a9340 tx=0x7fabcc00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 -- 172.21.15.154:0/630286473 shutdown_connections 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd40c2460 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 --2- 172.21.15.154:0/630286473 >> v2:172.21.15.191:3300/0 conn(0x7fabd414c5c0 0x7fabd4152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 --2- 172.21.15.154:0/630286473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd4072290 0x7fabd4072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 -- 172.21.15.154:0/630286473 >> 172.21.15.154:0/630286473 conn(0x7fabd406a770 msgr2=0x7fabd406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 -- 172.21.15.154:0/630286473 shutdown_connections 2024-03-14T09:03:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.161+0000 7fabdc62b700 1 -- 172.21.15.154:0/630286473 wait complete. 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 Processor -- start 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 -- start start 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 0x7fabd4155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 0x7fabd41eb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd414d440 con 0x7fabd40c1fd0 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd414ceb0 con 0x7fabd4072290 2024-03-14T09:03:43.167 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdc62b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40ac620 con 0x7fabd414c5c0 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabd9bc6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabd9bc6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54494/0 (socket says 172.21.15.154:54494) 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabd9bc6700 1 -- 172.21.15.154:0/4188200615 learned_addr learned my addr 172.21.15.154:0/4188200615 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabda3c7700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 0x7fabd4155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.163+0000 7fabdabc8700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 0x7fabd41eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 -- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 msgr2=0x7fabd41eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 0x7fabd41eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 -- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 msgr2=0x7fabd4155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 0x7fabd4155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 -- 172.21.15.154:0/4188200615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc009b30 con 0x7fabd40c1fd0 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabda3c7700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 0x7fabd4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabdabc8700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 0x7fabd41eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabd9bc6700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c7f0 tx=0x7fabd000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011800 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd0011960 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd4152750 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011d10 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.164+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd40ac2b0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.165+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd40697d0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.165+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fabd0018360 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.166+0000 7fabcaffd700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 0x7fabb806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.166+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fabd0088d20 con 0x7fabd40c1fd0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.166+0000 7fabda3c7700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 0x7fabb806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.167+0000 7fabda3c7700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 0x7fabb806ccf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fabc400a9b0 tx=0x7fabc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.170+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fabd00540e0 con 0x7fabd40c1fd0 2024-03-14T09:03:43.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.405+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd414d580 con 0x7fabd40c1fd0 2024-03-14T09:03:43.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.406+0000 7fabcaffd700 1 -- 172.21.15.154:0/4188200615 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4750 (secure 0 0 0) 0x7fabd001f090 con 0x7fabd40c1fd0 2024-03-14T09:03:43.406 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:43.406 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3880316034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3880316034},{"type":"v1","addr":"172.21.15.191:6837","nonce":3880316034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 msgr2=0x7fabb806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 0x7fabb806ccf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fabc400a9b0 tx=0x7fabc4008040 comp rx=0 tx=0).stop 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 msgr2=0x7fabd41eb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c7f0 tx=0x7fabd000cbb0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 shutdown_connections 2024-03-14T09:03:43.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabd414c5c0 0x7fabd41eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabd4072290 0x7fabd4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fabb806a830 0x7fabb806ccf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 --2- 172.21.15.154:0/4188200615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fabd40c1fd0 0x7fabd41eb000 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 >> 172.21.15.154:0/4188200615 conn(0x7fabd406a770 msgr2=0x7fabd414fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 shutdown_connections 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.408+0000 7fabdc62b700 1 -- 172.21.15.154:0/4188200615 wait complete. 2024-03-14T09:03:43.409 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 37 2024-03-14T09:03:43.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:43.541 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 Processor -- start 2024-03-14T09:03:43.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 -- start start 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98cc0702f0 0x7f98cc0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f98cc14fbd0 0x7f98cc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0ac7b0 con 0x7f98cc0702f0 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0acd10 con 0x7f98cc14fbd0 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.542+0000 7f98d2f3b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc150050 con 0x7f98cc0c78c0 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98cc0702f0 0x7f98cc0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98cc0702f0 0x7f98cc0c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54508/0 (socket says 172.21.15.154:54508) 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 -- 172.21.15.154:0/539128231 learned_addr learned my addr 172.21.15.154:0/539128231 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d14d8700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.544 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98cbfff700 1 --2- 172.21.15.154:0/539128231 >> v2:172.21.15.191:3300/0 conn(0x7f98cc14fbd0 0x7f98cc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 -- 172.21.15.154:0/539128231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 msgr2=0x7f98cc0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 -- 172.21.15.154:0/539128231 >> v2:172.21.15.191:3300/0 conn(0x7f98cc14fbd0 msgr2=0x7f98cc0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/539128231 >> v2:172.21.15.191:3300/0 conn(0x7f98cc14fbd0 0x7f98cc0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 -- 172.21.15.154:0/539128231 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0ac9c0 con 0x7f98cc0702f0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c4380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c5980 tx=0x7f98c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd5495ee470cec server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98cb7fe700 1 -- 172.21.15.154:0/539128231 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0011830 con 0x7f98cc0702f0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98cb7fe700 1 -- 172.21.15.154:0/539128231 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0011990 con 0x7f98cc0702f0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.543+0000 7f98cb7fe700 1 -- 172.21.15.154:0/539128231 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0011d20 con 0x7f98cc0702f0 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 -- 172.21.15.154:0/539128231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 msgr2=0x7f98cc0c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c4380 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0c5980 tx=0x7f98c0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 -- 172.21.15.154:0/539128231 shutdown_connections 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/539128231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c4380 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.545 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/539128231 >> v2:172.21.15.191:3300/0 conn(0x7f98cc14fbd0 0x7f98cc0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 -- 172.21.15.154:0/539128231 >> 172.21.15.154:0/539128231 conn(0x7f98cc0a9bf0 msgr2=0x7f98cc0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 -- 172.21.15.154:0/539128231 shutdown_connections 2024-03-14T09:03:43.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.544+0000 7f98d2f3b700 1 -- 172.21.15.154:0/539128231 wait complete. 2024-03-14T09:03:43.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.545+0000 7f98d2f3b700 1 Processor -- start 2024-03-14T09:03:43.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.545+0000 7f98d2f3b700 1 -- start start 2024-03-14T09:03:43.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98cc14fbd0 0x7f98cc151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc144f80 con 0x7f98cc0702f0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1eaaa0 con 0x7f98cc14fbd0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d2f3b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc145160 con 0x7f98cc0c78c0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54518/0 (socket says 172.21.15.154:54518) 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 -- 172.21.15.154:0/4007900514 learned_addr learned my addr 172.21.15.154:0/4007900514 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98cbfff700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d14d8700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98cc14fbd0 0x7f98cc151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 -- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 msgr2=0x7f98cc151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 -- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98cc14fbd0 msgr2=0x7f98cc151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98cc14fbd0 0x7f98cc151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 -- 172.21.15.154:0/4007900514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009550 con 0x7f98cc0702f0 2024-03-14T09:03:43.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.546+0000 7f98d0cd7700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009c10 tx=0x7f98c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.547+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004450 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.547+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c001e070 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.547+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc1eacf0 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.547+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00038e0 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.547+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc145ab0 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.548+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc151670 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.548+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f98c0003a40 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.548+0000 7f98c97fa700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 0x7f98b006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.549+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f98c0089a60 con 0x7f98cc0702f0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.549+0000 7f98cbfff700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 0x7f98b006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.549+0000 7f98cbfff700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 0x7f98b006ccf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f98cc06da60 tx=0x7f98b8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.553+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f98c0054eb0 con 0x7f98cc0702f0 2024-03-14T09:03:43.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.749+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98cc0697d0 con 0x7f98cc0702f0 2024-03-14T09:03:43.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.750+0000 7f98c97fa700 1 -- 172.21.15.154:0/4007900514 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16283 (secure 0 0 0) 0x7f98c0015090 con 0x7f98cc0702f0 2024-03-14T09:03:43.750 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:43.750 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":58,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:41.626557+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/1333349284":"2024-03-15T09:03:38.613073+0000","172.21.15.191:6836/1333349284":"2024-03-15T09:03:38.613073+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 msgr2=0x7f98b006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 0x7f98b006ccf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f98cc06da60 tx=0x7f98b8009210 comp rx=0 tx=0).stop 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 msgr2=0x7f98cc0c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009c10 tx=0x7f98c0004750 comp rx=0 tx=0).stop 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 shutdown_connections 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98cc14fbd0 0x7f98cc151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f98b006a830 0x7f98b006ccf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98cc0702f0 0x7f98cc0c14e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 --2- 172.21.15.154:0/4007900514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98cc0c78c0 0x7f98cc151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 >> 172.21.15.154:0/4007900514 conn(0x7f98cc0a9bf0 msgr2=0x7f98cc0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 shutdown_connections 2024-03-14T09:03:43.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.752+0000 7f98d2f3b700 1 -- 172.21.15.154:0/4007900514 wait complete. 2024-03-14T09:03:43.764 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-14T09:03:43.765 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:43.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.886+0000 7fa2481f5700 1 Processor -- start 2024-03-14T09:03:43.886 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.886+0000 7fa2481f5700 1 -- start start 2024-03-14T09:03:43.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa2481f5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa2400a8540 0x7fa24013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa2481f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa2400c1780 0x7fa2400c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa2481f5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24013ade0 con 0x7fa2400c3250 2024-03-14T09:03:43.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa2481f5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24006fb30 con 0x7fa2400c1780 2024-03-14T09:03:43.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa2481f5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24006ec30 con 0x7fa2400a8540 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa2400c1780 0x7fa2400c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa245790700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa2400c3250 0x7fa240154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37878/0 (socket says 172.21.15.154:37878) 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa245790700 1 -- 172.21.15.154:0/3483522368 learned_addr learned my addr 172.21.15.154:0/3483522368 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 -- 172.21.15.154:0/3483522368 >> v2:172.21.15.191:3300/0 conn(0x7fa2400a8540 msgr2=0x7fa24013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 --2- 172.21.15.154:0/3483522368 >> v2:172.21.15.191:3300/0 conn(0x7fa2400a8540 0x7fa24013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 -- 172.21.15.154:0/3483522368 >> v1:172.21.15.154:6789/0 conn(0x7fa2400c3250 legacy=0x7fa240154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 -- 172.21.15.154:0/3483522368 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24006ef10 con 0x7fa2400c1780 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa244f8f700 1 -- 172.21.15.154:0/3483522368 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714234695 0 0) 0x7fa230001110 con 0x7fa2400c3250 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa246792700 1 --2- 172.21.15.154:0/3483522368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c1780 0x7fa2400c1c10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa240070da0 tx=0x7fa23800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da1f1c1cf9fdaf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.887+0000 7fa244f8f700 1 -- 172.21.15.154:0/3483522368 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23800e070 con 0x7fa2400c1780 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa244f8f700 1 -- 172.21.15.154:0/3483522368 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23800bbd0 con 0x7fa2400c1780 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa244f8f700 1 -- 172.21.15.154:0/3483522368 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238012430 con 0x7fa2400c1780 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 -- 172.21.15.154:0/3483522368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c1780 msgr2=0x7fa2400c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.889 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 --2- 172.21.15.154:0/3483522368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c1780 0x7fa2400c1c10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa240070da0 tx=0x7fa23800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 -- 172.21.15.154:0/3483522368 shutdown_connections 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 --2- 172.21.15.154:0/3483522368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c1780 0x7fa2400c1c10 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 --2- 172.21.15.154:0/3483522368 >> v2:172.21.15.191:3300/0 conn(0x7fa2400a8540 0x7fa24013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 -- 172.21.15.154:0/3483522368 >> 172.21.15.154:0/3483522368 conn(0x7fa2400c3880 msgr2=0x7fa2400c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 -- 172.21.15.154:0/3483522368 shutdown_connections 2024-03-14T09:03:43.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.888+0000 7fa2481f5700 1 -- 172.21.15.154:0/3483522368 wait complete. 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.889+0000 7fa2481f5700 1 Processor -- start 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.889+0000 7fa2481f5700 1 -- start start 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2400a8540 0x7fa2400ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 0x7fa2401e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401e82a0 con 0x7fa2400c3250 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24012e5c0 con 0x7fa2400c1780 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa2481f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401ebc40 con 0x7fa2400a8540 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54532/0 (socket says 172.21.15.154:54532) 2024-03-14T09:03:43.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 -- 172.21.15.154:0/2531842947 learned_addr learned my addr 172.21.15.154:0/2531842947 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa245f91700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2400a8540 0x7fa2400ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa245790700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 0x7fa2401e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 -- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2400a8540 msgr2=0x7fa2400ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2400a8540 0x7fa2400ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 -- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 msgr2=0x7fa2401e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 0x7fa2401e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa246792700 1 -- 172.21.15.154:0/2531842947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238009a70 con 0x7fa2400c3250 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.890+0000 7fa245790700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 0x7fa2401e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa246792700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa23800b370 tx=0x7fa238004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238012430 con 0x7fa2400c3250 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa238020070 con 0x7fa2400c3250 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2401e8000 con 0x7fa2400c3250 2024-03-14T09:03:43.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238003ea0 con 0x7fa2400c3250 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.891+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2401eb8d0 con 0x7fa2400c3250 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.892+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2400697d0 con 0x7fa2400c3250 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.892+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa238004280 con 0x7fa2400c3250 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.892+0000 7fa2367fc700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 0x7fa22406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.893+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fa238022070 con 0x7fa2400c3250 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.893+0000 7fa245f91700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 0x7fa22406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:43.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.893+0000 7fa245f91700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 0x7fa22406ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa23c006fd0 tx=0x7fa23c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:43.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:43.897+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa238056090 con 0x7fa2400c3250 2024-03-14T09:03:44.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.133+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa24006d420 con 0x7fa2400c3250 2024-03-14T09:03:44.133 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.133+0000 7fa2367fc700 1 -- 172.21.15.154:0/2531842947 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4750 (secure 0 0 0) 0x7fa23802c020 con 0x7fa2400c3250 2024-03-14T09:03:44.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:44.133 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3880316034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3880316034},{"type":"v1","addr":"172.21.15.191:6837","nonce":3880316034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.135+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 msgr2=0x7fa22406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.135+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 0x7fa22406ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa23c006fd0 tx=0x7fa23c008040 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.135+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 msgr2=0x7fa24006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.135+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa23800b370 tx=0x7fa238004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 shutdown_connections 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa22406a830 0x7fa22406ccf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2400c3250 0x7fa24006a770 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2400a8540 0x7fa2400ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 --2- 172.21.15.154:0/2531842947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2400c1780 0x7fa2401e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 >> 172.21.15.154:0/2531842947 conn(0x7fa2400c3880 msgr2=0x7fa240134ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 shutdown_connections 2024-03-14T09:03:44.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.136+0000 7fa2481f5700 1 -- 172.21.15.154:0/2531842947 wait complete. 2024-03-14T09:03:44.137 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 37 2024-03-14T09:03:44.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:44.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 Processor -- start 2024-03-14T09:03:44.269 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 -- start start 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3b4c0c3250 0x7f3b4c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c13ade0 con 0x7f3b4c0c1780 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c06fb30 con 0x7f3b4c0a8540 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.269+0000 7f3b51ae3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c06ec30 con 0x7f3b4c0c3250 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3b4c0c1780 0x7f3b4c0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37882/0 (socket says 172.21.15.154:37882) 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4bfff700 1 -- 172.21.15.154:0/2054013817 learned_addr learned my addr 172.21.15.154:0/2054013817 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4affd700 1 --2- 172.21.15.154:0/2054013817 >> v2:172.21.15.191:3300/0 conn(0x7f3b4c0c3250 0x7f3b4c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4a7fc700 1 -- 172.21.15.154:0/2054013817 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912637489 0 0) 0x7f3b44001490 con 0x7f3b4c0c1780 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4a7fc700 1 -- 172.21.15.154:0/2054013817 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b340035b0 con 0x7f3b4c0c1780 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4affd700 1 -- 172.21.15.154:0/2054013817 >> v1:172.21.15.191:6789/0 conn(0x7f3b4c0a8540 legacy=0x7f3b4c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4affd700 1 -- 172.21.15.154:0/2054013817 >> v1:172.21.15.154:6789/0 conn(0x7f3b4c0c1780 legacy=0x7f3b4c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4affd700 1 -- 172.21.15.154:0/2054013817 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c06ef10 con 0x7f3b4c0c3250 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4affd700 1 --2- 172.21.15.154:0/2054013817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c13a8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009840 tx=0x7f3b40009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700aa245541e33c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4a7fc700 1 -- 172.21.15.154:0/2054013817 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40011830 con 0x7f3b4c0c3250 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4a7fc700 1 -- 172.21.15.154:0/2054013817 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40011990 con 0x7f3b4c0c3250 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.270+0000 7f3b4a7fc700 1 -- 172.21.15.154:0/2054013817 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40011d20 con 0x7f3b4c0c3250 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 -- 172.21.15.154:0/2054013817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 msgr2=0x7f3b4c13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/2054013817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c13a8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009840 tx=0x7f3b40009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 -- 172.21.15.154:0/2054013817 shutdown_connections 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/2054013817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c13a8a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 -- 172.21.15.154:0/2054013817 >> 172.21.15.154:0/2054013817 conn(0x7f3b4c0c3880 msgr2=0x7f3b4c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 -- 172.21.15.154:0/2054013817 shutdown_connections 2024-03-14T09:03:44.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.271+0000 7f3b51ae3700 1 -- 172.21.15.154:0/2054013817 wait complete. 2024-03-14T09:03:44.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.272+0000 7f3b51ae3700 1 Processor -- start 2024-03-14T09:03:44.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.272+0000 7f3b51ae3700 1 -- start start 2024-03-14T09:03:44.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b4c0c1780 0x7f3b4c136b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c0ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c152500 con 0x7f3b4c0a8540 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0abed0 con 0x7f3b4c0c3250 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b51ae3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0ab8e0 con 0x7f3b4c0c1780 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54542/0 (socket says 172.21.15.154:54542) 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 -- 172.21.15.154:0/3550338481 learned_addr learned my addr 172.21.15.154:0/3550338481 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4affd700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b4c0c1780 0x7f3b4c136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4bfff700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c0ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 -- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b4c0c1780 msgr2=0x7f3b4c136b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b4c0c1780 0x7f3b4c136b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 -- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 msgr2=0x7f3b4c0ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c0ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4b7fe700 1 -- 172.21.15.154:0/3550338481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b400094a0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.273+0000 7f3b4bfff700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c0ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b4b7fe700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c480 tx=0x7f3b3c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0052d0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c005430 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c151f40 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0038c0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.274+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c137080 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.275+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c0697d0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.275+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3b3c003a20 con 0x7f3b4c0a8540 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.276+0000 7f3b33fff700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 0x7f3b3806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.276+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f3b3c088080 con 0x7f3b4c0a8540 2024-03-14T09:03:44.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.276+0000 7f3b4affd700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 0x7f3b3806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.276+0000 7f3b4affd700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 0x7f3b3806ccf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b400090b0 tx=0x7f3b40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.280+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3b3c0534d0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.520+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b4c06d420 con 0x7f3b4c0a8540 2024-03-14T09:03:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.520+0000 7f3b33fff700 1 -- 172.21.15.154:0/3550338481 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f3b3c056af0 con 0x7f3b4c0a8540 2024-03-14T09:03:44.520 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:44.520 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3880316034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3880316034},{"type":"v1","addr":"172.21.15.191:6837","nonce":3880316034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:12.760442+0000","modified":"2024-03-14T09:03:15.772158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6835/2685722593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":2685722593},{"type":"v1","addr":"172.21.15.154:6835","nonce":2685722593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 msgr2=0x7f3b3806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 0x7f3b3806ccf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3b400090b0 tx=0x7f3b40008030 comp rx=0 tx=0).stop 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 msgr2=0x7f3b4c0ab3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c480 tx=0x7f3b3c00c790 comp rx=0 tx=0).stop 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 shutdown_connections 2024-03-14T09:03:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b4c0c3250 0x7f3b4c0ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b3806a830 0x7f3b3806ccf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4c0a8540 0x7f3b4c0ab3a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 --2- 172.21.15.154:0/3550338481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b4c0c1780 0x7f3b4c136b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 >> 172.21.15.154:0/3550338481 conn(0x7f3b4c0c3880 msgr2=0x7f3b4c138af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 shutdown_connections 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.522+0000 7f3b51ae3700 1 -- 172.21.15.154:0/3550338481 wait complete. 2024-03-14T09:03:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 37 2024-03-14T09:03:44.534 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:44.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.657+0000 7f50e7c59700 1 Processor -- start 2024-03-14T09:03:44.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 -- start start 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f50e00c7f70 0x7f50e00c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50e00c8350 0x7f50e0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e0157e80 con 0x7f50e00c4f30 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e0158ac0 con 0x7f50e00c8350 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e7c59700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e0158120 con 0x7f50e00c7f70 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e61f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50e00c8350 0x7f50e0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e61f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50e00c8350 0x7f50e0154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54548/0 (socket says 172.21.15.154:54548) 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.658+0000 7f50e61f6700 1 -- 172.21.15.154:0/593203982 learned_addr learned my addr 172.21.15.154:0/593203982 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e61f6700 1 -- 172.21.15.154:0/593203982 >> v2:172.21.15.191:3300/0 conn(0x7f50e00c7f70 msgr2=0x7f50e00c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e61f6700 1 --2- 172.21.15.154:0/593203982 >> v2:172.21.15.191:3300/0 conn(0x7f50e00c7f70 0x7f50e00c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e61f6700 1 -- 172.21.15.154:0/593203982 >> v1:172.21.15.191:6789/0 conn(0x7f50e00c4f30 legacy=0x7f50e01547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e61f6700 1 -- 172.21.15.154:0/593203982 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0158400 con 0x7f50e00c8350 2024-03-14T09:03:44.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e61f6700 1 --2- 172.21.15.154:0/593203982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e0154f00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f50e00ca350 tx=0x7f50d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e083f1a06acc70ae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e49f3700 1 -- 172.21.15.154:0/593203982 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d800e070 con 0x7f50e00c8350 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e49f3700 1 -- 172.21.15.154:0/593203982 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8003a20 con 0x7f50e00c8350 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e49f3700 1 -- 172.21.15.154:0/593203982 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8003dd0 con 0x7f50e00c8350 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.659+0000 7f50e7c59700 1 -- 172.21.15.154:0/593203982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 msgr2=0x7f50e0154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 --2- 172.21.15.154:0/593203982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e0154f00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f50e00ca350 tx=0x7f50d800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 -- 172.21.15.154:0/593203982 shutdown_connections 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 --2- 172.21.15.154:0/593203982 >> v2:172.21.15.191:3300/0 conn(0x7f50e00c7f70 0x7f50e00c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 --2- 172.21.15.154:0/593203982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e0154f00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 -- 172.21.15.154:0/593203982 >> 172.21.15.154:0/593203982 conn(0x7f50e0072080 msgr2=0x7f50e0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 -- 172.21.15.154:0/593203982 shutdown_connections 2024-03-14T09:03:44.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.660+0000 7f50e7c59700 1 -- 172.21.15.154:0/593203982 wait complete. 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.661+0000 7f50e7c59700 1 Processor -- start 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.661+0000 7f50e7c59700 1 -- start start 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e00c4f30 0x7f50e0145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e00c7f70 0x7f50e01ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e01ee390 con 0x7f50e00c8350 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e0153ca0 con 0x7f50e00c4f30 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e014b5b0 con 0x7f50e00c7f70 2024-03-14T09:03:44.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54558/0 (socket says 172.21.15.154:54558) 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 -- 172.21.15.154:0/4257746039 learned_addr learned my addr 172.21.15.154:0/4257746039 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 -- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e00c7f70 msgr2=0x7f50e01ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e00c7f70 0x7f50e01ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 -- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e00c4f30 msgr2=0x7f50e0145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e00c4f30 0x7f50e0145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 -- 172.21.15.154:0/4257746039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009b30 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e61f6700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50e00cb0b0 tx=0x7f50d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004440 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80045c0 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e0154410 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004970 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.662+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e0153930 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.663+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e0153790 con 0x7f50e00c8350 2024-03-14T09:03:44.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.664+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f50d8025450 con 0x7f50e00c8350 2024-03-14T09:03:44.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.664+0000 7f50d67fc700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 0x7f50c406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:44.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.664+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f50d808a5f0 con 0x7f50e00c8350 2024-03-14T09:03:44.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.664+0000 7f50e59f5700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 0x7f50c406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:44.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.665+0000 7f50e59f5700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 0x7f50c406ccf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50e014b120 tx=0x7f50d0006d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:44.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.669+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f50e0153790 con 0x7f50e00c8350 2024-03-14T09:03:44.867 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.867+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50e0153790 con 0x7f50e00c8350 2024-03-14T09:03:44.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.868+0000 7f50d67fc700 1 -- 172.21.15.154:0/4257746039 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16283 (secure 0 0 0) 0x7f50e0153790 con 0x7f50e00c8350 2024-03-14T09:03:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:44.868 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":58,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:41.626557+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:03:11.026192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:03:11.722376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6837/1333349284":"2024-03-15T09:03:38.613073+0000","172.21.15.191:6836/1333349284":"2024-03-15T09:03:38.613073+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 msgr2=0x7f50c406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 0x7f50c406ccf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f50e014b120 tx=0x7f50d0006d20 comp rx=0 tx=0).stop 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 msgr2=0x7f50e014a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50e00cb0b0 tx=0x7f50d8004ad0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 shutdown_connections 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f50c406a830 0x7f50c406ccf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50e00c8350 0x7f50e014a6f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50e00c4f30 0x7f50e0145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 --2- 172.21.15.154:0/4257746039 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f50e00c7f70 0x7f50e01ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:44.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 >> 172.21.15.154:0/4257746039 conn(0x7f50e0072080 msgr2=0x7f50e0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:44.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 shutdown_connections 2024-03-14T09:03:44.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:44.871+0000 7f50e7c59700 1 -- 172.21.15.154:0/4257746039 wait complete. 2024-03-14T09:03:44.883 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-14T09:03:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 Processor -- start 2024-03-14T09:03:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 -- start start 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe3c0c1080 0x7efe3c0c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe3c06ae80 0x7efe3c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c06f5e0 con 0x7efe3c1550e0 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c06dc90 con 0x7efe3c06ae80 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.006+0000 7efe41c61700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c154c50 con 0x7efe3c0c1080 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe3c06ae80 0x7efe3c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe3c06ae80 0x7efe3c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54570/0 (socket says 172.21.15.154:54570) 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 -- 172.21.15.154:0/3445182478 learned_addr learned my addr 172.21.15.154:0/3445182478 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3b7fe700 1 --2- 172.21.15.154:0/3445182478 >> v2:172.21.15.191:3300/0 conn(0x7efe3c0c1080 0x7efe3c0c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 -- 172.21.15.154:0/3445182478 >> v2:172.21.15.191:3300/0 conn(0x7efe3c0c1080 msgr2=0x7efe3c0c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 --2- 172.21.15.154:0/3445182478 >> v2:172.21.15.191:3300/0 conn(0x7efe3c0c1080 0x7efe3c0c0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 -- 172.21.15.154:0/3445182478 >> v1:172.21.15.191:6789/0 conn(0x7efe3c1550e0 legacy=0x7efe3c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 -- 172.21.15.154:0/3445182478 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0c0860 con 0x7efe3c06ae80 2024-03-14T09:03:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3bfff700 1 --2- 172.21.15.154:0/3445182478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c15f0c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efe3c152580 tx=0x7efe3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d015dfeb57f1887 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3a7fc700 1 -- 172.21.15.154:0/3445182478 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400e070 con 0x7efe3c06ae80 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3a7fc700 1 -- 172.21.15.154:0/3445182478 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34003a20 con 0x7efe3c06ae80 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.007+0000 7efe3a7fc700 1 -- 172.21.15.154:0/3445182478 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34003dd0 con 0x7efe3c06ae80 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 -- 172.21.15.154:0/3445182478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 msgr2=0x7efe3c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 --2- 172.21.15.154:0/3445182478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c15f0c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efe3c152580 tx=0x7efe3400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 -- 172.21.15.154:0/3445182478 shutdown_connections 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 --2- 172.21.15.154:0/3445182478 >> v2:172.21.15.191:3300/0 conn(0x7efe3c0c1080 0x7efe3c0c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 --2- 172.21.15.154:0/3445182478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c15f0c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 -- 172.21.15.154:0/3445182478 >> 172.21.15.154:0/3445182478 conn(0x7efe3c0c39f0 msgr2=0x7efe3c0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 -- 172.21.15.154:0/3445182478 shutdown_connections 2024-03-14T09:03:45.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.008+0000 7efe41c61700 1 -- 172.21.15.154:0/3445182478 wait complete. 2024-03-14T09:03:45.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.009+0000 7efe41c61700 1 Processor -- start 2024-03-14T09:03:45.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.009+0000 7efe41c61700 1 -- start start 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe3c0c1080 0x7efe3c149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3c1550e0 0x7efe3c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c147470 con 0x7efe3c06ae80 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c14f150 con 0x7efe3c1550e0 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe41c61700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c1476c0 con 0x7efe3c0c1080 2024-03-14T09:03:45.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54582/0 (socket says 172.21.15.154:54582) 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 -- 172.21.15.154:0/2191798527 learned_addr learned my addr 172.21.15.154:0/2191798527 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3affd700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe3c0c1080 0x7efe3c149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3bfff700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3c1550e0 0x7efe3c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 -- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe3c0c1080 msgr2=0x7efe3c149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe3c0c1080 0x7efe3c149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 -- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3c1550e0 msgr2=0x7efe3c14a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3c1550e0 0x7efe3c14a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 -- 172.21.15.154:0/2191798527 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34009b30 con 0x7efe3c06ae80 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe3b7fe700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c820 tx=0x7efe3000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.010+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30004220 con 0x7efe3c06ae80 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.011+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30004380 con 0x7efe3c06ae80 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.011+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe300036a0 con 0x7efe3c06ae80 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.011+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c1edcc0 con 0x7efe3c06ae80 2024-03-14T09:03:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.011+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1e1960 con 0x7efe3c06ae80 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.011+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c04ea80 con 0x7efe3c06ae80 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.012+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7efe30003800 con 0x7efe3c06ae80 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.012+0000 7efe23fff700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 0x7efe2806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.012+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5799+0+0 (secure 0 0 0) 0x7efe30088880 con 0x7efe3c06ae80 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.013+0000 7efe3affd700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 0x7efe2806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.013+0000 7efe3affd700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 0x7efe2806ccf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06d660 tx=0x7efe2c0090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.017+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7efe30053c40 con 0x7efe3c06ae80 2024-03-14T09:03:45.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.254+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7efe3c1ee1e0 con 0x7efe3c06ae80 2024-03-14T09:03:45.255 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:03:45.255+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:03:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.262+0000 7efe23fff700 1 -- 172.21.15.154:0/2191798527 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 163+0+0 (secure 0 0 0) 0x7efe3001e020 con 0x7efe3c06ae80 2024-03-14T09:03:45.264 INFO:tasks.ceph.mds.a.smithi154.stderr: -3403> 2024-03-14T09:03:15.547+0000 7f2a5ea9c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.264+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 msgr2=0x7efe2806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 0x7efe2806ccf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06d660 tx=0x7efe2c0090e0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 msgr2=0x7efe3c1e1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c820 tx=0x7efe3000cb30 comp rx=0 tx=0).stop 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 shutdown_connections 2024-03-14T09:03:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe3c1550e0 0x7efe3c14a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe2806a830 0x7efe2806ccf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3c06ae80 0x7efe3c1e1420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 --2- 172.21.15.154:0/2191798527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe3c0c1080 0x7efe3c149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 >> 172.21.15.154:0/2191798527 conn(0x7efe3c0c39f0 msgr2=0x7efe3c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 shutdown_connections 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.265+0000 7efe41c61700 1 -- 172.21.15.154:0/2191798527 wait complete. 2024-03-14T09:03:45.266 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:03:45.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-14T09:03:45.294 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:03:45.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.402+0000 7f721287e700 1 Processor -- start 2024-03-14T09:03:45.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.402+0000 7f721287e700 1 -- start start 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f721287e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f720c155df0 0x7f720c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f721287e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c0ac8e0 con 0x7f720c15d270 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f721287e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c06de30 con 0x7f720c14eaf0 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f721287e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c06b310 con 0x7f720c155df0 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f720b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f720c155df0 0x7f720c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f7210e1b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f720c15d270 0x7f720c15f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37906/0 (socket says 172.21.15.154:37906) 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f720b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f720c155df0 0x7f720c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54594/0 (socket says 172.21.15.154:54594) 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.403+0000 7f7210e1b700 1 -- 172.21.15.154:0/669337809 learned_addr learned my addr 172.21.15.154:0/669337809 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720b7fe700 1 -- 172.21.15.154:0/669337809 >> v1:172.21.15.191:6789/0 conn(0x7f720c14eaf0 legacy=0x7f720c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:45.405 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720b7fe700 1 -- 172.21.15.154:0/669337809 >> v1:172.21.15.154:6789/0 conn(0x7f720c15d270 legacy=0x7f720c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720b7fe700 1 -- 172.21.15.154:0/669337809 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0acb30 con 0x7f720c155df0 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720affd700 1 -- 172.21.15.154:0/669337809 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284607904 0 0) 0x7f720c0ac8e0 con 0x7f720c15d270 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720b7fe700 1 --2- 172.21.15.154:0/669337809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c155df0 0x7f720c156250 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009840 tx=0x7f71fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bc9324e9d1e059 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720affd700 1 -- 172.21.15.154:0/669337809 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004230 con 0x7f720c155df0 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720affd700 1 -- 172.21.15.154:0/669337809 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc004390 con 0x7f720c155df0 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f720affd700 1 -- 172.21.15.154:0/669337809 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0036a0 con 0x7f720c155df0 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f721287e700 1 -- 172.21.15.154:0/669337809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c155df0 msgr2=0x7f720c156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f721287e700 1 --2- 172.21.15.154:0/669337809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c155df0 0x7f720c156250 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009840 tx=0x7f71fc009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f721287e700 1 -- 172.21.15.154:0/669337809 shutdown_connections 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f721287e700 1 --2- 172.21.15.154:0/669337809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c155df0 0x7f720c156250 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.404+0000 7f721287e700 1 -- 172.21.15.154:0/669337809 >> 172.21.15.154:0/669337809 conn(0x7f720c0c3880 msgr2=0x7f720c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.405+0000 7f721287e700 1 -- 172.21.15.154:0/669337809 shutdown_connections 2024-03-14T09:03:45.406 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.405+0000 7f721287e700 1 -- 172.21.15.154:0/669337809 wait complete. 2024-03-14T09:03:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.406+0000 7f721287e700 1 Processor -- start 2024-03-14T09:03:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.406+0000 7f721287e700 1 -- start start 2024-03-14T09:03:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720c14eaf0 0x7f720c14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f720c155df0 0x7f720c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c159bb0 con 0x7f720c15d270 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1eaf60 con 0x7f720c14eaf0 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1eb490 con 0x7f720c155df0 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54596/0 (socket says 172.21.15.154:54596) 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 -- 172.21.15.154:0/104659772 learned_addr learned my addr 172.21.15.154:0/104659772 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f720b7fe700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f720c155df0 0x7f720c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 -- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f720c155df0 msgr2=0x7f720c0c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f720c155df0 0x7f720c0c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 -- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720c14eaf0 msgr2=0x7f720c14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f720bfff700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720c14eaf0 0x7f720c14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720c14eaf0 0x7f720c14a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 -- 172.21.15.154:0/104659772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0094a0 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7210e1b700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f720400cca0 tx=0x7f720400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400cea0 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400ade0 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204007310 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c1eb1b0 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.407+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c0c1880 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.408+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c14e650 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.409+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7204007470 con 0x7f720c15d270 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.409+0000 7f7208ff9700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 0x7f71f806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.409+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5885+0+0 (secure 0 0 0) 0x7f7204088a10 con 0x7f720c15d270 2024-03-14T09:03:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.409+0000 7f720bfff700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 0x7f71f806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.410+0000 7f720bfff700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 0x7f71f806ccf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f720c14aa10 tx=0x7f7200008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:45.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.414+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7204053e10 con 0x7f720c15d270 2024-03-14T09:03:45.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:45.655+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f720c0697d0 con 0x7f720c15d270 2024-03-14T09:03:46.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.268+0000 7f7208ff9700 1 -- 172.21.15.154:0/104659772 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7204018070 con 0x7f720c15d270 2024-03-14T09:03:46.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 msgr2=0x7f71f806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 0x7f71f806ccf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f720c14aa10 tx=0x7f7200008040 comp rx=0 tx=0).stop 2024-03-14T09:03:46.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 msgr2=0x7f720c1eda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f720400cca0 tx=0x7f720400a340 comp rx=0 tx=0).stop 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 shutdown_connections 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f71f806a830 0x7f71f806ccf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f720c15d270 0x7f720c1eda50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f720c14eaf0 0x7f720c14a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 --2- 172.21.15.154:0/104659772 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f720c155df0 0x7f720c0c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 >> 172.21.15.154:0/104659772 conn(0x7f720c0c3880 msgr2=0x7f720c14c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 shutdown_connections 2024-03-14T09:03:46.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.271+0000 7f721287e700 1 -- 172.21.15.154:0/104659772 wait complete. 2024-03-14T09:03:46.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-14T09:03:46.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.452+0000 7f343544c700 1 Processor -- start 2024-03-14T09:03:46.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.452+0000 7f343544c700 1 -- start start 2024-03-14T09:03:46.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f343544c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34300c52f0 0x7f343006bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f343544c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 0x7f34300c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f343544c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34300c4cc0 con 0x7f34300c7f80 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f343544c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343006c260 con 0x7f34300c52f0 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f343544c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300c4fa0 con 0x7f34300c65a0 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34300c52f0 0x7f343006bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34300c52f0 0x7f343006bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54608/0 (socket says 172.21.15.154:54608) 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 -- 172.21.15.154:0/530811291 learned_addr learned my addr 172.21.15.154:0/530811291 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342f7fe700 1 --2- 172.21.15.154:0/530811291 >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 0x7f34300c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 -- 172.21.15.154:0/530811291 >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 msgr2=0x7f34300c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 --2- 172.21.15.154:0/530811291 >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 0x7f34300c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342effd700 1 -- 172.21.15.154:0/530811291 >> v1:172.21.15.154:6789/0 conn(0x7f34300c7f80 legacy=0x7f34300c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342effd700 1 -- 172.21.15.154:0/530811291 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430155580 con 0x7f34300c52f0 2024-03-14T09:03:46.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.453+0000 7f342dffb700 1 -- 172.21.15.154:0/530811291 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513167108 0 0) 0x7f34300c4cc0 con 0x7f34300c7f80 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342f7fe700 1 --2- 172.21.15.154:0/530811291 >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 0x7f34300c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342effd700 1 --2- 172.21.15.154:0/530811291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c52f0 0x7f343006bb50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f343006cb00 tx=0x7f3424009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add7cf25af6f547 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342dffb700 1 -- 172.21.15.154:0/530811291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400b870 con 0x7f34300c52f0 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342dffb700 1 -- 172.21.15.154:0/530811291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400b9d0 con 0x7f34300c52f0 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f342dffb700 1 -- 172.21.15.154:0/530811291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400bd60 con 0x7f34300c52f0 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 -- 172.21.15.154:0/530811291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c52f0 msgr2=0x7f343006bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 --2- 172.21.15.154:0/530811291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c52f0 0x7f343006bb50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f343006cb00 tx=0x7f3424009bd0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 -- 172.21.15.154:0/530811291 shutdown_connections 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 --2- 172.21.15.154:0/530811291 >> v2:172.21.15.191:3300/0 conn(0x7f34300c65a0 0x7f34300c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 --2- 172.21.15.154:0/530811291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c52f0 0x7f343006bb50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 -- 172.21.15.154:0/530811291 >> 172.21.15.154:0/530811291 conn(0x7f34300ca260 msgr2=0x7f34300ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:46.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 -- 172.21.15.154:0/530811291 shutdown_connections 2024-03-14T09:03:46.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.454+0000 7f343544c700 1 -- 172.21.15.154:0/530811291 wait complete. 2024-03-14T09:03:46.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 Processor -- start 2024-03-14T09:03:46.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 -- start start 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 0x7f34300ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 0x7f34301ed710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3430153ae0 con 0x7f34300c7f80 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301edfe0 con 0x7f34300c52f0 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.456+0000 7f343544c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34301ee2c0 con 0x7f34300c65a0 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54622/0 (socket says 172.21.15.154:54622) 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 -- 172.21.15.154:0/2109563448 learned_addr learned my addr 172.21.15.154:0/2109563448 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342effd700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 0x7f34300ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342e7fc700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 0x7f34301ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 -- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 msgr2=0x7f34301ed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 0x7f34301ed710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 -- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 msgr2=0x7f34300ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 0x7f34300ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 -- 172.21.15.154:0/2109563448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009550 con 0x7f34300c7f80 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342effd700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 0x7f34300ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342e7fc700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 0x7f34301ed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f342f7fe700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f342800cef0 tx=0x7f342800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800ae50 con 0x7f34300c7f80 2024-03-14T09:03:46.466 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3428015440 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3430153420 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34280157f0 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.457+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34301edc50 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.458+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34300697d0 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.459+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f342800d430 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.459+0000 7f34177fe700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 0x7f341c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.459+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5885+0+0 (secure 0 0 0) 0x7f34280893c0 con 0x7f34300c7f80 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.459+0000 7f342effd700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 0x7f341c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.460+0000 7f342effd700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 0x7f341c06ccf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3424014040 tx=0x7f3424008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:46.467 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.464+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f34280547c0 con 0x7f34300c7f80 2024-03-14T09:03:46.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:46.667+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34300c9d90 con 0x7f34300c7f80 2024-03-14T09:03:47.647 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.647+0000 7f34177fe700 1 -- 172.21.15.154:0/2109563448 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v60) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3428057de0 con 0x7f34300c7f80 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 msgr2=0x7f341c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 0x7f341c06ccf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3424014040 tx=0x7f3424008030 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 msgr2=0x7f343014a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f342800cef0 tx=0x7f342800a340 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 shutdown_connections 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f341c06a830 0x7f341c06ccf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34300c7f80 0x7f343014a7c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34300c52f0 0x7f34300ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 --2- 172.21.15.154:0/2109563448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34300c65a0 0x7f34301ed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 >> 172.21.15.154:0/2109563448 conn(0x7f34300ca260 msgr2=0x7f34301509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:47.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 shutdown_connections 2024-03-14T09:03:47.652 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.651+0000 7f343544c700 1 -- 172.21.15.154:0/2109563448 wait complete. 2024-03-14T09:03:47.652 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' removed 2024-03-14T09:03:47.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-14T09:03:47.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.796+0000 7f903e463700 1 Processor -- start 2024-03-14T09:03:47.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.796+0000 7f903e463700 1 -- start start 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903e463700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f903806d4d0 0x7f9038158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903e463700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9038157ed0 con 0x7f90380c1380 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903e463700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9038158490 con 0x7f9038155360 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903e463700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903806d8b0 con 0x7f903806d4d0 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f9037fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f903806d4d0 0x7f9038158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903ca00700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f90380c1380 0x7f903814ace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37912/0 (socket says 172.21.15.154:37912) 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.797+0000 7f903ca00700 1 -- 172.21.15.154:0/1782839743 learned_addr learned my addr 172.21.15.154:0/1782839743 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9037fff700 1 -- 172.21.15.154:0/1782839743 >> v1:172.21.15.191:6789/0 conn(0x7f9038155360 legacy=0x7f903806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:47.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9037fff700 1 -- 172.21.15.154:0/1782839743 >> v1:172.21.15.154:6789/0 conn(0x7f90380c1380 legacy=0x7f903814ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9037fff700 1 -- 172.21.15.154:0/1782839743 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903806cd20 con 0x7f903806d4d0 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9036ffd700 1 -- 172.21.15.154:0/1782839743 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914848214 0 0) 0x7f9038157ed0 con 0x7f90380c1380 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9037fff700 1 --2- 172.21.15.154:0/1782839743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f903806d4d0 0x7f9038158920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f902c009840 tx=0x7f902c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1768865540f4eee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9036ffd700 1 -- 172.21.15.154:0/1782839743 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011830 con 0x7f903806d4d0 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9036ffd700 1 -- 172.21.15.154:0/1782839743 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c011990 con 0x7f903806d4d0 2024-03-14T09:03:47.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.798+0000 7f9036ffd700 1 -- 172.21.15.154:0/1782839743 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011d20 con 0x7f903806d4d0 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 -- 172.21.15.154:0/1782839743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f903806d4d0 msgr2=0x7f9038158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 --2- 172.21.15.154:0/1782839743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f903806d4d0 0x7f9038158920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f902c009840 tx=0x7f902c009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 -- 172.21.15.154:0/1782839743 shutdown_connections 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 --2- 172.21.15.154:0/1782839743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f903806d4d0 0x7f9038158920 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 -- 172.21.15.154:0/1782839743 >> 172.21.15.154:0/1782839743 conn(0x7f90380a9d00 msgr2=0x7f90380aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 -- 172.21.15.154:0/1782839743 shutdown_connections 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.799+0000 7f903e463700 1 -- 172.21.15.154:0/1782839743 wait complete. 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.800+0000 7f903e463700 1 Processor -- start 2024-03-14T09:03:47.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.800+0000 7f903e463700 1 -- start start 2024-03-14T09:03:47.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 0x7f9038153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:47.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9038155360 0x7f903806e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038153e50 con 0x7f90380c1380 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038153480 con 0x7f9038155360 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903e463700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90381defc0 con 0x7f903806d4d0 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54636/0 (socket says 172.21.15.154:54636) 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 -- 172.21.15.154:0/2208536708 learned_addr learned my addr 172.21.15.154:0/2208536708 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f903ca00700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9038155360 0x7f903806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f9037fff700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 0x7f9038153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 -- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 msgr2=0x7f9038153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 0x7f9038153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 -- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9038155360 msgr2=0x7f903806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9038155360 0x7f903806e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:47.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 -- 172.21.15.154:0/2208536708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0094a0 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f9037fff700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 0x7f9038153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.801+0000 7f90377fe700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f902800c450 tx=0x7f902800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.802+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90280052d0 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.802+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9028005430 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.802+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90381dea00 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.802+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90280038c0 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.802+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038153110 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.803+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038152f70 con 0x7f90380c1380 2024-03-14T09:03:47.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.803+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9028003a20 con 0x7f90380c1380 2024-03-14T09:03:47.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.803+0000 7f9034ff9700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 0x7f902406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:47.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.803+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5445+0+0 (secure 0 0 0) 0x7f9028087f40 con 0x7f90380c1380 2024-03-14T09:03:47.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.804+0000 7f9037fff700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 0x7f902406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:47.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.804+0000 7f9037fff700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 0x7f902406ccf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9038154510 tx=0x7f902c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:47.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:47.808+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9038152f70 con 0x7f90380c1380 2024-03-14T09:03:48.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.001+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90381532a0 con 0x7f90380c1380 2024-03-14T09:03:48.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.649+0000 7f9034ff9700 1 -- 172.21.15.154:0/2208536708 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v61) v1 ==== 172+0+0 (secure 0 0 0) 0x7f90280534f0 con 0x7f90380c1380 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 msgr2=0x7f902406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 0x7f902406ccf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9038154510 tx=0x7f902c01c040 comp rx=0 tx=0).stop 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 msgr2=0x7f90381e0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f902800c450 tx=0x7f902800c760 comp rx=0 tx=0).stop 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 shutdown_connections 2024-03-14T09:03:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9038155360 0x7f903806e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903806d4d0 0x7f9038153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f902406a830 0x7f902406ccf0 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9038154510 tx=0x7f902c01c040 comp rx=0 tx=0).stop 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 --2- 172.21.15.154:0/2208536708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90380c1380 0x7f90381e0790 secure :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f902800c450 tx=0x7f902800c760 comp rx=0 tx=0).stop 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.654+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 >> 172.21.15.154:0/2208536708 conn(0x7f90380a9d00 msgr2=0x7f90380c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.655+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 shutdown_connections 2024-03-14T09:03:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.655+0000 7f903e463700 1 -- 172.21.15.154:0/2208536708 wait complete. 2024-03-14T09:03:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' removed 2024-03-14T09:03:48.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T09:03:48.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.794+0000 7f2593cd1700 1 Processor -- start 2024-03-14T09:03:48.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.794+0000 7f2593cd1700 1 -- start start 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 0x7f258c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 0x7f258c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f258c1550b0 0x7f258c0c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c13ade0 con 0x7f258c1550b0 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c06fb30 con 0x7f258c0a8540 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2593cd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c06ec30 con 0x7f258c0c3250 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f258c1550b0 0x7f258c0c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f258c1550b0 0x7f258c0c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54638/0 (socket says 172.21.15.154:54638) 2024-03-14T09:03:48.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 -- 172.21.15.154:0/847831035 learned_addr learned my addr 172.21.15.154:0/847831035 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259126c700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 0x7f258c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2591a6d700 1 --2- 172.21.15.154:0/847831035 >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 0x7f258c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 -- 172.21.15.154:0/847831035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 msgr2=0x7f258c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 0x7f258c154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 -- 172.21.15.154:0/847831035 >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 msgr2=0x7f258c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 --2- 172.21.15.154:0/847831035 >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 0x7f258c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 -- 172.21.15.154:0/847831035 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c06ef10 con 0x7f258c1550b0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259126c700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 0x7f258c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2591a6d700 1 --2- 172.21.15.154:0/847831035 >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 0x7f258c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f259226e700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c1550b0 0x7f258c0c1780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f258c070da0 tx=0x7f258400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e707bc263d879419 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.795+0000 7f2590a6b700 1 -- 172.21.15.154:0/847831035 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584010070 con 0x7f258c1550b0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2590a6b700 1 -- 172.21.15.154:0/847831035 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258400dd90 con 0x7f258c1550b0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2590a6b700 1 -- 172.21.15.154:0/847831035 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400b310 con 0x7f258c1550b0 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 -- 172.21.15.154:0/847831035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c1550b0 msgr2=0x7f258c0c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:48.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c1550b0 0x7f258c0c1780 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f258c070da0 tx=0x7f258400d430 comp rx=0 tx=0).stop 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 -- 172.21.15.154:0/847831035 shutdown_connections 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c1550b0 0x7f258c0c1780 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 --2- 172.21.15.154:0/847831035 >> v2:172.21.15.191:3300/0 conn(0x7f258c0a8540 0x7f258c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 --2- 172.21.15.154:0/847831035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c0c3250 0x7f258c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 -- 172.21.15.154:0/847831035 >> 172.21.15.154:0/847831035 conn(0x7f258c0c3880 msgr2=0x7f258c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 -- 172.21.15.154:0/847831035 shutdown_connections 2024-03-14T09:03:48.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.796+0000 7f2593cd1700 1 -- 172.21.15.154:0/847831035 wait complete. 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.797+0000 7f2593cd1700 1 Processor -- start 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.797+0000 7f2593cd1700 1 -- start start 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f258c0c3250 0x7f258c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c1550b0 0x7f258c1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0aabd0 con 0x7f258c0a8540 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1e9ca0 con 0x7f258c0c3250 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2593cd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0aa8f0 con 0x7f258c1550b0 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54652/0 (socket says 172.21.15.154:54652) 2024-03-14T09:03:48.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 -- 172.21.15.154:0/1667484374 learned_addr learned my addr 172.21.15.154:0/1667484374 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f259126c700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f258c0c3250 0x7f258c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f259226e700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c1550b0 0x7f258c1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 -- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c1550b0 msgr2=0x7f258c1336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c1550b0 0x7f258c1336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 -- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f258c0c3250 msgr2=0x7f258c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f258c0c3250 0x7f258c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.798+0000 7f2591a6d700 1 -- 172.21.15.154:0/1667484374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258400bd40 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f2591a6d700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f258800c750 tx=0x7f258800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588011800 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588011960 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c150740 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588011cd0 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.799+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c150d00 con 0x7f258c0a8540 2024-03-14T09:03:48.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.800+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c0697d0 con 0x7f258c0a8540 2024-03-14T09:03:48.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.800+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2588021350 con 0x7f258c0a8540 2024-03-14T09:03:48.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.801+0000 7f25827fc700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 0x7f257006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:48.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.801+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2588088880 con 0x7f258c0a8540 2024-03-14T09:03:48.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.801+0000 7f259126c700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 0x7f257006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:48.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.801+0000 7f259126c700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 0x7f257006ccf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad3e0 tx=0x7f257c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:48.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.805+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2588053fd0 con 0x7f258c0a8540 2024-03-14T09:03:48.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:48.998+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f258c06d420 con 0x7f258c0a8540 2024-03-14T09:03:49.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.002+0000 7f25827fc700 1 -- 172.21.15.154:0/1667484374 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f258801e0a0 con 0x7f258c0a8540 2024-03-14T09:03:49.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 msgr2=0x7f257006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 0x7f257006ccf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad3e0 tx=0x7f257c008040 comp rx=0 tx=0).stop 2024-03-14T09:03:49.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 msgr2=0x7f258c1e9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f258800c750 tx=0x7f258800ca60 comp rx=0 tx=0).stop 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 shutdown_connections 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f258c1550b0 0x7f258c1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f257006a830 0x7f257006ccf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f258c0a8540 0x7f258c1e9760 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 --2- 172.21.15.154:0/1667484374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f258c0c3250 0x7f258c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 >> 172.21.15.154:0/1667484374 conn(0x7f258c0c3880 msgr2=0x7f258c139840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 shutdown_connections 2024-03-14T09:03:49.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.004+0000 7f2593cd1700 1 -- 172.21.15.154:0/1667484374 wait complete. 2024-03-14T09:03:49.016 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent' 2024-03-14T09:03:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.139+0000 7f289fa46700 1 Processor -- start 2024-03-14T09:03:49.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.139+0000 7f289fa46700 1 -- start start 2024-03-14T09:03:49.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289fa46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f289813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289fa46700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289813ade0 con 0x7f28980c1780 2024-03-14T09:03:49.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289fa46700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289806fb30 con 0x7f28980a8540 2024-03-14T09:03:49.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289fa46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289806ec30 con 0x7f28980c3250 2024-03-14T09:03:49.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289dfe3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f28980c1780 0x7f28980c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37920/0 (socket says 172.21.15.154:37920) 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289dfe3700 1 -- 172.21.15.154:0/1819378765 learned_addr learned my addr 172.21.15.154:0/1819378765 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f289cfe1700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f289813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334816187 0 0) 0x7f2890001490 con 0x7f28980c1780 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.140+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28840035b0 con 0x7f28980c1780 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 680692665 0 0) 0x7f28840035b0 con 0x7f28980c1780 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 msgr2=0x7f289813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f289813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 >> v1:172.21.15.191:6789/0 conn(0x7f28980a8540 legacy=0x7f2898154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289806ef10 con 0x7f28980c1780 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2890001b80 con 0x7f28980c1780 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 >> v1:172.21.15.154:6789/0 conn(0x7f28980c1780 legacy=0x7f28980c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f2884003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2884003fb0 0x7f2884004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884004d90 0x7f2884005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289806ec30 con 0x7f2884003fb0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288400bc50 con 0x7f2884004d90 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288400be10 con 0x7f28980c3250 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2884003fb0 0x7f2884004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289dfe3700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884004d90 0x7f2884005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289d7e2700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f2884003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 -- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 msgr2=0x7f2884003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28980c3250 0x7f2884003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 -- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884004d90 msgr2=0x7f2884005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884004d90 0x7f2884005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 -- 172.21.15.154:0/1819378765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840035b0 con 0x7f2884003fb0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289dfe3700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884004d90 0x7f2884005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f289cfe1700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2884003fb0 0x7f2884004850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f288800be00 tx=0x7f2884005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f26be74d4a94e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888010db0 con 0x7f2884003fb0 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888011450 con 0x7f2884003fb0 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.141+0000 7f288ffff700 1 -- 172.21.15.154:0/1819378765 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28880117c0 con 0x7f2884003fb0 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 -- 172.21.15.154:0/1819378765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2884003fb0 msgr2=0x7f2884004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 --2- 172.21.15.154:0/1819378765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2884003fb0 0x7f2884004850 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f288800be00 tx=0x7f2884005880 comp rx=0 tx=0).stop 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289dfe3700 1 -- 172.21.15.154:0/1819378765 reap_dead start 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 -- 172.21.15.154:0/1819378765 shutdown_connections 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 -- 172.21.15.154:0/1819378765 >> 172.21.15.154:0/1819378765 conn(0x7f28980c3880 msgr2=0x7f28980c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 -- 172.21.15.154:0/1819378765 shutdown_connections 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.142+0000 7f289fa46700 1 -- 172.21.15.154:0/1819378765 wait complete. 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.143+0000 7f289fa46700 1 Processor -- start 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.143+0000 7f289fa46700 1 -- start start 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884003fb0 0x7f28981512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28981dcf10 0x7f28981dd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2898151d70 con 0x7f28980aad70 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980ab730 con 0x7f2884003fb0 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289fa46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28981e62c0 con 0x7f28981dcf10 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54668/0 (socket says 172.21.15.154:54668) 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 -- 172.21.15.154:0/3555826359 learned_addr learned my addr 172.21.15.154:0/3555826359 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289d7e2700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884003fb0 0x7f28981512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289cfe1700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28981dcf10 0x7f28981dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 -- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28981dcf10 msgr2=0x7f28981dd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28981dcf10 0x7f28981dd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 -- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884003fb0 msgr2=0x7f28981512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884003fb0 0x7f28981512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.144+0000 7f289dfe3700 1 -- 172.21.15.154:0/3555826359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28981dd9e0 con 0x7f28980aad70 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f289dfe3700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2890000f80 tx=0x7f289000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000eaa0 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289000ec00 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28981e6580 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890013420 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.145+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28981e6b40 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.146+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28980697d0 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.146+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2890013580 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.147+0000 7f288dffb700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 0x7f287c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.147+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2890088cd0 con 0x7f28980aad70 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.147+0000 7f289d7e2700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 0x7f287c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.147+0000 7f289d7e2700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 0x7f287c06ccf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2894007fd0 tx=0x7f2894009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.151+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2890054380 con 0x7f28980aad70 2024-03-14T09:03:49.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.433+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]} v 0) v1 -- 0x7f289813ade0 con 0x7f28980aad70 2024-03-14T09:03:49.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.656+0000 7f288dffb700 1 -- 172.21.15.154:0/3555826359 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth.test_mount_mon_and_osd_caps_present_mds_caps_absent"]}]=0 v162) v1 ==== 184+0+0 (secure 0 0 0) 0x7f289813ade0 con 0x7f28980aad70 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 msgr2=0x7f287c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 0x7f287c06ccf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2894007fd0 tx=0x7f2894009040 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 msgr2=0x7f28980ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2890000f80 tx=0x7f289000e440 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 shutdown_connections 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f287c06a830 0x7f287c06ccf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28980aad70 0x7f28980ab1f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2884003fb0 0x7f28981512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 --2- 172.21.15.154:0/3555826359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28981dcf10 0x7f28981dd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 >> 172.21.15.154:0/3555826359 conn(0x7f28980c3880 msgr2=0x7f28980c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 shutdown_connections 2024-03-14T09:03:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.658+0000 7f289fa46700 1 -- 172.21.15.154:0/3555826359 wait complete. 2024-03-14T09:03:49.671 INFO:tasks.cephfs_test_runner:test_mount_mon_and_osd_caps_present_mds_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) ... ok 2024-03-14T09:03:49.671 INFO:tasks.cephfs_test_runner:Starting test: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-03-14T09:03:49.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap' 2024-03-14T09:03:49.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.793+0000 7f1277a36700 1 Processor -- start 2024-03-14T09:03:49.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.793+0000 7f1277a36700 1 -- start start 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1277a36700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12700c3250 0x7f127013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1277a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f12700c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1277a36700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127013ade0 con 0x7f12700a8540 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1277a36700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127006fb30 con 0x7f12700c3250 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1277a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127006ec30 con 0x7f12700c1780 2024-03-14T09:03:49.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12700c3250 0x7f127013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f12757d2700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f12700a8540 0x7f1270154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46692/0 (socket says 172.21.15.154:46692) 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f12700c3250 0x7f127013a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56414/0 (socket says 172.21.15.154:56414) 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f12757d2700 1 -- 172.21.15.154:0/1853545818 learned_addr learned my addr 172.21.15.154:0/1853545818 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1275fd3700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f12700c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 -- 172.21.15.154:0/1853545818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 msgr2=0x7f12700c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f12700c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 -- 172.21.15.154:0/1853545818 >> v1:172.21.15.154:6789/0 conn(0x7f12700a8540 legacy=0x7f1270154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 -- 172.21.15.154:0/1853545818 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127006ef10 con 0x7f12700c3250 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1267fff700 1 -- 172.21.15.154:0/1853545818 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301621676 0 0) 0x7f126c001110 con 0x7f12700a8540 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1274fd1700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127013a8a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1260009840 tx=0x7f1260009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0f4cd32ebf92df server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1267fff700 1 -- 172.21.15.154:0/1853545818 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011830 con 0x7f12700c3250 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1267fff700 1 -- 172.21.15.154:0/1853545818 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260011990 con 0x7f12700c3250 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.794+0000 7f1267fff700 1 -- 172.21.15.154:0/1853545818 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260011d20 con 0x7f12700c3250 2024-03-14T09:03:49.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 -- 172.21.15.154:0/1853545818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 msgr2=0x7f127013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127013a8a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1260009840 tx=0x7f1260009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 -- 172.21.15.154:0/1853545818 shutdown_connections 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f12700c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 --2- 172.21.15.154:0/1853545818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127013a8a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 -- 172.21.15.154:0/1853545818 >> 172.21.15.154:0/1853545818 conn(0x7f12700c3880 msgr2=0x7f12700c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 -- 172.21.15.154:0/1853545818 shutdown_connections 2024-03-14T09:03:49.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.795+0000 7f1277a36700 1 -- 172.21.15.154:0/1853545818 wait complete. 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.796+0000 7f1277a36700 1 Processor -- start 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.796+0000 7f1277a36700 1 -- start start 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 0x7f12700c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f1270150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12700c08e0 con 0x7f12700c3250 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12700c1240 con 0x7f12700a8540 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1277a36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12700c02b0 con 0x7f12700c1780 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56426/0 (socket says 172.21.15.154:56426) 2024-03-14T09:03:49.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 -- 172.21.15.154:0/3318006177 learned_addr learned my addr 172.21.15.154:0/3318006177 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1274fd1700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f1270150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f12757d2700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 0x7f12700c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 -- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 msgr2=0x7f1270150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f1270150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 -- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 msgr2=0x7f12700c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 0x7f12700c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 -- 172.21.15.154:0/3318006177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600094a0 con 0x7f12700c3250 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f12757d2700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 0x7f12700c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.797+0000 7f1275fd3700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1268007ec0 tx=0x7f126800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.798+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126800ede0 con 0x7f12700c3250 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.798+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268014420 con 0x7f12700c3250 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.798+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127013ab90 con 0x7f12700c3250 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.798+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12680147d0 con 0x7f12700c3250 2024-03-14T09:03:49.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.798+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12701ee300 con 0x7f12700c3250 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.799+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12700c0a20 con 0x7f12700c3250 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.799+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1268014930 con 0x7f12700c3250 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.799+0000 7f1265ffb700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 0x7f125406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.800+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1268089140 con 0x7f12700c3250 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.800+0000 7f12757d2700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 0x7f125406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:49.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.800+0000 7f12757d2700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 0x7f125406cd20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f126c000f60 tx=0x7f126c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:49.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:49.804+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f12700c0a20 con 0x7f12700c3250 2024-03-14T09:03:50.079 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.079+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]} v 0) v1 -- 0x7f12700c0a20 con 0x7f12700c3250 2024-03-14T09:03:50.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.657+0000 7f1265ffb700 1 -- 172.21.15.154:0/3318006177 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]}]=0 v163) v1 ==== 162+0+0 (secure 0 0 0) 0x7f12700c0a20 con 0x7f12700c3250 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 msgr2=0x7f125406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 0x7f125406cd20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f126c000f60 tx=0x7f126c007d30 comp rx=0 tx=0).stop 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 msgr2=0x7f127014e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1268007ec0 tx=0x7f126800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 shutdown_connections 2024-03-14T09:03:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f125406a860 0x7f125406cd20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12700c3250 0x7f127014e500 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12700a8540 0x7f12700c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 --2- 172.21.15.154:0/3318006177 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12700c1780 0x7f1270150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 >> 172.21.15.154:0/3318006177 conn(0x7f12700c3880 msgr2=0x7f1270136740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 shutdown_connections 2024-03-14T09:03:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.660+0000 7f1277a36700 1 -- 172.21.15.154:0/3318006177 wait complete. 2024-03-14T09:03:50.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-14T09:03:50.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.799+0000 7fd64ba43700 1 Processor -- start 2024-03-14T09:03:50.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.799+0000 7fd64ba43700 1 -- start start 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd64ba43700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd644155df0 0x7fd644156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd64ba43700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6440ac8e0 con 0x7fd64415d270 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd64ba43700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64406de30 con 0x7fd64414eaf0 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd64ba43700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64406b310 con 0x7fd644155df0 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd648fde700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd644155df0 0x7fd644156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd649fe0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd64415d270 0x7fd64415f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46696/0 (socket says 172.21.15.154:46696) 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd649fe0700 1 -- 172.21.15.154:0/3968950978 learned_addr learned my addr 172.21.15.154:0/3968950978 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd648fde700 1 -- 172.21.15.154:0/3968950978 >> v1:172.21.15.191:6789/0 conn(0x7fd64414eaf0 legacy=0x7fd64406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd648fde700 1 -- 172.21.15.154:0/3968950978 >> v1:172.21.15.154:6789/0 conn(0x7fd64415d270 legacy=0x7fd64415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd648fde700 1 -- 172.21.15.154:0/3968950978 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440acb30 con 0x7fd644155df0 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd63bfff700 1 -- 172.21.15.154:0/3968950978 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989482799 0 0) 0x7fd6440ac8e0 con 0x7fd64415d270 2024-03-14T09:03:50.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd648fde700 1 --2- 172.21.15.154:0/3968950978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644156250 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd634009840 tx=0x7fd634009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289d32d3ae739898 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.800+0000 7fd63bfff700 1 -- 172.21.15.154:0/3968950978 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004230 con 0x7fd644155df0 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd63bfff700 1 -- 172.21.15.154:0/3968950978 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634004390 con 0x7fd644155df0 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd63bfff700 1 -- 172.21.15.154:0/3968950978 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340036a0 con 0x7fd644155df0 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 -- 172.21.15.154:0/3968950978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 msgr2=0x7fd644156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3968950978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644156250 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd634009840 tx=0x7fd634009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 -- 172.21.15.154:0/3968950978 shutdown_connections 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3968950978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644156250 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 -- 172.21.15.154:0/3968950978 >> 172.21.15.154:0/3968950978 conn(0x7fd6440c3880 msgr2=0x7fd6440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 -- 172.21.15.154:0/3968950978 shutdown_connections 2024-03-14T09:03:50.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.801+0000 7fd64ba43700 1 -- 172.21.15.154:0/3968950978 wait complete. 2024-03-14T09:03:50.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.802+0000 7fd64ba43700 1 Processor -- start 2024-03-14T09:03:50.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.802+0000 7fd64ba43700 1 -- start start 2024-03-14T09:03:50.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 0x7fd644144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:50.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:50.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd64415d270 0x7fd644156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6441ed720 con 0x7fd644155df0 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6441eda00 con 0x7fd64415d270 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd64ba43700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6441edce0 con 0x7fd64414eaf0 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56444/0 (socket says 172.21.15.154:56444) 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 -- 172.21.15.154:0/3045769787 learned_addr learned my addr 172.21.15.154:0/3045769787 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd6497df700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 0x7fd644144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd649fe0700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd64415d270 0x7fd644156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 -- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 msgr2=0x7fd644144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 0x7fd644144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 -- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd64415d270 msgr2=0x7fd644156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd64415d270 0x7fd644156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:50.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 -- 172.21.15.154:0/3045769787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340094a0 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd6497df700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 0x7fd644144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.803+0000 7fd648fde700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd6440c43c0 tx=0x7fd634004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.804+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634011630 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.804+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63401e070 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.804+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6441edfc0 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.804+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340119f0 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.804+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd644158450 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.805+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6441ee4e0 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.805+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd634011b50 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.806+0000 7fd639ffb700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 0x7fd62806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.806+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd634089e40 con 0x7fd644155df0 2024-03-14T09:03:50.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.806+0000 7fd6497df700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 0x7fd62806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:50.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.806+0000 7fd6497df700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 0x7fd62806ccf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd64414e850 tx=0x7fd640008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:50.809 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:50.810+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd6340554f0 con 0x7fd644155df0 2024-03-14T09:03:51.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.002+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd6440697d0 con 0x7fd644155df0 2024-03-14T09:03:51.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.006+0000 7fd639ffb700 1 -- 172.21.15.154:0/3045769787 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6340200c0 con 0x7fd644155df0 2024-03-14T09:03:51.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 msgr2=0x7fd62806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 0x7fd62806ccf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd64414e850 tx=0x7fd640008040 comp rx=0 tx=0).stop 2024-03-14T09:03:51.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 msgr2=0x7fd644157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd6440c43c0 tx=0x7fd634004750 comp rx=0 tx=0).stop 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 shutdown_connections 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd64415d270 0x7fd644156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd64414eaf0 0x7fd644144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd62806a830 0x7fd62806ccf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 --2- 172.21.15.154:0/3045769787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd644155df0 0x7fd644157f10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 >> 172.21.15.154:0/3045769787 conn(0x7fd6440c3880 msgr2=0x7fd6440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 shutdown_connections 2024-03-14T09:03:51.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.008+0000 7fd64ba43700 1 -- 172.21.15.154:0/3045769787 wait complete. 2024-03-14T09:03:51.020 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:03:51.020 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:03:51.060 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:03:51.108 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:03:51.108 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:03:51.108 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:51.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:03:51.126 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:03:51.126 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:03:51.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:03:51.146 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-14T09:03:51.146 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:03:51.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.287+0000 7ffacd90d700 1 Processor -- start 2024-03-14T09:03:51.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 -- start start 2024-03-14T09:03:51.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 0x7ffac814e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffac80c1100 0x7ffac80c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac806d940 con 0x7ffac8150ec0 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80aa800 con 0x7ffac80c1100 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffacd90d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac806cd30 con 0x7ffac806c740 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffac67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffac80c1100 0x7ffac80c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffac77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffac8150ec0 0x7ffac8151310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46700/0 (socket says 172.21.15.154:46700) 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.288+0000 7ffac77fe700 1 -- 172.21.15.154:0/1963589935 learned_addr learned my addr 172.21.15.154:0/1963589935 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac6ffd700 1 --2- 172.21.15.154:0/1963589935 >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 0x7ffac814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac67fc700 1 -- 172.21.15.154:0/1963589935 >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 msgr2=0x7ffac814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac67fc700 1 --2- 172.21.15.154:0/1963589935 >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 0x7ffac814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac67fc700 1 -- 172.21.15.154:0/1963589935 >> v1:172.21.15.154:6789/0 conn(0x7ffac8150ec0 legacy=0x7ffac8151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac67fc700 1 -- 172.21.15.154:0/1963589935 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac806d010 con 0x7ffac80c1100 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac5ffb700 1 -- 172.21.15.154:0/1963589935 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290721574 0 0) 0x7ffac806d940 con 0x7ffac8150ec0 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac6ffd700 1 --2- 172.21.15.154:0/1963589935 >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 0x7ffac814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac67fc700 1 --2- 172.21.15.154:0/1963589935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac80c1100 0x7ffac80c1580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffac806e9a0 tx=0x7ffabc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931dd53ccac03d15 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac5ffb700 1 -- 172.21.15.154:0/1963589935 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011830 con 0x7ffac80c1100 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac5ffb700 1 -- 172.21.15.154:0/1963589935 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc011990 con 0x7ffac80c1100 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.289+0000 7ffac5ffb700 1 -- 172.21.15.154:0/1963589935 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011d20 con 0x7ffac80c1100 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 -- 172.21.15.154:0/1963589935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac80c1100 msgr2=0x7ffac80c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1963589935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac80c1100 0x7ffac80c1580 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffac806e9a0 tx=0x7ffabc009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 -- 172.21.15.154:0/1963589935 shutdown_connections 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1963589935 >> v2:172.21.15.191:3300/0 conn(0x7ffac806c740 0x7ffac814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1963589935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac80c1100 0x7ffac80c1580 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 -- 172.21.15.154:0/1963589935 >> 172.21.15.154:0/1963589935 conn(0x7ffac80c5e70 msgr2=0x7ffac80c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:51.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 -- 172.21.15.154:0/1963589935 shutdown_connections 2024-03-14T09:03:51.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.290+0000 7ffacd90d700 1 -- 172.21.15.154:0/1963589935 wait complete. 2024-03-14T09:03:51.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.291+0000 7ffacd90d700 1 Processor -- start 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.291+0000 7ffacd90d700 1 -- start start 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffac806c740 0x7ffac81db350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffac80c1100 0x7ffac814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80bcaa0 con 0x7ffac8150ec0 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80bbc40 con 0x7ffac80c1100 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffacd90d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81e9d30 con 0x7ffac806c740 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffac77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56448/0 (socket says 172.21.15.154:56448) 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffac77fe700 1 -- 172.21.15.154:0/1967363117 learned_addr learned my addr 172.21.15.154:0/1967363117 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffac67fc700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffac80c1100 0x7ffac814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.292+0000 7ffac6ffd700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffac806c740 0x7ffac81db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 -- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffac806c740 msgr2=0x7ffac81db350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffac806c740 0x7ffac81db350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 -- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffac80c1100 msgr2=0x7ffac814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffac80c1100 0x7ffac814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 -- 172.21.15.154:0/1967363117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0094a0 con 0x7ffac8150ec0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffac77fe700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ccd0 tx=0x7ffac000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000ab30 con 0x7ffac8150ec0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac000ac90 con 0x7ffac8150ec0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac81ea590 con 0x7ffac8150ec0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00070d0 con 0x7ffac8150ec0 2024-03-14T09:03:51.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.293+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac80bba10 con 0x7ffac8150ec0 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.294+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac81ea720 con 0x7ffac8150ec0 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.294+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffac0007230 con 0x7ffac8150ec0 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.295+0000 7ffaaffff700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 0x7ffab406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.295+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ffac0088900 con 0x7ffac8150ec0 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.295+0000 7ffac6ffd700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 0x7ffab406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.296+0000 7ffac6ffd700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 0x7ffab406ccf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffab8006fd0 tx=0x7ffab8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.300+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffac0053fb0 con 0x7ffac8150ec0 2024-03-14T09:03:51.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.539+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac8073f10 con 0x7ffac8150ec0 2024-03-14T09:03:51.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.540+0000 7ffaaffff700 1 -- 172.21.15.154:0/1967363117 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3541 (secure 0 0 0) 0x7ffac0018020 con 0x7ffac8150ec0 2024-03-14T09:03:51.540 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:51.540 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4164861738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":4164861738},{"type":"v1","addr":"172.21.15.191:6835","nonce":4164861738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/1096909305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":1096909305},{"type":"v1","addr":"172.21.15.154:6834","nonce":1096909305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5812,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/3880316034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":3880316034},{"type":"v1","addr":"172.21.15.191:6837","nonce":3880316034}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6360,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1720371613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1720371613},{"type":"v1","addr":"172.21.15.154:6835","nonce":1720371613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[]} 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 msgr2=0x7ffab406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 0x7ffab406ccf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffab8006fd0 tx=0x7ffab8008040 comp rx=0 tx=0).stop 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 msgr2=0x7ffac81eaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffac000ccd0 tx=0x7ffac000a060 comp rx=0 tx=0).stop 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 shutdown_connections 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffab406a830 0x7ffab406ccf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffac8150ec0 0x7ffac81eaa20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.542 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffac806c740 0x7ffac81db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 --2- 172.21.15.154:0/1967363117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffac80c1100 0x7ffac814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 >> 172.21.15.154:0/1967363117 conn(0x7ffac80c5e70 msgr2=0x7ffac80708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:51.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 shutdown_connections 2024-03-14T09:03:51.543 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.542+0000 7ffacd90d700 1 -- 172.21.15.154:0/1967363117 wait complete. 2024-03-14T09:03:51.543 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 39 2024-03-14T09:03:51.554 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-14T09:03:51.554 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-14T09:03:51.555 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T09:03:51.555 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:03:51.555 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-14T09:03:51.555 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-14T09:03:51.555 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T09:03:51.555 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:03:51.555 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-14T09:03:51.555 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-14T09:03:51.556 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T09:03:51.556 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:03:51.556 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-14T09:03:51.556 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-14T09:03:51.556 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T09:03:51.556 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:03:51.564 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T09:03:51.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-14T09:03:51.565 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T09:03:51.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-14T09:03:51.568 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T09:03:51.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-14T09:03:51.569 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T09:03:51.569 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-14T09:03:51.607 INFO:tasks.ceph.mds.a:Started 2024-03-14T09:03:51.607 DEBUG:teuthology.parallel:result is None 2024-03-14T09:03:51.607 INFO:tasks.ceph.mds.c:Started 2024-03-14T09:03:51.608 DEBUG:teuthology.parallel:result is None 2024-03-14T09:03:51.610 INFO:tasks.ceph.mds.b:Started 2024-03-14T09:03:51.610 DEBUG:teuthology.parallel:result is None 2024-03-14T09:03:51.611 INFO:tasks.ceph.mds.d:Started 2024-03-14T09:03:51.611 DEBUG:teuthology.parallel:result is None 2024-03-14T09:03:51.611 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-14T09:03:51.687 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:03:51.687 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:03:51.696 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-03-14T09:03:51.696 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2024-03-14T09:03:51.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.744+0000 7f25b0251700 1 Processor -- start 2024-03-14T09:03:51.744 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 -- start start 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a814ed60 con 0x7f25a80c4a80 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80c6b10 con 0x7f25a814e980 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.745+0000 7f25b0251700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8155820 con 0x7f25a8158a80 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25adfed700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:52830/0 (socket says 172.21.15.154:52830) 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ad7ec700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f25a80c4a80 0x7f25a8158370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:35488/0 (socket says 172.21.15.154:35488) 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 -- 172.21.15.154:0/1304620923 learned_addr learned my addr 172.21.15.154:0/1304620923 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:51.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25adfed700 1 --2- 172.21.15.154:0/1304620923 >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:54134/0 (socket says 172.21.15.154:54134) 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 -- 172.21.15.154:0/1304620923 >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 msgr2=0x7f25a81552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 --2- 172.21.15.154:0/1304620923 >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 -- 172.21.15.154:0/1304620923 >> v1:172.21.15.191:6789/0 conn(0x7f25a80c4a80 legacy=0x7f25a8158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 -- 172.21.15.154:0/1304620923 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80c6cf0 con 0x7f25a8158a80 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25adfed700 1 --2- 172.21.15.154:0/1304620923 >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25ae7ee700 1 --2- 172.21.15.154:0/1304620923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009e50 tx=0x7f25a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0104e393cc08943 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25acfeb700 1 -- 172.21.15.154:0/1304620923 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000e070 con 0x7f25a8158a80 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25acfeb700 1 -- 172.21.15.154:0/1304620923 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a000b910 con 0x7f25a8158a80 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.746+0000 7f25acfeb700 1 -- 172.21.15.154:0/1304620923 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a000bcc0 con 0x7f25a8158a80 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 -- 172.21.15.154:0/1304620923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 msgr2=0x7f25a80c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 --2- 172.21.15.154:0/1304620923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009e50 tx=0x7f25a000b330 comp rx=0 tx=0).stop 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 -- 172.21.15.154:0/1304620923 shutdown_connections 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 --2- 172.21.15.154:0/1304620923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a8158a80 0x7f25a80c4f10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 --2- 172.21.15.154:0/1304620923 >> v2:172.21.15.191:3300/0 conn(0x7f25a814e980 0x7f25a81552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 -- 172.21.15.154:0/1304620923 >> 172.21.15.154:0/1304620923 conn(0x7f25a80abcf0 msgr2=0x7f25a80ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:51.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 -- 172.21.15.154:0/1304620923 shutdown_connections 2024-03-14T09:03:51.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.747+0000 7f25b0251700 1 -- 172.21.15.154:0/1304620923 wait complete. 2024-03-14T09:03:51.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.748+0000 7f25b0251700 1 Processor -- start 2024-03-14T09:03:51.751 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.748+0000 7f25b0251700 1 -- start start 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a814e980 0x7f25a8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25a8158a80 0x7f25a81519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81eb020 con 0x7f25a80c4a80 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80adfa0 con 0x7f25a8158a80 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25b0251700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a80ae280 con 0x7f25a814e980 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56502/0 (socket says 172.21.15.154:56502) 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 -- 172.21.15.154:0/3627426226 learned_addr learned my addr 172.21.15.154:0/3627426226 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25ae7ee700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25a8158a80 0x7f25a81519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25ad7ec700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a814e980 0x7f25a8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 -- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a814e980 msgr2=0x7f25a8151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a814e980 0x7f25a8151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 -- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25a8158a80 msgr2=0x7f25a81519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25a8158a80 0x7f25a81519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 -- 172.21.15.154:0/3627426226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980094a0 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.749+0000 7f25adfed700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f259800ebe0 tx=0x7f259800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.750+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800c850 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.750+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800c9b0 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.750+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a0009b30 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.750+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800cd40 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.750+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a80ae840 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.751+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80aea60 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.751+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2598013470 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.751+0000 7f259e7fc700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 0x7f258c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.752+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2598088b70 con 0x7f25a80c4a80 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.752+0000 7f25ad7ec700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 0x7f258c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.752+0000 7f25ad7ec700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 0x7f258c06ccf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f25a8154830 tx=0x7f25a4007ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:51.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.756+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2598054230 con 0x7f25a80c4a80 2024-03-14T09:03:51.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:51.964+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f25a80697d0 con 0x7f25a80c4a80 2024-03-14T09:03:52.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.731+0000 7f259e7fc700 1 -- 172.21.15.154:0/3627426226 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f259800cea0 con 0x7f25a80c4a80 2024-03-14T09:03:52.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 msgr2=0x7f258c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:52.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 0x7f258c06ccf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f25a8154830 tx=0x7f25a4007ed0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 msgr2=0x7f25a8153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f259800ebe0 tx=0x7f259800eef0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 shutdown_connections 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25a8158a80 0x7f25a81519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f258c06a830 0x7f258c06ccf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25a80c4a80 0x7f25a8153ac0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 --2- 172.21.15.154:0/3627426226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25a814e980 0x7f25a8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 >> 172.21.15.154:0/3627426226 conn(0x7f25a80abcf0 msgr2=0x7f25a806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 shutdown_connections 2024-03-14T09:03:52.735 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.734+0000 7f25b0251700 1 -- 172.21.15.154:0/3627426226 wait complete. 2024-03-14T09:03:52.736 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2024-03-14T09:03:52.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.870+0000 7f6d3aa07700 1 Processor -- start 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 -- start start 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d3415d270 0x7f6d3415f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d340ac8e0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3406b310 con 0x7f6d3415d270 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d3aa07700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3406de30 con 0x7f6d34155df0 2024-03-14T09:03:52.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d38fa4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d3415d270 0x7f6d3415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d38fa4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d3415d270 0x7f6d3415f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56514/0 (socket says 172.21.15.154:56514) 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.871+0000 7f6d38fa4700 1 -- 172.21.15.154:0/1342455208 learned_addr learned my addr 172.21.15.154:0/1342455208 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d337fe700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d38fa4700 1 -- 172.21.15.154:0/1342455208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 msgr2=0x7f6d34156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d38fa4700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d38fa4700 1 -- 172.21.15.154:0/1342455208 >> v1:172.21.15.191:6789/0 conn(0x7f6d3414eaf0 legacy=0x7f6d3406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d38fa4700 1 -- 172.21.15.154:0/1342455208 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d340acb30 con 0x7f6d3415d270 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d337fe700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d38fa4700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3415d270 0x7f6d3415f660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009e50 tx=0x7f6d2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d0ffd517549ba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d32ffd700 1 -- 172.21.15.154:0/1342455208 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00e070 con 0x7f6d3415d270 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d32ffd700 1 -- 172.21.15.154:0/1342455208 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c003a20 con 0x7f6d3415d270 2024-03-14T09:03:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.872+0000 7f6d32ffd700 1 -- 172.21.15.154:0/1342455208 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c003dd0 con 0x7f6d3415d270 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 -- 172.21.15.154:0/1342455208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3415d270 msgr2=0x7f6d3415f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3415d270 0x7f6d3415f660 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009e50 tx=0x7f6d2c00b330 comp rx=0 tx=0).stop 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 -- 172.21.15.154:0/1342455208 shutdown_connections 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3415d270 0x7f6d3415f660 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/1342455208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 -- 172.21.15.154:0/1342455208 >> 172.21.15.154:0/1342455208 conn(0x7f6d340c3880 msgr2=0x7f6d340c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 -- 172.21.15.154:0/1342455208 shutdown_connections 2024-03-14T09:03:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.873+0000 7f6d3aa07700 1 -- 172.21.15.154:0/1342455208 wait complete. 2024-03-14T09:03:52.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.874+0000 7f6d3aa07700 1 Processor -- start 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.874+0000 7f6d3aa07700 1 -- start start 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 0x7f6d34150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c34e0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c2eb0 con 0x7f6d3415d270 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d3aa07700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341ee570 con 0x7f6d34155df0 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56528/0 (socket says 172.21.15.154:56528) 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 -- 172.21.15.154:0/2614851311 learned_addr learned my addr 172.21.15.154:0/2614851311 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 -- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 msgr2=0x7f6d34152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.875+0000 7f6d33fff700 1 -- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 msgr2=0x7f6d34150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d38fa4700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 0x7f6d34150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d33fff700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 0x7f6d34150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d33fff700 1 -- 172.21.15.154:0/2614851311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c009b30 con 0x7f6d3414eaf0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d38fa4700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 0x7f6d34150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d33fff700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c4d0 tx=0x7f6d2800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011790 con 0x7f6d3414eaf0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d280118f0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d341ee850 con 0x7f6d3414eaf0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011ca0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.876+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d341eb460 con 0x7f6d3414eaf0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.877+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d340697d0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.877+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6d2801c460 con 0x7f6d3414eaf0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.878+0000 7f6d30ff9700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 0x7f6d2006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.878+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d280887a0 con 0x7f6d3414eaf0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.878+0000 7f6d337fe700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 0x7f6d2006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.879+0000 7f6d337fe700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 0x7f6d2006ccf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d340c2c70 tx=0x7f6d24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:52.882+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6d28053fa0 con 0x7f6d3414eaf0 2024-03-14T09:03:53.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.157+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6d34072bf0 con 0x7f6d3414eaf0 2024-03-14T09:03:53.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.158+0000 7f6d30ff9700 1 -- 172.21.15.154:0/2614851311 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5081 (secure 0 0 0) 0x7f6d280575c0 con 0x7f6d3414eaf0 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlnCs5CxAA/mf9z7eSw0Fh5yqCxmvhww==", 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:03:53.158 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlq18XGBAA9Nr/cXAtRbeaKmJDCLUz6w==", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.159 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlvIuREhAAO1gOyqDP2tTJcb1IuOdQ0A==", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlAAuqHxAAy5SQz2lJHPwRad7HCIjjSg==", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:03:53.160 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCvvPJlDzrPMRAAEAP7kHfAaxZGM+W0rY0DIA==", 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2024-03-14T09:03:53.161 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCyvPJlgS2rMBAAoPtjrTbc7bMLMJAIUKD+vg==", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC1vPJltO6ULxAAvkKiSoRBoKxFz8zPjGCOnA==", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.162 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC4vPJlWIs/LhAAi5LaiHbS1svlQ0UesC9CLg==", 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.163 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQC+vPJl5TleMRAAjIGbvgOjoV3mJYIK1fQoYA==", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDBvPJlhiV5MBAAe7XI8s0kWQRAhvWWqMInrg==", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDEvPJlbghELxAASO+DuPJa7zXCH/KkhPi1WA==", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQDHvPJlRba5LhAASpiL/U05lgta2f4xKra5ZQ==", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlhDgFJBAA47spIEFGnChaoxmMQkaA+w==", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJl58xgJxAAfqdSCJED8MwHCScl1fqdFA==", 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.166 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw pool=cephfs2_data" 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlMjx9JRAAScnZoVr95MollZzYB2rQIA==", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.testuser", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQBevfJlxyP7HxAAx6ZCcXGsFBJT6rZUkFaUsA==", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow rw fsname=cephfs", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw fsname=cephfs, allow rw fsname=cephfs2", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs, allow rw tag cephfs data=cephfs2" 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCsvPJlSRLFBxAAkIg2TMRbiwcdMggjl+vMWw==", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:03:53.168 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQCrvPJlQsV4OxAADoB6pH163AX2+5II4EtdFw==", 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-03-14T09:03:53.169 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:03:53.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 msgr2=0x7f6d2006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 0x7f6d2006ccf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d340c2c70 tx=0x7f6d24008040 comp rx=0 tx=0).stop 2024-03-14T09:03:53.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 msgr2=0x7f6d341ea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c4d0 tx=0x7f6d2800c890 comp rx=0 tx=0).stop 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 shutdown_connections 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3415d270 0x7f6d34150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6d2006a830 0x7f6d2006ccf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d3414eaf0 0x7f6d341ea550 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 --2- 172.21.15.154:0/2614851311 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d34155df0 0x7f6d34152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 >> 172.21.15.154:0/2614851311 conn(0x7f6d340c3880 msgr2=0x7f6d34157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 shutdown_connections 2024-03-14T09:03:53.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.160+0000 7f6d3aa07700 1 -- 172.21.15.154:0/2614851311 wait complete. 2024-03-14T09:03:53.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.testuser 2024-03-14T09:03:53.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.295+0000 7f600697d700 1 Processor -- start 2024-03-14T09:03:53.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.295+0000 7f600697d700 1 -- start start 2024-03-14T09:03:53.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f600697d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f60000702f0 0x7f60000c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f600697d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f600697d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60000acd10 con 0x7f600014fbd0 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f600697d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000ac7b0 con 0x7f60000702f0 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f600697d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6000150050 con 0x7f60000c5d60 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f60000702f0 0x7f60000c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f6004f1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f6004f1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:52864/0 (socket says 172.21.15.154:52864) 2024-03-14T09:03:53.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f60000702f0 0x7f60000c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:54160/0 (socket says 172.21.15.154:54160) 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f6004f1a700 1 -- 172.21.15.154:0/3014670727 learned_addr learned my addr 172.21.15.154:0/3014670727 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5fff7fe700 1 --1- 172.21.15.154:0/3014670727 >> v1:172.21.15.191:6789/0 conn(0x7f600014fbd0 0x7f60000c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:35500/0 (socket says 172.21.15.154:35500) 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffeffd700 1 -- 172.21.15.154:0/3014670727 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230453570 0 0) 0x7f60000acd10 con 0x7f600014fbd0 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffeffd700 1 -- 172.21.15.154:0/3014670727 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe80035b0 con 0x7f600014fbd0 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 -- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 msgr2=0x7f60000c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 -- 172.21.15.154:0/3014670727 >> v1:172.21.15.191:6789/0 conn(0x7f600014fbd0 legacy=0x7f60000c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.296+0000 7f5ffffff700 1 -- 172.21.15.154:0/3014670727 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000ac9c0 con 0x7f60000702f0 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f6004f1a700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f5ffffff700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60000702f0 0x7f60000c4380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c897270f54d435d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f5ffeffd700 1 -- 172.21.15.154:0/3014670727 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011830 con 0x7f60000702f0 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f5ffeffd700 1 -- 172.21.15.154:0/3014670727 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011990 con 0x7f60000702f0 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f5ffeffd700 1 -- 172.21.15.154:0/3014670727 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011d00 con 0x7f60000702f0 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 -- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60000702f0 msgr2=0x7f60000c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60000702f0 0x7f60000c4380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 -- 172.21.15.154:0/3014670727 shutdown_connections 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000c5d60 0x7f60000c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 --2- 172.21.15.154:0/3014670727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60000702f0 0x7f60000c4380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 -- 172.21.15.154:0/3014670727 >> 172.21.15.154:0/3014670727 conn(0x7f60000a9bf0 msgr2=0x7f60000a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 -- 172.21.15.154:0/3014670727 shutdown_connections 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.297+0000 7f600697d700 1 -- 172.21.15.154:0/3014670727 wait complete. 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 Processor -- start 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 -- start start 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000702f0 0x7f60001eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f600014fbd0 0x7f6000142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000142f50 con 0x7f60000c5d60 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000151220 con 0x7f600014fbd0 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.299+0000 7f600697d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6000150cf0 con 0x7f60000702f0 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56534/0 (socket says 172.21.15.154:56534) 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 -- 172.21.15.154:0/1401641588 learned_addr learned my addr 172.21.15.154:0/1401641588 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5ffffff700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000702f0 0x7f60001eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f6004f1a700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f600014fbd0 0x7f6000142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 -- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000702f0 msgr2=0x7f60001eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000702f0 0x7f60001eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 -- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f600014fbd0 msgr2=0x7f6000142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f600014fbd0 0x7f6000142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 -- 172.21.15.154:0/1401641588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff8009b30 con 0x7f60000c5d60 2024-03-14T09:03:53.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5fff7fe700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c500 tx=0x7f5ff400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4005cb0 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff4005e10 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff00094a0 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4007310 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.300+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6000150f40 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.301+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60000c5a30 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.302+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5ff4007470 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.302+0000 7f5ffcff9700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 0x7f5fec06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.302+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ff4087db0 con 0x7f60000c5d60 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.302+0000 7f5ffffff700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 0x7f5fec06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.303+0000 7f5ffffff700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 0x7f5fec06ccf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009fd0 tx=0x7f5ff001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.306+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5ff40535f0 con 0x7f60000c5d60 2024-03-14T09:03:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.581+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.testuser"} v 0) v1 -- 0x7f60000697d0 con 0x7f60000c5d60 2024-03-14T09:03:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.585+0000 7f5ffcff9700 1 -- 172.21.15.154:0/1401641588 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.testuser"}]=0 v22) v1 ==== 85+0+0 (secure 0 0 0) 0x7f5ff4007720 con 0x7f60000c5d60 2024-03-14T09:03:53.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 msgr2=0x7f5fec06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 0x7f5fec06ccf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009fd0 tx=0x7f5ff001c040 comp rx=0 tx=0).stop 2024-03-14T09:03:53.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 msgr2=0x7f60001424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c500 tx=0x7f5ff400c810 comp rx=0 tx=0).stop 2024-03-14T09:03:53.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 shutdown_connections 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f600014fbd0 0x7f6000142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60000702f0 0x7f60001eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5fec06a830 0x7f5fec06ccf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 --2- 172.21.15.154:0/1401641588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60000c5d60 0x7f60001424d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 >> 172.21.15.154:0/1401641588 conn(0x7f60000a9bf0 msgr2=0x7f60000c2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 shutdown_connections 2024-03-14T09:03:53.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.587+0000 7f600697d700 1 -- 172.21.15.154:0/1401641588 wait complete. 2024-03-14T09:03:53.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:53.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 Processor -- start 2024-03-14T09:03:53.722 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 -- start start 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff55c0c1e90 0x7ff55c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c13ade0 con 0x7ff55c0a8540 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c06fb30 con 0x7ff55c0c3250 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.722+0000 7ff562ab6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55c06ec30 con 0x7ff55c0c1e90 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff55c0c1e90 0x7ff55c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff560852700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff55c0a8540 0x7ff55c154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46716/0 (socket says 172.21.15.154:46716) 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff55c0c1e90 0x7ff55c13a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56536/0 (socket says 172.21.15.154:56536) 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff560852700 1 -- 172.21.15.154:0/3799660232 learned_addr learned my addr 172.21.15.154:0/3799660232 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 -- 172.21.15.154:0/3799660232 >> v1:172.21.15.191:6789/0 conn(0x7ff55c0c3250 legacy=0x7ff55c0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:53.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff55b7fe700 1 -- 172.21.15.154:0/3799660232 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568010862 0 0) 0x7ff550001110 con 0x7ff55c0a8540 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 -- 172.21.15.154:0/3799660232 >> v1:172.21.15.154:6789/0 conn(0x7ff55c0a8540 legacy=0x7ff55c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 -- 172.21.15.154:0/3799660232 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c06ef10 con 0x7ff55c0c1e90 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff561053700 1 --2- 172.21.15.154:0/3799660232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c13a8a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff554009d90 tx=0x7ff55400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffe88f0bfa9bacb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff55b7fe700 1 -- 172.21.15.154:0/3799660232 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400e070 con 0x7ff55c0c1e90 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff55b7fe700 1 -- 172.21.15.154:0/3799660232 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55400bbd0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.723+0000 7ff55b7fe700 1 -- 172.21.15.154:0/3799660232 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554012430 con 0x7ff55c0c1e90 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 -- 172.21.15.154:0/3799660232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 msgr2=0x7ff55c13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 --2- 172.21.15.154:0/3799660232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c13a8a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff554009d90 tx=0x7ff55400b330 comp rx=0 tx=0).stop 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 -- 172.21.15.154:0/3799660232 shutdown_connections 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 --2- 172.21.15.154:0/3799660232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c13a8a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 -- 172.21.15.154:0/3799660232 >> 172.21.15.154:0/3799660232 conn(0x7ff55c0c3880 msgr2=0x7ff55c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 -- 172.21.15.154:0/3799660232 shutdown_connections 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.724+0000 7ff562ab6700 1 -- 172.21.15.154:0/3799660232 wait complete. 2024-03-14T09:03:53.726 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.725+0000 7ff562ab6700 1 Processor -- start 2024-03-14T09:03:53.727 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.725+0000 7ff562ab6700 1 -- start start 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff55c0a8540 0x7ff55c151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff55c0c3250 0x7ff55c1eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1e9130 con 0x7ff55c0c1e90 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c0ac490 con 0x7ff55c0a8540 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff562ab6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c14f590 con 0x7ff55c0c3250 2024-03-14T09:03:53.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56538/0 (socket says 172.21.15.154:56538) 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 -- 172.21.15.154:0/544473414 learned_addr learned my addr 172.21.15.154:0/544473414 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff560852700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff55c0a8540 0x7ff55c151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff561053700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff55c0c3250 0x7ff55c1eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 -- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff55c0c3250 msgr2=0x7ff55c1eb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff55c0c3250 0x7ff55c1eb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 -- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff55c0a8540 msgr2=0x7ff55c151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff55c0a8540 0x7ff55c151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.726+0000 7ff55bfff700 1 -- 172.21.15.154:0/544473414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009a70 con 0x7ff55c0c1e90 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff55bfff700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c4e0 tx=0x7ff54800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480052d0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548005430 con 0x7ff55c0c1e90 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480038c0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.730 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c14f870 con 0x7ff55c0c1e90 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.727+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c14fe30 con 0x7ff55c0c1e90 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.728+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c06d6d0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.728+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff548003a20 con 0x7ff55c0c1e90 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.728+0000 7ff5597fa700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 0x7ff54006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.729+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff548087cd0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.729+0000 7ff560852700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 0x7ff54006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:53.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.729+0000 7ff560852700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 0x7ff54006cd20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550007c40 tx=0x7ff550007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:53.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.733+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff5480534d0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.927+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff55c155200 con 0x7ff55c0c1e90 2024-03-14T09:03:53.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.928+0000 7ff5597fa700 1 -- 172.21.15.154:0/544473414 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+12438 (secure 0 0 0) 0x7ff548056af0 con 0x7ff55c0c1e90 2024-03-14T09:03:53.927 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:53.928 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":66,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:52.728804+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 msgr2=0x7ff54006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 0x7ff54006cd20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff550007c40 tx=0x7ff550007b90 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 msgr2=0x7ff55c150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c4e0 tx=0x7ff54800c7f0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 shutdown_connections 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff55c0c3250 0x7ff55c1eb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff55c0a8540 0x7ff55c151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff54006a860 0x7ff54006cd20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 --2- 172.21.15.154:0/544473414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff55c0c1e90 0x7ff55c150350 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 >> 172.21.15.154:0/544473414 conn(0x7ff55c0c3880 msgr2=0x7ff55c139890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 shutdown_connections 2024-03-14T09:03:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:53.930+0000 7ff562ab6700 1 -- 172.21.15.154:0/544473414 wait complete. 2024-03-14T09:03:53.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:03:53.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:03:54.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.067+0000 7fa6fdb7b700 1 Processor -- start 2024-03-14T09:03:54.067 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.067+0000 7fa6fdb7b700 1 -- start start 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6fdb7b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6f80c1780 0x7fa6f806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6fdb7b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 0x7fa6f815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6fdb7b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8156bf0 con 0x7fa6f80a9650 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6fdb7b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f8156050 con 0x7fa6f80c1780 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6fdb7b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80c1c00 con 0x7fa6f806acb0 2024-03-14T09:03:54.069 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6f80c1780 0x7fa6f806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa6f80a9650 0x7fa6f81564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46732/0 (socket says 172.21.15.154:46732) 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f77fe700 1 -- 172.21.15.154:0/1190707517 learned_addr learned my addr 172.21.15.154:0/1190707517 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 -- 172.21.15.154:0/1190707517 >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 msgr2=0x7fa6f815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f7fff700 1 --2- 172.21.15.154:0/1190707517 >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 0x7fa6f815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f67fc700 1 -- 172.21.15.154:0/1190707517 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416848695 0 0) 0x7fa6ec001110 con 0x7fa6f80a9650 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 --2- 172.21.15.154:0/1190707517 >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 0x7fa6f815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 -- 172.21.15.154:0/1190707517 >> v1:172.21.15.154:6789/0 conn(0x7fa6f80a9650 legacy=0x7fa6f81564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 -- 172.21.15.154:0/1190707517 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80a86f0 con 0x7fa6f80c1780 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f7fff700 1 --2- 172.21.15.154:0/1190707517 >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 0x7fa6f815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f6ffd700 1 --2- 172.21.15.154:0/1190707517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f80c1780 0x7fa6f806a770 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8072e40 tx=0x7fa6e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b32088c36d824ba server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f67fc700 1 -- 172.21.15.154:0/1190707517 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011830 con 0x7fa6f80c1780 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f67fc700 1 -- 172.21.15.154:0/1190707517 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8011990 con 0x7fa6f80c1780 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.068+0000 7fa6f67fc700 1 -- 172.21.15.154:0/1190707517 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8011d20 con 0x7fa6f80c1780 2024-03-14T09:03:54.070 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1190707517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f80c1780 msgr2=0x7fa6f806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1190707517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f80c1780 0x7fa6f806a770 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8072e40 tx=0x7fa6e8009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1190707517 shutdown_connections 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1190707517 >> v2:172.21.15.191:3300/0 conn(0x7fa6f806acb0 0x7fa6f815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1190707517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f80c1780 0x7fa6f806a770 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1190707517 >> 172.21.15.154:0/1190707517 conn(0x7fa6f80c3880 msgr2=0x7fa6f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1190707517 shutdown_connections 2024-03-14T09:03:54.071 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.069+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1190707517 wait complete. 2024-03-14T09:03:54.073 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.070+0000 7fa6fdb7b700 1 Processor -- start 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.070+0000 7fa6fdb7b700 1 -- start start 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f80a9650 0x7fa6f8140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f80c1780 0x7fa6f8140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81ee7d0 con 0x7fa6f806acb0 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8140fa0 con 0x7fa6f80c1780 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6fdb7b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c5d10 con 0x7fa6f80a9650 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56568/0 (socket says 172.21.15.154:56568) 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 -- 172.21.15.154:0/1381858346 learned_addr learned my addr 172.21.15.154:0/1381858346 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f6ffd700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f80a9650 0x7fa6f8140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f7fff700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f80c1780 0x7fa6f8140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 -- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f80a9650 msgr2=0x7fa6f8140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.074 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f80a9650 0x7fa6f8140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 -- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f80c1780 msgr2=0x7fa6f8140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f80c1780 0x7fa6f8140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.071+0000 7fa6f77fe700 1 -- 172.21.15.154:0/1381858346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80094a0 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6f77fe700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c550 tx=0x7fa6ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004c80 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec004e00 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec007310 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f81eda10 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.072+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81edfd0 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.073+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f80697d0 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.073+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa6ec007470 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.074+0000 7fa6dffff700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 0x7fa6e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.074+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6ec088580 con 0x7fa6f806acb0 2024-03-14T09:03:54.075 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.074+0000 7fa6f6ffd700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 0x7fa6e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.076 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.074+0000 7fa6f6ffd700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 0x7fa6e406ccf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8066c00 tx=0x7fa6e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.077 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.078+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa6ec053d80 con 0x7fa6f806acb0 2024-03-14T09:03:54.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.268+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6f80a8960 con 0x7fa6f806acb0 2024-03-14T09:03:54.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.268+0000 7fa6dffff700 1 -- 172.21.15.154:0/1381858346 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6ec0573a0 con 0x7fa6f806acb0 2024-03-14T09:03:54.268 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:03:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 msgr2=0x7fa6e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 0x7fa6e406ccf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8066c00 tx=0x7fa6e8008030 comp rx=0 tx=0).stop 2024-03-14T09:03:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 msgr2=0x7fa6f814e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00c550 tx=0x7fa6ec00c910 comp rx=0 tx=0).stop 2024-03-14T09:03:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 shutdown_connections 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6f80c1780 0x7fa6f8140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa6e406a830 0x7fa6e406ccf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6f806acb0 0x7fa6f814e550 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 --2- 172.21.15.154:0/1381858346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6f80a9650 0x7fa6f8140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 >> 172.21.15.154:0/1381858346 conn(0x7fa6f80c3880 msgr2=0x7fa6f8142600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 shutdown_connections 2024-03-14T09:03:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.270+0000 7fa6fdb7b700 1 -- 172.21.15.154:0/1381858346 wait complete. 2024-03-14T09:03:54.282 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:03:54.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.404+0000 7f76deab1700 1 Processor -- start 2024-03-14T09:03:54.404 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.404+0000 7f76deab1700 1 -- start start 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76deab1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 0x7f76d806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76deab1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f76d806acb0 0x7f76d8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76deab1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80ac8e0 con 0x7f76d8156330 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76deab1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d806b310 con 0x7f76d806acb0 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76deab1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d806de30 con 0x7f76d814eaf0 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dd04e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f76d8156330 0x7f76d8157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46734/0 (socket says 172.21.15.154:46734) 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dd04e700 1 -- 172.21.15.154:0/3172411394 learned_addr learned my addr 172.21.15.154:0/3172411394 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 --2- 172.21.15.154:0/3172411394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 0x7f76d806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535621137 0 0) 0x7f76d80ac8e0 con 0x7f76d8156330 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76d7fff700 1 --2- 172.21.15.154:0/3172411394 >> v2:172.21.15.191:3300/0 conn(0x7f76d806acb0 0x7f76d8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c80035b0 con 0x7f76d8156330 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 445003101 0 0) 0x7f76c80035b0 con 0x7f76d8156330 2024-03-14T09:03:54.407 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 -- 172.21.15.154:0/3172411394 >> v2:172.21.15.191:3300/0 conn(0x7f76d806acb0 msgr2=0x7f76d8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 --2- 172.21.15.154:0/3172411394 >> v2:172.21.15.191:3300/0 conn(0x7f76d806acb0 0x7f76d8155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 -- 172.21.15.154:0/3172411394 >> v1:172.21.15.154:6789/0 conn(0x7f76d8156330 legacy=0x7f76d8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 -- 172.21.15.154:0/3172411394 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80acb30 con 0x7f76d814eaf0 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.405+0000 7f76dc84d700 1 --2- 172.21.15.154:0/3172411394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 0x7f76d806a770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009840 tx=0x7f76cc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0b3dd5dde487dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00019f0 con 0x7f76d814eaf0 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc004030 con 0x7f76d814eaf0 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76d77fe700 1 -- 172.21.15.154:0/3172411394 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0043a0 con 0x7f76d814eaf0 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 -- 172.21.15.154:0/3172411394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 msgr2=0x7f76d806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 --2- 172.21.15.154:0/3172411394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 0x7f76d806a770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009840 tx=0x7f76cc009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 -- 172.21.15.154:0/3172411394 shutdown_connections 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 --2- 172.21.15.154:0/3172411394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d814eaf0 0x7f76d806a770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 --2- 172.21.15.154:0/3172411394 >> v2:172.21.15.191:3300/0 conn(0x7f76d806acb0 0x7f76d8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 -- 172.21.15.154:0/3172411394 >> 172.21.15.154:0/3172411394 conn(0x7f76d80c3880 msgr2=0x7f76d80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 -- 172.21.15.154:0/3172411394 shutdown_connections 2024-03-14T09:03:54.408 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.406+0000 7f76deab1700 1 -- 172.21.15.154:0/3172411394 wait complete. 2024-03-14T09:03:54.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.407+0000 7f76deab1700 1 Processor -- start 2024-03-14T09:03:54.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.407+0000 7f76deab1700 1 -- start start 2024-03-14T09:03:54.411 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d806acb0 0x7f76d81497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76d8156330 0x7f76d8151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8150df0 con 0x7f76d814eaf0 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d814f370 con 0x7f76d8156330 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76deab1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d814f650 con 0x7f76d806acb0 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76d7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56576/0 (socket says 172.21.15.154:56576) 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.408+0000 7f76d7fff700 1 -- 172.21.15.154:0/2597010494 learned_addr learned my addr 172.21.15.154:0/2597010494 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76dd04e700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76d8156330 0x7f76d8151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 -- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d806acb0 msgr2=0x7f76d81497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d806acb0 0x7f76d81497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 -- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76d8156330 msgr2=0x7f76d8151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76d8156330 0x7f76d8151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.412 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 -- 172.21.15.154:0/2597010494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0094a0 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d7fff700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e870 tx=0x7f76c400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4009590 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c400ee70 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d8151000 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c400ca50 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.409+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d8153740 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.410+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d814f870 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.410+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f76c400cbb0 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.411+0000 7f76d57fa700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 0x7f76bc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.411+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76c4088a00 con 0x7f76d814eaf0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.411+0000 7f76dc84d700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 0x7f76bc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.413 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.412+0000 7f76dc84d700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 0x7f76bc06ccf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.415 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.415+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f76c40540f0 con 0x7f76d814eaf0 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.612+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f76d80697d0 con 0x7f76d814eaf0 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.613+0000 7f76d57fa700 1 -- 172.21.15.154:0/2597010494 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) v1 ==== 84+0+65 (secure 0 0 0) 0x7f76c400ce60 con 0x7f76d814eaf0 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:03:54.613 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:03:54.614 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 msgr2=0x7f76bc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 0x7f76bc06ccf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc010000 comp rx=0 tx=0).stop 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 msgr2=0x7f76d8151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e870 tx=0x7f76c400ec30 comp rx=0 tx=0).stop 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 shutdown_connections 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76d8156330 0x7f76d8151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76d806acb0 0x7f76d81497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f76bc06a830 0x7f76bc06ccf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 --2- 172.21.15.154:0/2597010494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d814eaf0 0x7f76d8151520 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 >> 172.21.15.154:0/2597010494 conn(0x7f76d80c3880 msgr2=0x7f76d814d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:54.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 shutdown_connections 2024-03-14T09:03:54.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.615+0000 7f76deab1700 1 -- 172.21.15.154:0/2597010494 wait complete. 2024-03-14T09:03:54.630 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-14T09:03:54.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-14T09:03:54.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.757+0000 7fed64ba6700 1 Processor -- start 2024-03-14T09:03:54.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.757+0000 7fed64ba6700 1 -- start start 2024-03-14T09:03:54.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed64ba6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed6006f740 0x7fed601564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed64ba6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 0x7fed6015d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed64ba6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed60153460 con 0x7fed6006f360 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed64ba6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed60152f70 con 0x7fed6006f740 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed64ba6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6006fba0 con 0x7fed6015ae60 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed6006f740 0x7fed601564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5e59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fed6006f360 0x7fed60155dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46736/0 (socket says 172.21.15.154:46736) 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5e59c700 1 -- 172.21.15.154:0/1764441152 learned_addr learned my addr 172.21.15.154:0/1764441152 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5ed9d700 1 --2- 172.21.15.154:0/1764441152 >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 0x7fed6015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5d59a700 1 -- 172.21.15.154:0/1764441152 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910815712 0 0) 0x7fed60153460 con 0x7fed6006f360 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5d59a700 1 -- 172.21.15.154:0/1764441152 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed480035b0 con 0x7fed6006f360 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5dd9b700 1 -- 172.21.15.154:0/1764441152 >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 msgr2=0x7fed6015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:54.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5dd9b700 1 --2- 172.21.15.154:0/1764441152 >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 0x7fed6015d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5dd9b700 1 -- 172.21.15.154:0/1764441152 >> v1:172.21.15.154:6789/0 conn(0x7fed6006f360 legacy=0x7fed60155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5dd9b700 1 -- 172.21.15.154:0/1764441152 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60072820 con 0x7fed6006f740 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.758+0000 7fed5ed9d700 1 --2- 172.21.15.154:0/1764441152 >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 0x7fed6015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed5dd9b700 1 --2- 172.21.15.154:0/1764441152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f740 0x7fed601564d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fed600c1f10 tx=0x7fed50009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f542794e694e09e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed5d59a700 1 -- 172.21.15.154:0/1764441152 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50011830 con 0x7fed6006f740 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed5d59a700 1 -- 172.21.15.154:0/1764441152 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed50011990 con 0x7fed6006f740 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed5d59a700 1 -- 172.21.15.154:0/1764441152 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50011d20 con 0x7fed6006f740 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 -- 172.21.15.154:0/1764441152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f740 msgr2=0x7fed601564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1764441152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f740 0x7fed601564d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fed600c1f10 tx=0x7fed50009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 -- 172.21.15.154:0/1764441152 shutdown_connections 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1764441152 >> v2:172.21.15.191:3300/0 conn(0x7fed6015ae60 0x7fed6015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1764441152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f740 0x7fed601564d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 -- 172.21.15.154:0/1764441152 >> 172.21.15.154:0/1764441152 conn(0x7fed600c39f0 msgr2=0x7fed600c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 -- 172.21.15.154:0/1764441152 shutdown_connections 2024-03-14T09:03:54.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.759+0000 7fed64ba6700 1 -- 172.21.15.154:0/1764441152 wait complete. 2024-03-14T09:03:54.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 Processor -- start 2024-03-14T09:03:54.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 -- start start 2024-03-14T09:03:54.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed6006f740 0x7fed60146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed6015ae60 0x7fed601e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c1300 con 0x7fed6006f360 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed601e8970 con 0x7fed6006f740 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed64ba6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed60147470 con 0x7fed6015ae60 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.761+0000 7fed5e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56582/0 (socket says 172.21.15.154:56582) 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 -- 172.21.15.154:0/1754613896 learned_addr learned my addr 172.21.15.154:0/1754613896 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5dd9b700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed6006f740 0x7fed60146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5ed9d700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed6015ae60 0x7fed601e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 -- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed6015ae60 msgr2=0x7fed601e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed6015ae60 0x7fed601e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 -- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed6006f740 msgr2=0x7fed60146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed6006f740 0x7fed60146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 -- 172.21.15.154:0/1754613896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500094a0 con 0x7fed6006f360 2024-03-14T09:03:54.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed5e59c700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fed54001eb0 tx=0x7fed5400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54005d70 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed540070d0 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed60147750 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54007460 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.762+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed601ebdb0 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.763+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed600697d0 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.764+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fed540075c0 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.764+0000 7fed4effd700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 0x7fed4006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.764+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed54087d70 con 0x7fed6006f360 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.764+0000 7fed5dd9b700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 0x7fed4006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:54.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.765+0000 7fed5dd9b700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 0x7fed4006ccf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fed500090b0 tx=0x7fed50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:54.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.768+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fed54053570 con 0x7fed6006f360 2024-03-14T09:03:54.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:54.964+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed60146b50 con 0x7fed6006f360 2024-03-14T09:03:55.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.742+0000 7fed4effd700 1 -- 172.21.15.154:0/1754613896 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed54056b90 con 0x7fed6006f360 2024-03-14T09:03:55.746 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 msgr2=0x7fed4006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 0x7fed4006ccf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fed500090b0 tx=0x7fed50008030 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 msgr2=0x7fed601541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fed54001eb0 tx=0x7fed5400c7b0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 shutdown_connections 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed6015ae60 0x7fed601e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fed4006a830 0x7fed4006ccf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed6006f360 0x7fed601541c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 --2- 172.21.15.154:0/1754613896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed6006f740 0x7fed60146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.747+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 >> 172.21.15.154:0/1754613896 conn(0x7fed600c39f0 msgr2=0x7fed600711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:55.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.748+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 shutdown_connections 2024-03-14T09:03:55.748 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.748+0000 7fed64ba6700 1 -- 172.21.15.154:0/1754613896 wait complete. 2024-03-14T09:03:55.751 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2024-03-14T09:03:55.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:03:55.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.896+0000 7f6e16017700 1 Processor -- start 2024-03-14T09:03:55.896 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.896+0000 7f6e16017700 1 -- start start 2024-03-14T09:03:55.902 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e16017700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6e1015df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e16017700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100acd10 con 0x7f6e100702f0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e16017700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100ac9c0 con 0x7f6e100c4380 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e16017700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100ac7b0 con 0x7f6e1014fbd0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e0f7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6e100702f0 0x7f6e100c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46744/0 (socket says 172.21.15.154:46744) 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e0f7fe700 1 -- 172.21.15.154:0/1776693748 learned_addr learned my addr 172.21.15.154:0/1776693748 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.897+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129088864 0 0) 0x7f6e100acd10 con 0x7f6e100702f0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dfc0035b0 con 0x7f6e100702f0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6e1015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932820098 0 0) 0x7f6dfc0035b0 con 0x7f6e100702f0 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 msgr2=0x7f6e1015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6e1015df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 >> v1:172.21.15.191:6789/0 conn(0x7f6e100c4380 legacy=0x7f6e100c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.903 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e10150020 con 0x7f6e100702f0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f6df8001670 con 0x7f6e100702f0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 >> v1:172.21.15.154:6789/0 conn(0x7f6e100702f0 legacy=0x7f6e100c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6dfc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dfc003fb0 0x7f6dfc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dfc004d90 0x7f6dfc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100ac7b0 con 0x7f6dfc004d90 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc00baf0 con 0x7f6dfc003fb0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc00bcb0 con 0x7f6e1014fbd0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dfc004d90 0x7f6dfc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0f7fe700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6dfc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0effd700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dfc003fb0 0x7f6dfc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 -- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 msgr2=0x7f6dfc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1014fbd0 0x7f6dfc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 -- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dfc003fb0 msgr2=0x7f6dfc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:55.904 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dfc003fb0 0x7f6dfc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 -- 172.21.15.154:0/1776693748 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0035b0 con 0x7f6dfc004d90 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.898+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dfc004d90 0x7f6dfc005210 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009fc0 tx=0x7f6dfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4760d0cc2cd60ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800ec30 con 0x7f6dfc004d90 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0800f040 con 0x7f6dfc004d90 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e0e7fc700 1 -- 172.21.15.154:0/1776693748 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08014480 con 0x7f6dfc004d90 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 -- 172.21.15.154:0/1776693748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dfc004d90 msgr2=0x7f6dfc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 --2- 172.21.15.154:0/1776693748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6dfc004d90 0x7f6dfc005210 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009fc0 tx=0x7f6dfc005880 comp rx=0 tx=0).stop 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e0ffff700 1 -- 172.21.15.154:0/1776693748 reap_dead start 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 -- 172.21.15.154:0/1776693748 shutdown_connections 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 -- 172.21.15.154:0/1776693748 >> 172.21.15.154:0/1776693748 conn(0x7f6e100a9bf0 msgr2=0x7f6e100a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 -- 172.21.15.154:0/1776693748 shutdown_connections 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.899+0000 7f6e16017700 1 -- 172.21.15.154:0/1776693748 wait complete. 2024-03-14T09:03:55.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.900+0000 7f6e16017700 1 Processor -- start 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 -- start start 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 0x7f6e1006c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 0x7f6e1006cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1006fa10 con 0x7f6e100c12a0 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1006d2d0 con 0x7f6e101dbf60 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e16017700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1006ee00 con 0x7f6e1006e970 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56602/0 (socket says 172.21.15.154:56602) 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e0f7fe700 1 -- 172.21.15.154:0/1310402933 learned_addr learned my addr 172.21.15.154:0/1310402933 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e0effd700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 0x7f6e1006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.901+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 0x7f6e1006cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 -- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 msgr2=0x7f6e1006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 0x7f6e1006c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 -- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 msgr2=0x7f6e1006cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 0x7f6e1006cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 -- 172.21.15.154:0/1310402933 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1006f0e0 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0ffff700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 0x7f6e1006cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0effd700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 0x7f6e1006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e0f7fe700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007290 tx=0x7f6df8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800f8d0 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800fa50 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1006f3c0 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800fe00 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.902+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e101ed1b0 con 0x7f6e100c12a0 2024-03-14T09:03:55.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.903+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e100c2510 con 0x7f6e100c12a0 2024-03-14T09:03:55.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.904+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6df801d460 con 0x7f6e100c12a0 2024-03-14T09:03:55.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.904+0000 7f6e07fff700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 0x7f6df006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:55.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.904+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6df8088c90 con 0x7f6e100c12a0 2024-03-14T09:03:55.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.905+0000 7f6e0effd700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 0x7f6df006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:55.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.905+0000 7f6e0effd700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 0x7f6df006ccf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6e101ead30 tx=0x7f6e00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:55.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:55.910+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6df8054290 con 0x7f6e100c12a0 2024-03-14T09:03:56.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.104+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6e100697d0 con 0x7f6e100c12a0 2024-03-14T09:03:56.244 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:03:56.246 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:03:56.756 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.757+0000 7f6e07fff700 1 -- 172.21.15.154:0/1310402933 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6df80578b0 con 0x7f6e100c12a0 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 msgr2=0x7f6df006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 0x7f6df006ccf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6e101ead30 tx=0x7f6e00009040 comp rx=0 tx=0).stop 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 msgr2=0x7f6e100c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6df8007290 tx=0x7f6df8007650 comp rx=0 tx=0).stop 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 shutdown_connections 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e101dbf60 0x7f6e1006cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6df006a830 0x7f6df006ccf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.760 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e100c12a0 0x7f6e100c1680 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 --2- 172.21.15.154:0/1310402933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1006e970 0x7f6e1006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.761+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 >> 172.21.15.154:0/1310402933 conn(0x7f6e100a9bf0 msgr2=0x7f6e100a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:56.761 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.762+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 shutdown_connections 2024-03-14T09:03:56.762 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.763+0000 7f6e16017700 1 -- 172.21.15.154:0/1310402933 wait complete. 2024-03-14T09:03:56.765 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2024-03-14T09:03:56.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-14T09:03:56.905 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 Processor -- start 2024-03-14T09:03:56.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 -- start start 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1fd00c1780 0x7f1fd006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1fd006acb0 0x7f1fd015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd0156bf0 con 0x7f1fd00a9650 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd0156050 con 0x7f1fd006acb0 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd6f09700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd00c1c00 con 0x7f1fd00c1780 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.907+0000 7f1fd54a6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1fd006acb0 0x7f1fd015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd4ca5700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1fd00a9650 0x7f1fd01564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46746/0 (socket says 172.21.15.154:46746) 2024-03-14T09:03:56.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd4ca5700 1 -- 172.21.15.154:0/187925372 learned_addr learned my addr 172.21.15.154:0/187925372 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd54a6700 1 -- 172.21.15.154:0/187925372 >> v2:172.21.15.191:3300/0 conn(0x7f1fd00c1780 msgr2=0x7f1fd006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fcffff700 1 --2- 172.21.15.154:0/187925372 >> v2:172.21.15.191:3300/0 conn(0x7f1fd00c1780 0x7f1fd006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd54a6700 1 --2- 172.21.15.154:0/187925372 >> v2:172.21.15.191:3300/0 conn(0x7f1fd00c1780 0x7f1fd006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd54a6700 1 -- 172.21.15.154:0/187925372 >> v1:172.21.15.154:6789/0 conn(0x7f1fd00a9650 legacy=0x7f1fd01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fcf7fe700 1 -- 172.21.15.154:0/187925372 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071515500 0 0) 0x7f1fc4001110 con 0x7f1fd00a9650 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd54a6700 1 -- 172.21.15.154:0/187925372 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd00a86f0 con 0x7f1fd006acb0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fd54a6700 1 --2- 172.21.15.154:0/187925372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd015cfd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00a9fb0 tx=0x7f1fc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49df25818ab36e4e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fcf7fe700 1 -- 172.21.15.154:0/187925372 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800e070 con 0x7f1fd006acb0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fcf7fe700 1 -- 172.21.15.154:0/187925372 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc800bbd0 con 0x7f1fd006acb0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.908+0000 7f1fcf7fe700 1 -- 172.21.15.154:0/187925372 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8012430 con 0x7f1fd006acb0 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 -- 172.21.15.154:0/187925372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 msgr2=0x7f1fd015cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/187925372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd015cfd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00a9fb0 tx=0x7f1fc800b330 comp rx=0 tx=0).stop 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 -- 172.21.15.154:0/187925372 shutdown_connections 2024-03-14T09:03:56.909 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/187925372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd015cfd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/187925372 >> v2:172.21.15.191:3300/0 conn(0x7f1fd00c1780 0x7f1fd006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 -- 172.21.15.154:0/187925372 >> 172.21.15.154:0/187925372 conn(0x7f1fd00c3880 msgr2=0x7f1fd00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:56.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 -- 172.21.15.154:0/187925372 shutdown_connections 2024-03-14T09:03:56.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.909+0000 7f1fd6f09700 1 -- 172.21.15.154:0/187925372 wait complete. 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.910+0000 7f1fd6f09700 1 Processor -- start 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.910+0000 7f1fd6f09700 1 -- start start 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00a9650 0x7f1fd014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd00c1780 0x7f1fd0140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd0153f50 con 0x7f1fd006acb0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd014f820 con 0x7f1fd00a9650 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd6f09700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd01ec180 con 0x7f1fd00c1780 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd4ca5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd4ca5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56618/0 (socket says 172.21.15.154:56618) 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.911+0000 7f1fd4ca5700 1 -- 172.21.15.154:0/3704515686 learned_addr learned my addr 172.21.15.154:0/3704515686 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fcffff700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00a9650 0x7f1fd014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd54a6700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd00c1780 0x7f1fd0140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 -- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd00c1780 msgr2=0x7f1fd0140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd00c1780 0x7f1fd0140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 -- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00a9650 msgr2=0x7f1fd014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00a9650 0x7f1fd014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 -- 172.21.15.154:0/3704515686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8009a70 con 0x7f1fd006acb0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd4ca5700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400c580 tx=0x7f1fc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:56.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4004c80 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4004e00 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4007310 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd01ec6b0 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.912+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd0153b70 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.913+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd00697d0 con 0x7f1fd006acb0 2024-03-14T09:03:56.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.914+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1fc4007470 con 0x7f1fd006acb0 2024-03-14T09:03:56.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.914+0000 7f1fcd7fa700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 0x7f1fb406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:56.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.914+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1fc40887a0 con 0x7f1fd006acb0 2024-03-14T09:03:56.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.914+0000 7f1fcffff700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 0x7f1fb406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:56.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.915+0000 7f1fcffff700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 0x7f1fb406ccf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1fd01e9330 tx=0x7f1fbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:56.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:56.919+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1fc4053cb0 con 0x7f1fd006acb0 2024-03-14T09:03:57.157 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.158+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1fd0156bf0 con 0x7f1fd006acb0 2024-03-14T09:03:57.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.165+0000 7f1fcd7fa700 1 -- 172.21.15.154:0/3704515686 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v41) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1fd0156bf0 con 0x7f1fd006acb0 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 msgr2=0x7f1fb406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 0x7f1fb406ccf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1fd01e9330 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 msgr2=0x7f1fd0155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400c580 tx=0x7f1fc400c940 comp rx=0 tx=0).stop 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 shutdown_connections 2024-03-14T09:03:57.176 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1fd00c1780 0x7f1fd0140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1fb406a830 0x7f1fb406ccf0 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1fd01e9330 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fd006acb0 0x7f1fd0155790 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 --2- 172.21.15.154:0/3704515686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fd00a9650 0x7f1fd014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.177+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 >> 172.21.15.154:0/3704515686 conn(0x7f1fd00c3880 msgr2=0x7f1fd0071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.179+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 shutdown_connections 2024-03-14T09:03:57.177 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.179+0000 7f1fd6f09700 1 -- 172.21.15.154:0/3704515686 wait complete. 2024-03-14T09:03:57.180 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 12 and data pool 13 2024-03-14T09:03:57.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:57.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.319+0000 7f91d29e6700 1 Processor -- start 2024-03-14T09:03:57.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.319+0000 7f91d29e6700 1 -- start start 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d29e6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f91cc0c8350 0x7f91cc1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d29e6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc157e80 con 0x7f91cc154d30 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d29e6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc158120 con 0x7f91cc0c7f70 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d29e6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc158ac0 con 0x7f91cc0c8350 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d0f83700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f91cc154d30 0x7f91cc15f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46756/0 (socket says 172.21.15.154:46756) 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91d0f83700 1 -- 172.21.15.154:0/1875682643 learned_addr learned my addr 172.21.15.154:0/1875682643 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/1875682643 >> v2:172.21.15.191:3300/0 conn(0x7f91cc0c8350 0x7f91cc1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.320+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807505479 0 0) 0x7f91cc157e80 con 0x7f91cc154d30 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b40035b0 con 0x7f91cc154d30 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91cb7fe700 1 -- 172.21.15.154:0/1875682643 >> v1:172.21.15.191:6789/0 conn(0x7f91cc0c7f70 legacy=0x7f91cc0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91cb7fe700 1 -- 172.21.15.154:0/1875682643 >> v1:172.21.15.154:6789/0 conn(0x7f91cc154d30 legacy=0x7f91cc15f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092280560 0 0) 0x7f91cc158120 con 0x7f91cc0c7f70 2024-03-14T09:03:57.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91cb7fe700 1 -- 172.21.15.154:0/1875682643 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc158400 con 0x7f91cc0c8350 2024-03-14T09:03:57.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/1875682643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c8350 0x7f91cc1547f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009840 tx=0x7f91c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401e406703b49b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0011830 con 0x7f91cc0c8350 2024-03-14T09:03:57.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c0011990 con 0x7f91cc0c8350 2024-03-14T09:03:57.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.321+0000 7f91caffd700 1 -- 172.21.15.154:0/1875682643 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0011d20 con 0x7f91cc0c8350 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 -- 172.21.15.154:0/1875682643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c8350 msgr2=0x7f91cc1547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 --2- 172.21.15.154:0/1875682643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c8350 0x7f91cc1547f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009840 tx=0x7f91c0009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 -- 172.21.15.154:0/1875682643 shutdown_connections 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 --2- 172.21.15.154:0/1875682643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c8350 0x7f91cc1547f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 -- 172.21.15.154:0/1875682643 >> 172.21.15.154:0/1875682643 conn(0x7f91cc072080 msgr2=0x7f91cc072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 -- 172.21.15.154:0/1875682643 shutdown_connections 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.322+0000 7f91d29e6700 1 -- 172.21.15.154:0/1875682643 wait complete. 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.323+0000 7f91d29e6700 1 Processor -- start 2024-03-14T09:03:57.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.323+0000 7f91d29e6700 1 -- start start 2024-03-14T09:03:57.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c7f70 0x7f91cc0c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc0c8350 0x7f91cc0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc150f70 con 0x7f91cc154d30 2024-03-14T09:03:57.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0c4030 con 0x7f91cc0c7f70 2024-03-14T09:03:57.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1506d0 con 0x7f91cc0c8350 2024-03-14T09:03:57.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.330 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56622/0 (socket says 172.21.15.154:56622) 2024-03-14T09:03:57.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 -- 172.21.15.154:0/2718745562 learned_addr learned my addr 172.21.15.154:0/2718745562 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:57.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91cbfff700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c7f70 0x7f91cc0c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91cb7fe700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc0c8350 0x7f91cc0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 -- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc0c8350 msgr2=0x7f91cc0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc0c8350 0x7f91cc0c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 -- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c7f70 msgr2=0x7f91cc0c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c7f70 0x7f91cc0c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 -- 172.21.15.154:0/2718745562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00094a0 con 0x7f91cc154d30 2024-03-14T09:03:57.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d0f83700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f91c400ca20 tx=0x7f91c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.332 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c400aaa0 con 0x7f91cc154d30 2024-03-14T09:03:57.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c400ac00 con 0x7f91cc154d30 2024-03-14T09:03:57.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4012400 con 0x7f91cc154d30 2024-03-14T09:03:57.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc1509b0 con 0x7f91cc154d30 2024-03-14T09:03:57.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.324+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc0c2490 con 0x7f91cc154d30 2024-03-14T09:03:57.333 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.325+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc0697d0 con 0x7f91cc154d30 2024-03-14T09:03:57.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.326+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f91c401b3b0 con 0x7f91cc154d30 2024-03-14T09:03:57.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.326+0000 7f91c8ff9700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 0x7f91b806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.326+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91c40889d0 con 0x7f91cc154d30 2024-03-14T09:03:57.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.327+0000 7f91cbfff700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 0x7f91b806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.327+0000 7f91cbfff700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 0x7f91b806ccf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f91bc000f60 tx=0x7f91bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.331+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f91c4053e80 con 0x7f91cc154d30 2024-03-14T09:03:57.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.527+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91cc06cc40 con 0x7f91cc154d30 2024-03-14T09:03:57.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.529+0000 7f91c8ff9700 1 -- 172.21.15.154:0/2718745562 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16153 (secure 0 0 0) 0x7f91c4017020 con 0x7f91cc154d30 2024-03-14T09:03:57.528 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:57.528 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:57.159105+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 msgr2=0x7f91b806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 0x7f91b806ccf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f91bc000f60 tx=0x7f91bc009040 comp rx=0 tx=0).stop 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 msgr2=0x7f91cc147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f91c400ca20 tx=0x7f91c400cde0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 shutdown_connections 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f91b806a830 0x7f91b806ccf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.530 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91cc154d30 0x7f91cc147350 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91cc0c7f70 0x7f91cc0c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 --2- 172.21.15.154:0/2718745562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91cc0c8350 0x7f91cc0c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 >> 172.21.15.154:0/2718745562 conn(0x7f91cc072080 msgr2=0x7f91cc0c98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:57.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 shutdown_connections 2024-03-14T09:03:57.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.531+0000 7f91d29e6700 1 -- 172.21.15.154:0/2718745562 wait complete. 2024-03-14T09:03:57.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:03:57.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 Processor -- start 2024-03-14T09:03:57.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 -- start start 2024-03-14T09:03:57.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23500c5e50 0x7f2350151d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f235006d1f0 0x7f23500c1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235006c520 con 0x7f23500c18c0 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2350155820 con 0x7f23500c5e50 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.665+0000 7f2356547700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235006cd60 con 0x7f235006d1f0 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23500c5e50 0x7f2350151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f2354ae4700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f23500c18c0 0x7f2350157dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46764/0 (socket says 172.21.15.154:46764) 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23500c5e50 0x7f2350151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56636/0 (socket says 172.21.15.154:56636) 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f2354ae4700 1 -- 172.21.15.154:0/3719479488 learned_addr learned my addr 172.21.15.154:0/3719479488 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234f7fe700 1 --2- 172.21.15.154:0/3719479488 >> v2:172.21.15.191:3300/0 conn(0x7f235006d1f0 0x7f23500c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 -- 172.21.15.154:0/3719479488 >> v2:172.21.15.191:3300/0 conn(0x7f235006d1f0 msgr2=0x7f23500c1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 --2- 172.21.15.154:0/3719479488 >> v2:172.21.15.191:3300/0 conn(0x7f235006d1f0 0x7f23500c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 -- 172.21.15.154:0/3719479488 >> v1:172.21.15.154:6789/0 conn(0x7f23500c18c0 legacy=0x7f2350157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 -- 172.21.15.154:0/3719479488 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350152250 con 0x7f23500c5e50 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234effd700 1 -- 172.21.15.154:0/3719479488 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489307521 0 0) 0x7f235006c520 con 0x7f23500c18c0 2024-03-14T09:03:57.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234ffff700 1 --2- 172.21.15.154:0/3719479488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23500c5e50 0x7f2350151d10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f235006b750 tx=0x7f2340009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4700d5747379a09b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234effd700 1 -- 172.21.15.154:0/3719479488 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340011830 con 0x7f23500c5e50 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234effd700 1 -- 172.21.15.154:0/3719479488 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2340011990 con 0x7f23500c5e50 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.666+0000 7f234effd700 1 -- 172.21.15.154:0/3719479488 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340011d20 con 0x7f23500c5e50 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 -- 172.21.15.154:0/3719479488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23500c5e50 msgr2=0x7f2350151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 --2- 172.21.15.154:0/3719479488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23500c5e50 0x7f2350151d10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f235006b750 tx=0x7f2340009b20 comp rx=0 tx=0).stop 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 -- 172.21.15.154:0/3719479488 shutdown_connections 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 --2- 172.21.15.154:0/3719479488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23500c5e50 0x7f2350151d10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 --2- 172.21.15.154:0/3719479488 >> v2:172.21.15.191:3300/0 conn(0x7f235006d1f0 0x7f23500c1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 -- 172.21.15.154:0/3719479488 >> 172.21.15.154:0/3719479488 conn(0x7f23500a9d00 msgr2=0x7f23500aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 -- 172.21.15.154:0/3719479488 shutdown_connections 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.667+0000 7f2356547700 1 -- 172.21.15.154:0/3719479488 wait complete. 2024-03-14T09:03:57.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.668+0000 7f2356547700 1 Processor -- start 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.668+0000 7f2356547700 1 -- start start 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23500c18c0 0x7f235014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23500c5e50 0x7f23500acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500aaf50 con 0x7f235006d1f0 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500ab230 con 0x7f23500c18c0 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2356547700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23501ea540 con 0x7f23500c5e50 2024-03-14T09:03:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56642/0 (socket says 172.21.15.154:56642) 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 -- 172.21.15.154:0/1809115151 learned_addr learned my addr 172.21.15.154:0/1809115151 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234f7fe700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23500c18c0 0x7f235014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f2354ae4700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23500c5e50 0x7f23500acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 -- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23500c5e50 msgr2=0x7f23500acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23500c5e50 0x7f23500acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 -- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23500c18c0 msgr2=0x7f235014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23500c18c0 0x7f235014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 -- 172.21.15.154:0/1809115151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400094a0 con 0x7f235006d1f0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.669+0000 7f234ffff700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2340000c00 tx=0x7f2340004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.670+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400036a0 con 0x7f235006d1f0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.670+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234001f070 con 0x7f235006d1f0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.670+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340003a20 con 0x7f235006d1f0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.670+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23501ea820 con 0x7f235006d1f0 2024-03-14T09:03:57.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.670+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235014baa0 con 0x7f235006d1f0 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.671+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500697d0 con 0x7f235006d1f0 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.671+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2340003b80 con 0x7f235006d1f0 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.671+0000 7f234cff9700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 0x7f233c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.671+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2340021070 con 0x7f235006d1f0 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.672+0000 7f234f7fe700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 0x7f233c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.672+0000 7f234f7fe700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 0x7f233c06ccf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f235014ba50 tx=0x7f2344008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.676+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2340055b70 con 0x7f235006d1f0 2024-03-14T09:03:57.870 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.870+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23500ab530 con 0x7f235006d1f0 2024-03-14T09:03:57.871 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.871+0000 7f234cff9700 1 -- 172.21.15.154:0/1809115151 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16153 (secure 0 0 0) 0x7f2340015090 con 0x7f235006d1f0 2024-03-14T09:03:57.871 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:03:57.871 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:57.159105+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:03:57.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 msgr2=0x7f233c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 0x7f233c06ccf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f235014ba50 tx=0x7f2344008040 comp rx=0 tx=0).stop 2024-03-14T09:03:57.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 msgr2=0x7f235014ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:57.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2340000c00 tx=0x7f2340004750 comp rx=0 tx=0).stop 2024-03-14T09:03:57.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 shutdown_connections 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f23500c5e50 0x7f23500acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f233c06a830 0x7f233c06ccf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f235006d1f0 0x7f235014ea30 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 --2- 172.21.15.154:0/1809115151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23500c18c0 0x7f235014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 >> 172.21.15.154:0/1809115151 conn(0x7f23500a9d00 msgr2=0x7f23501543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 shutdown_connections 2024-03-14T09:03:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:57.873+0000 7f2356547700 1 -- 172.21.15.154:0/1809115151 wait complete. 2024-03-14T09:03:57.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-14T09:03:58.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.010+0000 7fc505a00700 1 Processor -- start 2024-03-14T09:03:58.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 -- start start 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000c6a20 con 0x7fc500155350 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5000707b0 con 0x7fc5000c5150 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.011+0000 7fc505a00700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000c6460 con 0x7fc5000c87e0 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fe7fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4ff7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc500155350 0x7fc5000c3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:35532/0 (socket says 172.21.15.154:35532) 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fe7fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:54288/0 (socket says 172.21.15.154:54288) 2024-03-14T09:03:58.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:52982/0 (socket says 172.21.15.154:52982) 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4ff7fe700 1 -- 172.21.15.154:0/4008585685 learned_addr learned my addr 172.21.15.154:0/4008585685 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 -- 172.21.15.154:0/4008585685 >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 msgr2=0x7fc500154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 --2- 172.21.15.154:0/4008585685 >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 -- 172.21.15.154:0/4008585685 >> v1:172.21.15.191:6789/0 conn(0x7fc500155350 legacy=0x7fc5000c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 -- 172.21.15.154:0/4008585685 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000c6740 con 0x7fc5000c87e0 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fdffb700 1 -- 172.21.15.154:0/4008585685 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988842470 0 0) 0x7fc4f8001490 con 0x7fc500155350 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fe7fc700 1 --2- 172.21.15.154:0/4008585685 >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4feffd700 1 --2- 172.21.15.154:0/4008585685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d513d1cfbc1b812e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:58.014 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fdffb700 1 -- 172.21.15.154:0/4008585685 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8011830 con 0x7fc5000c87e0 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fdffb700 1 -- 172.21.15.154:0/4008585685 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8011990 con 0x7fc5000c87e0 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.012+0000 7fc4fdffb700 1 -- 172.21.15.154:0/4008585685 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8011d20 con 0x7fc5000c87e0 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 -- 172.21.15.154:0/4008585685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 msgr2=0x7fc5000c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 --2- 172.21.15.154:0/4008585685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).stop 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 -- 172.21.15.154:0/4008585685 shutdown_connections 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 --2- 172.21.15.154:0/4008585685 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c87e0 0x7fc5000c4c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 --2- 172.21.15.154:0/4008585685 >> v2:172.21.15.191:3300/0 conn(0x7fc5000c5150 0x7fc500154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 -- 172.21.15.154:0/4008585685 >> 172.21.15.154:0/4008585685 conn(0x7fc50006e4a0 msgr2=0x7fc50006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 -- 172.21.15.154:0/4008585685 shutdown_connections 2024-03-14T09:03:58.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.013+0000 7fc505a00700 1 -- 172.21.15.154:0/4008585685 wait complete. 2024-03-14T09:03:58.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.014+0000 7fc505a00700 1 Processor -- start 2024-03-14T09:03:58.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.014+0000 7fc505a00700 1 -- start start 2024-03-14T09:03:58.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c5150 0x7fc50006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.018 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5000c87e0 0x7fc5001e0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006b880 con 0x7fc500155350 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001ea8d0 con 0x7fc5000c87e0 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc505a00700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001eabb0 con 0x7fc5000c5150 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc4ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc4ff7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56644/0 (socket says 172.21.15.154:56644) 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc4ff7fe700 1 -- 172.21.15.154:0/3279617797 learned_addr learned my addr 172.21.15.154:0/3279617797 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc4feffd700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c5150 0x7fc50006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.015+0000 7fc4fe7fc700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5000c87e0 0x7fc5001e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 -- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c5150 msgr2=0x7fc50006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c5150 0x7fc50006d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 -- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5000c87e0 msgr2=0x7fc5001e0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5000c87e0 0x7fc5001e0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:58.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 -- 172.21.15.154:0/3279617797 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f00094a0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4ff7fe700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800cba0 tx=0x7fc4f800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800aaa0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800ac20 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80094a0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80070d0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.016+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50006bdb0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.017+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5000697d0 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.017+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc4f8007230 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.018+0000 7fc4f7fff700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 0x7fc4e006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.018+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4f8088820 con 0x7fc500155350 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.018+0000 7fc4feffd700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 0x7fc4e006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:58.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.018+0000 7fc4feffd700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 0x7fc4e006ccf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80090b0 tx=0x7fc4e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:58.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.022+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc4f8053cd0 con 0x7fc500155350 2024-03-14T09:03:58.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:58.265+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5000c6a20 con 0x7fc500155350 2024-03-14T09:03:59.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.171+0000 7fc4f7fff700 1 -- 172.21.15.154:0/3279617797 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc5000c6a20 con 0x7fc500155350 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.173+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 msgr2=0x7fc4e006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 0x7fc4e006ccf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80090b0 tx=0x7fc4e8008030 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 msgr2=0x7fc50014d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800cba0 tx=0x7fc4f800ceb0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 shutdown_connections 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc4e006a830 0x7fc4e006ccf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc500155350 0x7fc50014d420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5000c5150 0x7fc50006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 --2- 172.21.15.154:0/3279617797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5000c87e0 0x7fc5001e0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 >> 172.21.15.154:0/3279617797 conn(0x7fc50006e4a0 msgr2=0x7fc50006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 shutdown_connections 2024-03-14T09:03:59.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.174+0000 7fc505a00700 1 -- 172.21.15.154:0/3279617797 wait complete. 2024-03-14T09:03:59.188 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:03:59.189 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-14T09:03:59.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.318+0000 7f846048a700 1 Processor -- start 2024-03-14T09:03:59.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.318+0000 7f846048a700 1 -- start start 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f846048a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f845813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f846048a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845813ade0 con 0x7f84580a8540 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f846048a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845806fb30 con 0x7f84580c3250 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f846048a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845806ec30 con 0x7f84580c1e90 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845e226700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f84580a8540 0x7f8458154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:51970/0 (socket says 172.21.15.154:51970) 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845e226700 1 -- 172.21.15.154:0/1406621390 learned_addr learned my addr 172.21.15.154:0/1406621390 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845ea27700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f845813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874750534 0 0) 0x7f8454001110 con 0x7f84580a8540 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84440035b0 con 0x7f84580a8540 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962432576 0 0) 0x7f845806fb30 con 0x7f84580c3250 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454001110 con 0x7f84580c3250 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599392149 0 0) 0x7f84440035b0 con 0x7f84580a8540 2024-03-14T09:03:59.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 msgr2=0x7f845813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f845813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 >> v1:172.21.15.191:6789/0 conn(0x7f84580c3250 legacy=0x7f84580c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845806ef10 con 0x7f84580a8540 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8454001780 con 0x7f84580a8540 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 >> v1:172.21.15.154:6789/0 conn(0x7f84580a8540 legacy=0x7f8458154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.319+0000 7f845d224700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8444003bc0 0x7f8444003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8444004850 0x7f8444004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f8444005310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845806ec30 con 0x7f8444003bc0 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844400bc40 con 0x7f8444004850 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844400be00 con 0x7f84580c1e90 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8444003bc0 0x7f8444003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845da25700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8444004850 0x7f8444004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845ea27700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f8444005310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 -- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 msgr2=0x7f8444005310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f8444005310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 -- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8444004850 msgr2=0x7f8444004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8444004850 0x7f8444004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 -- 172.21.15.154:0/1406621390 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440035b0 con 0x7f8444003bc0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845ea27700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f84580c1e90 0x7f8444005310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845e226700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8444003bc0 0x7f8444003fc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8454007fa0 tx=0x7f8444005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711cf4fa83ef82a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400cc20 con 0x7f8444003bc0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845400cd80 con 0x7f8444003bc0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.320+0000 7f845d224700 1 -- 172.21.15.154:0/1406621390 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454005ba0 con 0x7f8444003bc0 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 -- 172.21.15.154:0/1406621390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8444003bc0 msgr2=0x7f8444003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 --2- 172.21.15.154:0/1406621390 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8444003bc0 0x7f8444003fc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8454007fa0 tx=0x7f8444005880 comp rx=0 tx=0).stop 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f845ea27700 1 -- 172.21.15.154:0/1406621390 reap_dead start 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 -- 172.21.15.154:0/1406621390 shutdown_connections 2024-03-14T09:03:59.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 -- 172.21.15.154:0/1406621390 >> 172.21.15.154:0/1406621390 conn(0x7f84580c3880 msgr2=0x7f84580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:03:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 -- 172.21.15.154:0/1406621390 shutdown_connections 2024-03-14T09:03:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.321+0000 7f846048a700 1 -- 172.21.15.154:0/1406621390 wait complete. 2024-03-14T09:03:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.322+0000 7f846048a700 1 Processor -- start 2024-03-14T09:03:59.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.322+0000 7f846048a700 1 -- start start 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8444003bc0 0x7f84581331c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84580c02b0 0x7f845812fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845812e690 con 0x7f8458152930 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580c10c0 con 0x7f84580c02b0 2024-03-14T09:03:59.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84581ee090 con 0x7f8444003bc0 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39202/0 (socket says 172.21.15.154:39202) 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 -- 172.21.15.154:0/136888254 learned_addr learned my addr 172.21.15.154:0/136888254 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845e226700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8444003bc0 0x7f84581331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845da25700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84580c02b0 0x7f845812fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 -- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8444003bc0 msgr2=0x7f84581331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8444003bc0 0x7f84581331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 -- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84580c02b0 msgr2=0x7f845812fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84580c02b0 0x7f845812fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 -- 172.21.15.154:0/136888254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84580c0b90 con 0x7f8458152930 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f845ea27700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ed0 tx=0x7f84500104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000e050 con 0x7f8458152930 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845000f040 con 0x7f8458152930 2024-03-14T09:03:59.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84580c0de0 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450010d70 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.323+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84581ee560 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.324+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84580697d0 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.325+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8450015440 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.325+0000 7f844effd700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 0x7f843c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.325+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8450088e30 con 0x7f8458152930 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.326+0000 7f845e226700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 0x7f843c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.326+0000 7f845e226700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 0x7f843c06ccf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:03:59.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.330+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f84500542e0 con 0x7f8458152930 2024-03-14T09:03:59.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:03:59.571+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84580c4dd0 con 0x7f8458152930 2024-03-14T09:04:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.179+0000 7f844effd700 1 -- 172.21.15.154:0/136888254 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8450057900 con 0x7f8458152930 2024-03-14T09:04:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 msgr2=0x7f843c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 0x7f843c06ccf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454007000 comp rx=0 tx=0).stop 2024-03-14T09:04:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 msgr2=0x7f8458130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8450009ed0 tx=0x7f84500104a0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 shutdown_connections 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f843c06a830 0x7f843c06ccf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8458152930 0x7f8458130330 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8444003bc0 0x7f84581331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 --2- 172.21.15.154:0/136888254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84580c02b0 0x7f845812fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 >> 172.21.15.154:0/136888254 conn(0x7f84580c3880 msgr2=0x7f84580c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 shutdown_connections 2024-03-14T09:04:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.181+0000 7f846048a700 1 -- 172.21.15.154:0/136888254 wait complete. 2024-03-14T09:04:00.195 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-14T09:04:00.195 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:00.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.318+0000 7f86ffadf700 1 Processor -- start 2024-03-14T09:04:00.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.318+0000 7f86ffadf700 1 -- start start 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86f80a9650 0x7f86f80c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f86f81564e0 0x7f86f806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8152690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f8156050 con 0x7f86f80a9650 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f80a86f0 con 0x7f86f81564e0 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86ffadf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80c34b0 con 0x7f86f8156960 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86fd87b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86f80a9650 0x7f86f80c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86fd87b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86f80a9650 0x7f86f80c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39208/0 (socket says 172.21.15.154:39208) 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86fd87b700 1 -- 172.21.15.154:0/757422663 learned_addr learned my addr 172.21.15.154:0/757422663 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86fe07c700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.319+0000 7f86fd07a700 1 --2- 172.21.15.154:0/757422663 >> v2:172.21.15.191:3300/0 conn(0x7f86f81564e0 0x7f86f806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 -- 172.21.15.154:0/757422663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 msgr2=0x7f86f8152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 -- 172.21.15.154:0/757422663 >> v2:172.21.15.191:3300/0 conn(0x7f86f81564e0 msgr2=0x7f86f806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 --2- 172.21.15.154:0/757422663 >> v2:172.21.15.191:3300/0 conn(0x7f86f81564e0 0x7f86f806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 -- 172.21.15.154:0/757422663 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80a8990 con 0x7f86f80a9650 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fd87b700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f80a9650 0x7f86f80c1780 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f86f80a9fb0 tx=0x7f86e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9d24e0594de733 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fc879700 1 -- 172.21.15.154:0/757422663 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8011830 con 0x7f86f80a9650 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fc879700 1 -- 172.21.15.154:0/757422663 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8011990 con 0x7f86f80a9650 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86fc879700 1 -- 172.21.15.154:0/757422663 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8011d20 con 0x7f86f80a9650 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86ffadf700 1 -- 172.21.15.154:0/757422663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f80a9650 msgr2=0x7f86f80c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86ffadf700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f80a9650 0x7f86f80c1780 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f86f80a9fb0 tx=0x7f86e8009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86ffadf700 1 -- 172.21.15.154:0/757422663 shutdown_connections 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86ffadf700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.320+0000 7f86ffadf700 1 --2- 172.21.15.154:0/757422663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f80a9650 0x7f86f80c1780 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.321+0000 7f86ffadf700 1 --2- 172.21.15.154:0/757422663 >> v2:172.21.15.191:3300/0 conn(0x7f86f81564e0 0x7f86f806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.321+0000 7f86ffadf700 1 -- 172.21.15.154:0/757422663 >> 172.21.15.154:0/757422663 conn(0x7f86f80c3880 msgr2=0x7f86f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.321+0000 7f86ffadf700 1 -- 172.21.15.154:0/757422663 shutdown_connections 2024-03-14T09:04:00.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.321+0000 7f86ffadf700 1 -- 172.21.15.154:0/757422663 wait complete. 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.322+0000 7f86ffadf700 1 Processor -- start 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.322+0000 7f86ffadf700 1 -- start start 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f80a9650 0x7f86f81e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8142ca0 con 0x7f86f81564e0 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81ec490 con 0x7f86f80a9650 2024-03-14T09:04:00.326 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86ffadf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81ec770 con 0x7f86f8156960 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39214/0 (socket says 172.21.15.154:39214) 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 -- 172.21.15.154:0/3823171890 learned_addr learned my addr 172.21.15.154:0/3823171890 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd87b700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f80a9650 0x7f86f81e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fe07c700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 -- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 msgr2=0x7f86f8147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 -- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f80a9650 msgr2=0x7f86f81e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f80a9650 0x7f86f81e83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.323+0000 7f86fd07a700 1 -- 172.21.15.154:0/3823171890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8009550 con 0x7f86f81564e0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86fd07a700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e870 tx=0x7f86f400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4009590 con 0x7f86f81564e0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4013680 con 0x7f86f81564e0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f80c66b0 con 0x7f86f81564e0 2024-03-14T09:04:00.327 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40139f0 con 0x7f86f81564e0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.324+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81484e0 con 0x7f86f81564e0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.325+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f80697d0 con 0x7f86f81564e0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.325+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f86f4013b50 con 0x7f86f81564e0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.325+0000 7f86ee7fc700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 0x7f86dc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.326+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86f4088b30 con 0x7f86f81564e0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.326+0000 7f86fd87b700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 0x7f86dc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.328 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.326+0000 7f86fd87b700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 0x7f86dc06ccf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f86e8014040 tx=0x7f86e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.329 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.330+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f86f4053fe0 con 0x7f86f81564e0 2024-03-14T09:04:00.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.568+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86f806b470 con 0x7f86f81564e0 2024-03-14T09:04:00.568 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.569+0000 7f86ee7fc700 1 -- 172.21.15.154:0/3823171890 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f86f4017090 con 0x7f86f81564e0 2024-03-14T09:04:00.569 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:00.569 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6438,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-14T09:04:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 msgr2=0x7f86dc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 0x7f86dc06ccf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f86e8014040 tx=0x7f86e8008030 comp rx=0 tx=0).stop 2024-03-14T09:04:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 msgr2=0x7f86f8142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e870 tx=0x7f86f400eb80 comp rx=0 tx=0).stop 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 shutdown_connections 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86f8156960 0x7f86f8147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86f80a9650 0x7f86f81e83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f86dc06a830 0x7f86dc06ccf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 --2- 172.21.15.154:0/3823171890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86f81564e0 0x7f86f8142760 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 >> 172.21.15.154:0/3823171890 conn(0x7f86f80c3880 msgr2=0x7f86f80ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 shutdown_connections 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.571+0000 7f86ffadf700 1 -- 172.21.15.154:0/3823171890 wait complete. 2024-03-14T09:04:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 45 2024-03-14T09:04:00.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.705+0000 7fe075ea2700 1 Processor -- start 2024-03-14T09:04:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.705+0000 7fe075ea2700 1 -- start start 2024-03-14T09:04:00.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe07014eaf0 0x7fe07006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07006acb0 0x7fe070155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07006b310 con 0x7fe07006acb0 2024-03-14T09:04:00.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700ac8e0 con 0x7fe07014eaf0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe075ea2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07006de30 con 0x7fe070156330 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07006acb0 0x7fe070155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe07006acb0 0x7fe070155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39218/0 (socket says 172.21.15.154:39218) 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 -- 172.21.15.154:0/1488614812 learned_addr learned my addr 172.21.15.154:0/1488614812 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 -- 172.21.15.154:0/1488614812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 msgr2=0x7fe070157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 --2- 172.21.15.154:0/1488614812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 -- 172.21.15.154:0/1488614812 >> v2:172.21.15.191:3300/0 conn(0x7fe07014eaf0 msgr2=0x7fe07006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 --2- 172.21.15.154:0/1488614812 >> v2:172.21.15.191:3300/0 conn(0x7fe07014eaf0 0x7fe07006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 -- 172.21.15.154:0/1488614812 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700acb30 con 0x7fe07006acb0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06effd700 1 --2- 172.21.15.154:0/1488614812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07006acb0 0x7fe070155df0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe070070da0 tx=0x7fe060009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7715b01d056581c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.706+0000 7fe06e7fc700 1 -- 172.21.15.154:0/1488614812 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06000b870 con 0x7fe07006acb0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe06e7fc700 1 -- 172.21.15.154:0/1488614812 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06000b9d0 con 0x7fe07006acb0 2024-03-14T09:04:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe06e7fc700 1 -- 172.21.15.154:0/1488614812 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06000bd60 con 0x7fe07006acb0 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 -- 172.21.15.154:0/1488614812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07006acb0 msgr2=0x7fe070155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 --2- 172.21.15.154:0/1488614812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07006acb0 0x7fe070155df0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe070070da0 tx=0x7fe060009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 -- 172.21.15.154:0/1488614812 shutdown_connections 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 --2- 172.21.15.154:0/1488614812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 --2- 172.21.15.154:0/1488614812 >> v2:172.21.15.191:3300/0 conn(0x7fe07014eaf0 0x7fe07006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 --2- 172.21.15.154:0/1488614812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07006acb0 0x7fe070155df0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 -- 172.21.15.154:0/1488614812 >> 172.21.15.154:0/1488614812 conn(0x7fe0700c3880 msgr2=0x7fe0700c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 -- 172.21.15.154:0/1488614812 shutdown_connections 2024-03-14T09:04:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.707+0000 7fe075ea2700 1 -- 172.21.15.154:0/1488614812 wait complete. 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.708+0000 7fe075ea2700 1 Processor -- start 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.708+0000 7fe075ea2700 1 -- start start 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 0x7fe0701ee240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701ee780 con 0x7fe07014eaf0 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0701493c0 con 0x7fe07006acb0 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe075ea2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe070148d90 con 0x7fe070156330 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.712 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39220/0 (socket says 172.21.15.154:39220) 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 -- 172.21.15.154:0/765194735 learned_addr learned my addr 172.21.15.154:0/765194735 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06f7fe700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 0x7fe0701ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 -- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 msgr2=0x7fe070149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 -- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 msgr2=0x7fe0701ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 0x7fe0701ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06effd700 1 -- 172.21.15.154:0/765194735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060009550 con 0x7fe07014eaf0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.709+0000 7fe06f7fe700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 0x7fe0701ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe06effd700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe060014040 tx=0x7fe060004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060004520 con 0x7fe07014eaf0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06001e070 con 0x7fe07014eaf0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600038c0 con 0x7fe07014eaf0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07013eac0 con 0x7fe07014eaf0 2024-03-14T09:04:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.710+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe070149050 con 0x7fe07014eaf0 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.711+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070149570 con 0x7fe07014eaf0 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.711+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe060003a20 con 0x7fe07014eaf0 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.712+0000 7fe067fff700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 0x7fe05006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.712+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe060020070 con 0x7fe07014eaf0 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.712+0000 7fe06f7fe700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 0x7fe05006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:00.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.712+0000 7fe06f7fe700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 0x7fe05006ccf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe058007900 tx=0x7fe058008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:00.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.716+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe070149570 con 0x7fe07014eaf0 2024-03-14T09:04:00.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.912+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe070149570 con 0x7fe07014eaf0 2024-03-14T09:04:00.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.914+0000 7fe067fff700 1 -- 172.21.15.154:0/765194735 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16140 (secure 0 0 0) 0x7fe070149570 con 0x7fe07014eaf0 2024-03-14T09:04:00.914 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:00.914 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":70,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:58.161285+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 msgr2=0x7fe05006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 0x7fe05006ccf0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe058007900 tx=0x7fe058008040 comp rx=0 tx=0).stop 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 msgr2=0x7fe070153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe060014040 tx=0x7fe060004750 comp rx=0 tx=0).stop 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 shutdown_connections 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe070156330 0x7fe070149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe07006acb0 0x7fe0701ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe05006a830 0x7fe05006ccf0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 --2- 172.21.15.154:0/765194735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe07014eaf0 0x7fe070153970 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 >> 172.21.15.154:0/765194735 conn(0x7fe0700c3880 msgr2=0x7fe0701ddf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 shutdown_connections 2024-03-14T09:04:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:00.916+0000 7fe075ea2700 1 -- 172.21.15.154:0/765194735 wait complete. 2024-03-14T09:04:00.928 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:04:00.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:01.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.050+0000 7f8eecf9d700 1 Processor -- start 2024-03-14T09:04:01.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 -- start start 2024-03-14T09:04:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ee80c4380 0x7f8ee814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ee80c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee80acd10 con 0x7f8ee80702f0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee8150110 con 0x7f8ee80c4380 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.051+0000 7f8eecf9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80ac7b0 con 0x7f8ee80c5d60 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee659c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8ee80702f0 0x7f8ee80c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:51972/0 (socket says 172.21.15.154:51972) 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee659c700 1 -- 172.21.15.154:0/3932526240 learned_addr learned my addr 172.21.15.154:0/3932526240 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/3932526240 >> v2:172.21.15.191:3300/0 conn(0x7f8ee80c4380 0x7f8ee814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee6d9d700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ee80c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590835251 0 0) 0x7f8ee80acd10 con 0x7f8ee80702f0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed00035b0 con 0x7f8ee80702f0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904337491 0 0) 0x7f8ed00035b0 con 0x7f8ee80702f0 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 msgr2=0x7f8ee80c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ee80c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 >> v2:172.21.15.191:3300/0 conn(0x7f8ee80c4380 msgr2=0x7f8ee814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 --2- 172.21.15.154:0/3932526240 >> v2:172.21.15.191:3300/0 conn(0x7f8ee80c4380 0x7f8ee814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80ac9c0 con 0x7f8ee80702f0 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8edc001670 con 0x7f8ee80702f0 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/3932526240 >> v2:172.21.15.191:3300/0 conn(0x7f8ee80c4380 0x7f8ee814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 >> v1:172.21.15.154:6789/0 conn(0x7f8ee80702f0 legacy=0x7f8ee80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ed0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ed0004850 0x7f8ed0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ed0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8150110 con 0x7f8ed0003a00 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80ac7b0 con 0x7f8ed0004850 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed000ba10 con 0x7f8ee80c5d60 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee659c700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ed0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ed0004850 0x7f8ed0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.052+0000 7f8ee6d9d700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ed0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 -- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 msgr2=0x7f8ed0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ed0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 -- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ed0004850 msgr2=0x7f8ed0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ed0004850 0x7f8ed0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 -- 172.21.15.154:0/3932526240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00035b0 con 0x7f8ed0003a00 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee6d9d700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee80c5d60 0x7f8ed0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee659c700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ed0003e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c370 tx=0x7f8ed0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1031a26a147582e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc007c60 con 0x7f8ed0003a00 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc005960 con 0x7f8ed0003a00 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee559a700 1 -- 172.21.15.154:0/3932526240 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc005d10 con 0x7f8ed0003a00 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 -- 172.21.15.154:0/3932526240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 msgr2=0x7f8ed0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/3932526240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ed0003e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c370 tx=0x7f8ed0005880 comp rx=0 tx=0).stop 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8ee6d9d700 1 -- 172.21.15.154:0/3932526240 reap_dead start 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 -- 172.21.15.154:0/3932526240 shutdown_connections 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 -- 172.21.15.154:0/3932526240 >> 172.21.15.154:0/3932526240 conn(0x7f8ee80a9bf0 msgr2=0x7f8ee80a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 -- 172.21.15.154:0/3932526240 shutdown_connections 2024-03-14T09:04:01.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.053+0000 7f8eecf9d700 1 -- 172.21.15.154:0/3932526240 wait complete. 2024-03-14T09:04:01.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 Processor -- start 2024-03-14T09:04:01.058 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 -- start start 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee81e7940 0x7f8ee8148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee814b1e0 0x7f8ee8152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee81477f0 con 0x7f8ed0003a00 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8147ad0 con 0x7f8ee81e7940 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.055+0000 7f8eecf9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8147db0 con 0x7f8ee814b1e0 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39236/0 (socket says 172.21.15.154:39236) 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 -- 172.21.15.154:0/700907122 learned_addr learned my addr 172.21.15.154:0/700907122 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 -- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee814b1e0 msgr2=0x7f8ee8152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee814b1e0 0x7f8ee8152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 -- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee81e7940 msgr2=0x7f8ee8148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee81e7940 0x7f8ee8148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee81e7940 0x7f8ee8148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 -- 172.21.15.154:0/700907122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8148090 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ee659c700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c960 tx=0x7f8edc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc004870 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc0049d0 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee8148370 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc004dc0 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.056+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee8153360 con 0x7f8ed0003a00 2024-03-14T09:04:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.057+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee8153550 con 0x7f8ed0003a00 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.058+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8edc0070d0 con 0x7f8ed0003a00 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.058+0000 7f8ed6ffd700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 0x7f8ec806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.058+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8edc08a4b0 con 0x7f8ed0003a00 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.058+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 0x7f8ec806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.059+0000 7f8ee5d9b700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 0x7f8ec806ccf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8ee81ebfd0 tx=0x7f8ed8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.062+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8edc055a00 con 0x7f8ed0003a00 2024-03-14T09:04:01.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.255+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ee81e7d20 con 0x7f8ec806a830 2024-03-14T09:04:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.263+0000 7f8ed6ffd700 1 -- 172.21.15.154:0/700907122 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289476 (secure 0 0 0) 0x7f8ee81e7d20 con 0x7f8ec806a830 2024-03-14T09:04:01.270 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 msgr2=0x7f8ec806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 0x7f8ec806ccf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8ee81ebfd0 tx=0x7f8ed8008040 comp rx=0 tx=0).stop 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 msgr2=0x7f8ee8147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c960 tx=0x7f8edc000fe0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 shutdown_connections 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee814b1e0 0x7f8ee8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8ec806a830 0x7f8ec806ccf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ed0003a00 0x7f8ee8147210 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 --2- 172.21.15.154:0/700907122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee81e7940 0x7f8ee8148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 >> 172.21.15.154:0/700907122 conn(0x7f8ee80a9bf0 msgr2=0x7f8ee80c2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.266+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 shutdown_connections 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.267+0000 7f8eecf9d700 1 -- 172.21.15.154:0/700907122 wait complete. 2024-03-14T09:04:01.276 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:01.278 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-03-14T09:03:59.555971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223996,"kb_used_data":7792,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750724,"statfs":{"total":773094113280,"available":772864741376,"internally_reserved":0,"allocated":7979008,"data_stored":4116771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000824"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188980+0000","last_change":"2024-03-14T09:03:57.183867+0000","last_active":"2024-03-14T09:03:57.188980+0000","last_peered":"2024-03-14T09:03:57.188980+0000","last_clean":"2024-03-14T09:03:57.188980+0000","last_became_active":"2024-03-14T09:03:57.183371+0000","last_became_peered":"2024-03-14T09:03:57.183371+0000","last_unstale":"2024-03-14T09:03:57.188980+0000","last_undegraded":"2024-03-14T09:03:57.188980+0000","last_fullsized":"2024-03-14T09:03:57.188980+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192502+0000","last_change":"2024-03-14T09:03:57.187973+0000","last_active":"2024-03-14T09:03:57.192502+0000","last_peered":"2024-03-14T09:03:57.192502+0000","last_clean":"2024-03-14T09:03:57.192502+0000","last_became_active":"2024-03-14T09:03:57.187737+0000","last_became_peered":"2024-03-14T09:03:57.187737+0000","last_unstale":"2024-03-14T09:03:57.192502+0000","last_undegraded":"2024-03-14T09:03:57.192502+0000","last_fullsized":"2024-03-14T09:03:57.192502+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:39.248247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184397+0000","last_change":"2024-03-14T09:03:57.180121+0000","last_active":"2024-03-14T09:03:57.184397+0000","last_peered":"2024-03-14T09:03:57.184397+0000","last_clean":"2024-03-14T09:03:57.184397+0000","last_became_active":"2024-03-14T09:03:57.179465+0000","last_became_peered":"2024-03-14T09:03:57.179465+0000","last_unstale":"2024-03-14T09:03:57.184397+0000","last_undegraded":"2024-03-14T09:03:57.184397+0000","last_fullsized":"2024-03-14T09:03:57.184397+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194063+0000","last_change":"2024-03-14T09:03:57.191808+0000","last_active":"2024-03-14T09:03:57.194063+0000","last_peered":"2024-03-14T09:03:57.194063+0000","last_clean":"2024-03-14T09:03:57.194063+0000","last_became_active":"2024-03-14T09:03:57.187786+0000","last_became_peered":"2024-03-14T09:03:57.187786+0000","last_unstale":"2024-03-14T09:03:57.194063+0000","last_undegraded":"2024-03-14T09:03:57.194063+0000","last_fullsized":"2024-03-14T09:03:57.194063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:23.462424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190948+0000","last_change":"2024-03-14T09:03:57.186814+0000","last_active":"2024-03-14T09:03:57.190948+0000","last_peered":"2024-03-14T09:03:57.190948+0000","last_clean":"2024-03-14T09:03:57.190948+0000","last_became_active":"2024-03-14T09:03:57.186480+0000","last_became_peered":"2024-03-14T09:03:57.186480+0000","last_unstale":"2024-03-14T09:03:57.190948+0000","last_undegraded":"2024-03-14T09:03:57.190948+0000","last_fullsized":"2024-03-14T09:03:57.190948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187781+0000","last_change":"2024-03-14T09:03:57.179120+0000","last_active":"2024-03-14T09:03:57.187781+0000","last_peered":"2024-03-14T09:03:57.187781+0000","last_clean":"2024-03-14T09:03:57.187781+0000","last_became_active":"2024-03-14T09:03:57.178824+0000","last_became_peered":"2024-03-14T09:03:57.178824+0000","last_unstale":"2024-03-14T09:03:57.187781+0000","last_undegraded":"2024-03-14T09:03:57.187781+0000","last_fullsized":"2024-03-14T09:03:57.187781+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186258+0000","last_change":"2024-03-14T09:03:57.176658+0000","last_active":"2024-03-14T09:03:57.186258+0000","last_peered":"2024-03-14T09:03:57.186258+0000","last_clean":"2024-03-14T09:03:57.186258+0000","last_became_active":"2024-03-14T09:03:57.176352+0000","last_became_peered":"2024-03-14T09:03:57.176352+0000","last_unstale":"2024-03-14T09:03:57.186258+0000","last_undegraded":"2024-03-14T09:03:57.186258+0000","last_fullsized":"2024-03-14T09:03:57.186258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:51:09.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190195+0000","last_change":"2024-03-14T09:03:57.186734+0000","last_active":"2024-03-14T09:03:57.190195+0000","last_peered":"2024-03-14T09:03:57.190195+0000","last_clean":"2024-03-14T09:03:57.190195+0000","last_became_active":"2024-03-14T09:03:57.186455+0000","last_became_peered":"2024-03-14T09:03:57.186455+0000","last_unstale":"2024-03-14T09:03:57.190195+0000","last_undegraded":"2024-03-14T09:03:57.190195+0000","last_fullsized":"2024-03-14T09:03:57.190195+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190164+0000","last_change":"2024-03-14T09:03:57.186757+0000","last_active":"2024-03-14T09:03:57.190164+0000","last_peered":"2024-03-14T09:03:57.190164+0000","last_clean":"2024-03-14T09:03:57.190164+0000","last_became_active":"2024-03-14T09:03:57.186486+0000","last_became_peered":"2024-03-14T09:03:57.186486+0000","last_unstale":"2024-03-14T09:03:57.190164+0000","last_undegraded":"2024-03-14T09:03:57.190164+0000","last_fullsized":"2024-03-14T09:03:57.190164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184362+0000","last_change":"2024-03-14T09:03:57.180488+0000","last_active":"2024-03-14T09:03:57.184362+0000","last_peered":"2024-03-14T09:03:57.184362+0000","last_clean":"2024-03-14T09:03:57.184362+0000","last_became_active":"2024-03-14T09:03:57.180202+0000","last_became_peered":"2024-03-14T09:03:57.180202+0000","last_unstale":"2024-03-14T09:03:57.184362+0000","last_undegraded":"2024-03-14T09:03:57.184362+0000","last_fullsized":"2024-03-14T09:03:57.184362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192354+0000","last_change":"2024-03-14T09:03:57.185725+0000","last_active":"2024-03-14T09:03:57.192354+0000","last_peered":"2024-03-14T09:03:57.192354+0000","last_clean":"2024-03-14T09:03:57.192354+0000","last_became_active":"2024-03-14T09:03:57.185457+0000","last_became_peered":"2024-03-14T09:03:57.185457+0000","last_unstale":"2024-03-14T09:03:57.192354+0000","last_undegraded":"2024-03-14T09:03:57.192354+0000","last_fullsized":"2024-03-14T09:03:57.192354+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:26.439102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188935+0000","last_change":"2024-03-14T09:03:57.182863+0000","last_active":"2024-03-14T09:03:57.188935+0000","last_peered":"2024-03-14T09:03:57.188935+0000","last_clean":"2024-03-14T09:03:57.188935+0000","last_became_active":"2024-03-14T09:03:57.181034+0000","last_became_peered":"2024-03-14T09:03:57.181034+0000","last_unstale":"2024-03-14T09:03:57.188935+0000","last_undegraded":"2024-03-14T09:03:57.188935+0000","last_fullsized":"2024-03-14T09:03:57.188935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182507+0000","last_change":"2024-03-14T09:03:57.177255+0000","last_active":"2024-03-14T09:03:57.182507+0000","last_peered":"2024-03-14T09:03:57.182507+0000","last_clean":"2024-03-14T09:03:57.182507+0000","last_became_active":"2024-03-14T09:03:57.176972+0000","last_became_peered":"2024-03-14T09:03:57.176972+0000","last_unstale":"2024-03-14T09:03:57.182507+0000","last_undegraded":"2024-03-14T09:03:57.182507+0000","last_fullsized":"2024-03-14T09:03:57.182507+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:41.881683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194078+0000","last_change":"2024-03-14T09:03:57.191926+0000","last_active":"2024-03-14T09:03:57.194078+0000","last_peered":"2024-03-14T09:03:57.194078+0000","last_clean":"2024-03-14T09:03:57.194078+0000","last_became_active":"2024-03-14T09:03:57.191416+0000","last_became_peered":"2024-03-14T09:03:57.191416+0000","last_unstale":"2024-03-14T09:03:57.194078+0000","last_undegraded":"2024-03-14T09:03:57.194078+0000","last_fullsized":"2024-03-14T09:03:57.194078+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:25.082584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180617+0000","last_change":"2024-03-14T09:03:57.175406+0000","last_active":"2024-03-14T09:03:57.180617+0000","last_peered":"2024-03-14T09:03:57.180617+0000","last_clean":"2024-03-14T09:03:57.180617+0000","last_became_active":"2024-03-14T09:03:57.175155+0000","last_became_peered":"2024-03-14T09:03:57.175155+0000","last_unstale":"2024-03-14T09:03:57.180617+0000","last_undegraded":"2024-03-14T09:03:57.180617+0000","last_fullsized":"2024-03-14T09:03:57.180617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:19.270920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190025+0000","last_change":"2024-03-14T09:03:57.181942+0000","last_active":"2024-03-14T09:03:57.190025+0000","last_peered":"2024-03-14T09:03:57.190025+0000","last_clean":"2024-03-14T09:03:57.190025+0000","last_became_active":"2024-03-14T09:03:57.181669+0000","last_became_peered":"2024-03-14T09:03:57.181669+0000","last_unstale":"2024-03-14T09:03:57.190025+0000","last_undegraded":"2024-03-14T09:03:57.190025+0000","last_fullsized":"2024-03-14T09:03:57.190025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182572+0000","last_change":"2024-03-14T09:03:57.177217+0000","last_active":"2024-03-14T09:03:57.182572+0000","last_peered":"2024-03-14T09:03:57.182572+0000","last_clean":"2024-03-14T09:03:57.182572+0000","last_became_active":"2024-03-14T09:03:57.176941+0000","last_became_peered":"2024-03-14T09:03:57.176941+0000","last_unstale":"2024-03-14T09:03:57.182572+0000","last_undegraded":"2024-03-14T09:03:57.182572+0000","last_fullsized":"2024-03-14T09:03:57.182572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186217+0000","last_change":"2024-03-14T09:03:57.176770+0000","last_active":"2024-03-14T09:03:57.186217+0000","last_peered":"2024-03-14T09:03:57.186217+0000","last_clean":"2024-03-14T09:03:57.186217+0000","last_became_active":"2024-03-14T09:03:57.176496+0000","last_became_peered":"2024-03-14T09:03:57.176496+0000","last_unstale":"2024-03-14T09:03:57.186217+0000","last_undegraded":"2024-03-14T09:03:57.186217+0000","last_fullsized":"2024-03-14T09:03:57.186217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:32.959955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193128+0000","last_change":"2024-03-14T09:03:57.188745+0000","last_active":"2024-03-14T09:03:57.193128+0000","last_peered":"2024-03-14T09:03:57.193128+0000","last_clean":"2024-03-14T09:03:57.193128+0000","last_became_active":"2024-03-14T09:03:57.187830+0000","last_became_peered":"2024-03-14T09:03:57.187830+0000","last_unstale":"2024-03-14T09:03:57.193128+0000","last_undegraded":"2024-03-14T09:03:57.193128+0000","last_fullsized":"2024-03-14T09:03:57.193128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192997+0000","last_change":"2024-03-14T09:03:57.186877+0000","last_active":"2024-03-14T09:03:57.192997+0000","last_peered":"2024-03-14T09:03:57.192997+0000","last_clean":"2024-03-14T09:03:57.192997+0000","last_became_active":"2024-03-14T09:03:57.186581+0000","last_became_peered":"2024-03-14T09:03:57.186581+0000","last_unstale":"2024-03-14T09:03:57.192997+0000","last_undegraded":"2024-03-14T09:03:57.192997+0000","last_fullsized":"2024-03-14T09:03:57.192997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:51.324739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193011+0000","last_change":"2024-03-14T09:03:57.187026+0000","last_active":"2024-03-14T09:03:57.193011+0000","last_peered":"2024-03-14T09:03:57.193011+0000","last_clean":"2024-03-14T09:03:57.193011+0000","last_became_active":"2024-03-14T09:03:57.186774+0000","last_became_peered":"2024-03-14T09:03:57.186774+0000","last_unstale":"2024-03-14T09:03:57.193011+0000","last_undegraded":"2024-03-14T09:03:57.193011+0000","last_fullsized":"2024-03-14T09:03:57.193011+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:55:25.363869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192570+0000","last_change":"2024-03-14T09:03:57.189350+0000","last_active":"2024-03-14T09:03:57.192570+0000","last_peered":"2024-03-14T09:03:57.192570+0000","last_clean":"2024-03-14T09:03:57.192570+0000","last_became_active":"2024-03-14T09:03:57.189109+0000","last_became_peered":"2024-03-14T09:03:57.189109+0000","last_unstale":"2024-03-14T09:03:57.192570+0000","last_undegraded":"2024-03-14T09:03:57.192570+0000","last_fullsized":"2024-03-14T09:03:57.192570+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:18.878301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190524+0000","last_change":"2024-03-14T09:03:57.186278+0000","last_active":"2024-03-14T09:03:57.190524+0000","last_peered":"2024-03-14T09:03:57.190524+0000","last_clean":"2024-03-14T09:03:57.190524+0000","last_became_active":"2024-03-14T09:03:57.186014+0000","last_became_peered":"2024-03-14T09:03:57.186014+0000","last_unstale":"2024-03-14T09:03:57.190524+0000","last_undegraded":"2024-03-14T09:03:57.190524+0000","last_fullsized":"2024-03-14T09:03:57.190524+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193008+0000","last_change":"2024-03-14T09:03:57.186749+0000","last_active":"2024-03-14T09:03:57.193008+0000","last_peered":"2024-03-14T09:03:57.193008+0000","last_clean":"2024-03-14T09:03:57.193008+0000","last_became_active":"2024-03-14T09:03:57.186502+0000","last_became_peered":"2024-03-14T09:03:57.186502+0000","last_unstale":"2024-03-14T09:03:57.193008+0000","last_undegraded":"2024-03-14T09:03:57.193008+0000","last_fullsized":"2024-03-14T09:03:57.193008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:10.996899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192643+0000","last_change":"2024-03-14T09:03:57.185209+0000","last_active":"2024-03-14T09:03:57.192643+0000","last_peered":"2024-03-14T09:03:57.192643+0000","last_clean":"2024-03-14T09:03:57.192643+0000","last_became_active":"2024-03-14T09:03:57.184942+0000","last_became_peered":"2024-03-14T09:03:57.184942+0000","last_unstale":"2024-03-14T09:03:57.192643+0000","last_undegraded":"2024-03-14T09:03:57.192643+0000","last_fullsized":"2024-03-14T09:03:57.192643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:37.052225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190868+0000","last_change":"2024-03-14T09:03:57.189881+0000","last_active":"2024-03-14T09:03:57.190868+0000","last_peered":"2024-03-14T09:03:57.190868+0000","last_clean":"2024-03-14T09:03:57.190868+0000","last_became_active":"2024-03-14T09:03:57.189632+0000","last_became_peered":"2024-03-14T09:03:57.189632+0000","last_unstale":"2024-03-14T09:03:57.190868+0000","last_undegraded":"2024-03-14T09:03:57.190868+0000","last_fullsized":"2024-03-14T09:03:57.190868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.170931+0000","last_change":"2024-03-14T09:03:57.168978+0000","last_active":"2024-03-14T09:03:57.170931+0000","last_peered":"2024-03-14T09:03:57.170931+0000","last_clean":"2024-03-14T09:03:57.170931+0000","last_became_active":"2024-03-14T09:03:57.167802+0000","last_became_peered":"2024-03-14T09:03:57.167802+0000","last_unstale":"2024-03-14T09:03:57.170931+0000","last_undegraded":"2024-03-14T09:03:57.170931+0000","last_fullsized":"2024-03-14T09:03:57.170931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188955+0000","last_change":"2024-03-14T09:03:57.183439+0000","last_active":"2024-03-14T09:03:57.188955+0000","last_peered":"2024-03-14T09:03:57.188955+0000","last_clean":"2024-03-14T09:03:57.188955+0000","last_became_active":"2024-03-14T09:03:57.183150+0000","last_became_peered":"2024-03-14T09:03:57.183150+0000","last_unstale":"2024-03-14T09:03:57.188955+0000","last_undegraded":"2024-03-14T09:03:57.188955+0000","last_fullsized":"2024-03-14T09:03:57.188955+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192237+0000","last_change":"2024-03-14T09:03:57.184782+0000","last_active":"2024-03-14T09:03:57.192237+0000","last_peered":"2024-03-14T09:03:57.192237+0000","last_clean":"2024-03-14T09:03:57.192237+0000","last_became_active":"2024-03-14T09:03:57.184507+0000","last_became_peered":"2024-03-14T09:03:57.184507+0000","last_unstale":"2024-03-14T09:03:57.192237+0000","last_undegraded":"2024-03-14T09:03:57.192237+0000","last_fullsized":"2024-03-14T09:03:57.192237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:48.425158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192552+0000","last_change":"2024-03-14T09:03:57.185432+0000","last_active":"2024-03-14T09:03:57.192552+0000","last_peered":"2024-03-14T09:03:57.192552+0000","last_clean":"2024-03-14T09:03:57.192552+0000","last_became_active":"2024-03-14T09:03:57.184879+0000","last_became_peered":"2024-03-14T09:03:57.184879+0000","last_unstale":"2024-03-14T09:03:57.192552+0000","last_undegraded":"2024-03-14T09:03:57.192552+0000","last_fullsized":"2024-03-14T09:03:57.192552+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:22.429275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190892+0000","last_change":"2024-03-14T09:03:57.186461+0000","last_active":"2024-03-14T09:03:57.190892+0000","last_peered":"2024-03-14T09:03:57.190892+0000","last_clean":"2024-03-14T09:03:57.190892+0000","last_became_active":"2024-03-14T09:03:57.186176+0000","last_became_peered":"2024-03-14T09:03:57.186176+0000","last_unstale":"2024-03-14T09:03:57.190892+0000","last_undegraded":"2024-03-14T09:03:57.190892+0000","last_fullsized":"2024-03-14T09:03:57.190892+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190459+0000","last_change":"2024-03-14T09:03:57.185583+0000","last_active":"2024-03-14T09:03:57.190459+0000","last_peered":"2024-03-14T09:03:57.190459+0000","last_clean":"2024-03-14T09:03:57.190459+0000","last_became_active":"2024-03-14T09:03:57.185292+0000","last_became_peered":"2024-03-14T09:03:57.185292+0000","last_unstale":"2024-03-14T09:03:57.190459+0000","last_undegraded":"2024-03-14T09:03:57.190459+0000","last_fullsized":"2024-03-14T09:03:57.190459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190124+0000","last_change":"2024-03-14T09:03:57.182704+0000","last_active":"2024-03-14T09:03:57.190124+0000","last_peered":"2024-03-14T09:03:57.190124+0000","last_clean":"2024-03-14T09:03:57.190124+0000","last_became_active":"2024-03-14T09:03:57.182438+0000","last_became_peered":"2024-03-14T09:03:57.182438+0000","last_unstale":"2024-03-14T09:03:57.190124+0000","last_undegraded":"2024-03-14T09:03:57.190124+0000","last_fullsized":"2024-03-14T09:03:57.190124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"69'4","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191382+0000","last_change":"2024-03-14T09:03:56.771973+0000","last_active":"2024-03-14T09:03:57.191382+0000","last_peered":"2024-03-14T09:03:57.191382+0000","last_clean":"2024-03-14T09:03:57.191382+0000","last_became_active":"2024-03-14T09:03:56.771714+0000","last_became_peered":"2024-03-14T09:03:56.771714+0000","last_unstale":"2024-03-14T09:03:57.191382+0000","last_undegraded":"2024-03-14T09:03:57.191382+0000","last_fullsized":"2024-03-14T09:03:57.191382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:17.873268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190487+0000","last_change":"2024-03-14T09:03:57.185961+0000","last_active":"2024-03-14T09:03:57.190487+0000","last_peered":"2024-03-14T09:03:57.190487+0000","last_clean":"2024-03-14T09:03:57.190487+0000","last_became_active":"2024-03-14T09:03:57.185658+0000","last_became_peered":"2024-03-14T09:03:57.185658+0000","last_unstale":"2024-03-14T09:03:57.190487+0000","last_undegraded":"2024-03-14T09:03:57.190487+0000","last_fullsized":"2024-03-14T09:03:57.190487+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182170+0000","last_change":"2024-03-14T09:03:56.769740+0000","last_active":"2024-03-14T09:03:57.182170+0000","last_peered":"2024-03-14T09:03:57.182170+0000","last_clean":"2024-03-14T09:03:57.182170+0000","last_became_active":"2024-03-14T09:03:56.769429+0000","last_became_peered":"2024-03-14T09:03:56.769429+0000","last_unstale":"2024-03-14T09:03:57.182170+0000","last_undegraded":"2024-03-14T09:03:57.182170+0000","last_fullsized":"2024-03-14T09:03:57.182170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190683+0000","last_change":"2024-03-14T09:03:57.188083+0000","last_active":"2024-03-14T09:03:57.190683+0000","last_peered":"2024-03-14T09:03:57.190683+0000","last_clean":"2024-03-14T09:03:57.190683+0000","last_became_active":"2024-03-14T09:03:57.187832+0000","last_became_peered":"2024-03-14T09:03:57.187832+0000","last_unstale":"2024-03-14T09:03:57.190683+0000","last_undegraded":"2024-03-14T09:03:57.190683+0000","last_fullsized":"2024-03-14T09:03:57.190683+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192826+0000","last_change":"2024-03-14T09:03:56.771927+0000","last_active":"2024-03-14T09:03:57.192826+0000","last_peered":"2024-03-14T09:03:57.192826+0000","last_clean":"2024-03-14T09:03:57.192826+0000","last_became_active":"2024-03-14T09:03:56.771684+0000","last_became_peered":"2024-03-14T09:03:56.771684+0000","last_unstale":"2024-03-14T09:03:57.192826+0000","last_undegraded":"2024-03-14T09:03:57.192826+0000","last_fullsized":"2024-03-14T09:03:57.192826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184425+0000","last_change":"2024-03-14T09:03:57.180285+0000","last_active":"2024-03-14T09:03:57.184425+0000","last_peered":"2024-03-14T09:03:57.184425+0000","last_clean":"2024-03-14T09:03:57.184425+0000","last_became_active":"2024-03-14T09:03:57.179838+0000","last_became_peered":"2024-03-14T09:03:57.179838+0000","last_unstale":"2024-03-14T09:03:57.184425+0000","last_undegraded":"2024-03-14T09:03:57.184425+0000","last_fullsized":"2024-03-14T09:03:57.184425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190644+0000","last_change":"2024-03-14T09:03:56.772021+0000","last_active":"2024-03-14T09:03:57.190644+0000","last_peered":"2024-03-14T09:03:57.190644+0000","last_clean":"2024-03-14T09:03:57.190644+0000","last_became_active":"2024-03-14T09:03:56.771764+0000","last_became_peered":"2024-03-14T09:03:56.771764+0000","last_unstale":"2024-03-14T09:03:57.190644+0000","last_undegraded":"2024-03-14T09:03:57.190644+0000","last_fullsized":"2024-03-14T09:03:57.190644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:30.727586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192528+0000","last_change":"2024-03-14T09:03:57.185267+0000","last_active":"2024-03-14T09:03:57.192528+0000","last_peered":"2024-03-14T09:03:57.192528+0000","last_clean":"2024-03-14T09:03:57.192528+0000","last_became_active":"2024-03-14T09:03:57.184614+0000","last_became_peered":"2024-03-14T09:03:57.184614+0000","last_unstale":"2024-03-14T09:03:57.192528+0000","last_undegraded":"2024-03-14T09:03:57.192528+0000","last_fullsized":"2024-03-14T09:03:57.192528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:25.314389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186165+0000","last_change":"2024-03-14T09:03:56.770949+0000","last_active":"2024-03-14T09:03:57.186165+0000","last_peered":"2024-03-14T09:03:57.186165+0000","last_clean":"2024-03-14T09:03:57.186165+0000","last_became_active":"2024-03-14T09:03:56.770678+0000","last_became_peered":"2024-03-14T09:03:56.770678+0000","last_unstale":"2024-03-14T09:03:57.186165+0000","last_undegraded":"2024-03-14T09:03:57.186165+0000","last_fullsized":"2024-03-14T09:03:57.186165+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182189+0000","last_change":"2024-03-14T09:03:57.178070+0000","last_active":"2024-03-14T09:03:57.182189+0000","last_peered":"2024-03-14T09:03:57.182189+0000","last_clean":"2024-03-14T09:03:57.182189+0000","last_became_active":"2024-03-14T09:03:57.177621+0000","last_became_peered":"2024-03-14T09:03:57.177621+0000","last_unstale":"2024-03-14T09:03:57.182189+0000","last_undegraded":"2024-03-14T09:03:57.182189+0000","last_fullsized":"2024-03-14T09:03:57.182189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:55.259556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.181107+0000","last_change":"2024-03-14T09:03:56.768242+0000","last_active":"2024-03-14T09:03:57.181107+0000","last_peered":"2024-03-14T09:03:57.181107+0000","last_clean":"2024-03-14T09:03:57.181107+0000","last_became_active":"2024-03-14T09:03:56.767960+0000","last_became_peered":"2024-03-14T09:03:56.767960+0000","last_unstale":"2024-03-14T09:03:57.181107+0000","last_undegraded":"2024-03-14T09:03:57.181107+0000","last_fullsized":"2024-03-14T09:03:57.181107+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190804+0000","last_change":"2024-03-14T09:03:57.187305+0000","last_active":"2024-03-14T09:03:57.190804+0000","last_peered":"2024-03-14T09:03:57.190804+0000","last_clean":"2024-03-14T09:03:57.190804+0000","last_became_active":"2024-03-14T09:03:57.186346+0000","last_became_peered":"2024-03-14T09:03:57.186346+0000","last_unstale":"2024-03-14T09:03:57.190804+0000","last_undegraded":"2024-03-14T09:03:57.190804+0000","last_fullsized":"2024-03-14T09:03:57.190804+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:04.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.178742+0000","last_change":"2024-03-14T09:03:56.773036+0000","last_active":"2024-03-14T09:03:57.178742+0000","last_peered":"2024-03-14T09:03:57.178742+0000","last_clean":"2024-03-14T09:03:57.178742+0000","last_became_active":"2024-03-14T09:03:56.772740+0000","last_became_peered":"2024-03-14T09:03:56.772740+0000","last_unstale":"2024-03-14T09:03:57.178742+0000","last_undegraded":"2024-03-14T09:03:57.178742+0000","last_fullsized":"2024-03-14T09:03:57.178742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:44.982623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180219+0000","last_change":"2024-03-14T09:03:57.172128+0000","last_active":"2024-03-14T09:03:57.180219+0000","last_peered":"2024-03-14T09:03:57.180219+0000","last_clean":"2024-03-14T09:03:57.180219+0000","last_became_active":"2024-03-14T09:03:57.171877+0000","last_became_peered":"2024-03-14T09:03:57.171877+0000","last_unstale":"2024-03-14T09:03:57.180219+0000","last_undegraded":"2024-03-14T09:03:57.180219+0000","last_fullsized":"2024-03-14T09:03:57.180219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:25.355300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.171531+0000","last_change":"2024-03-14T09:03:56.769464+0000","last_active":"2024-03-14T09:03:57.171531+0000","last_peered":"2024-03-14T09:03:57.171531+0000","last_clean":"2024-03-14T09:03:57.171531+0000","last_became_active":"2024-03-14T09:03:56.769231+0000","last_became_peered":"2024-03-14T09:03:56.769231+0000","last_unstale":"2024-03-14T09:03:57.171531+0000","last_undegraded":"2024-03-14T09:03:57.171531+0000","last_fullsized":"2024-03-14T09:03:57.171531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:50.319706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191239+0000","last_change":"2024-03-14T09:03:57.188282+0000","last_active":"2024-03-14T09:03:57.191239+0000","last_peered":"2024-03-14T09:03:57.191239+0000","last_clean":"2024-03-14T09:03:57.191239+0000","last_became_active":"2024-03-14T09:03:57.188018+0000","last_became_peered":"2024-03-14T09:03:57.188018+0000","last_unstale":"2024-03-14T09:03:57.191239+0000","last_undegraded":"2024-03-14T09:03:57.191239+0000","last_fullsized":"2024-03-14T09:03:57.191239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:22.289306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.174983+0000","last_change":"2024-03-14T09:03:56.768124+0000","last_active":"2024-03-14T09:03:57.174983+0000","last_peered":"2024-03-14T09:03:57.174983+0000","last_clean":"2024-03-14T09:03:57.174983+0000","last_became_active":"2024-03-14T09:03:56.766669+0000","last_became_peered":"2024-03-14T09:03:56.766669+0000","last_unstale":"2024-03-14T09:03:57.174983+0000","last_undegraded":"2024-03-14T09:03:57.174983+0000","last_fullsized":"2024-03-14T09:03:57.174983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:11.739150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187522+0000","last_change":"2024-03-14T09:03:57.179078+0000","last_active":"2024-03-14T09:03:57.187522+0000","last_peered":"2024-03-14T09:03:57.187522+0000","last_clean":"2024-03-14T09:03:57.187522+0000","last_became_active":"2024-03-14T09:03:57.178782+0000","last_became_peered":"2024-03-14T09:03:57.178782+0000","last_unstale":"2024-03-14T09:03:57.187522+0000","last_undegraded":"2024-03-14T09:03:57.187522+0000","last_fullsized":"2024-03-14T09:03:57.187522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.173970+0000","last_change":"2024-03-14T09:03:56.769911+0000","last_active":"2024-03-14T09:03:57.173970+0000","last_peered":"2024-03-14T09:03:57.173970+0000","last_clean":"2024-03-14T09:03:57.173970+0000","last_became_active":"2024-03-14T09:03:56.769598+0000","last_became_peered":"2024-03-14T09:03:56.769598+0000","last_unstale":"2024-03-14T09:03:57.173970+0000","last_undegraded":"2024-03-14T09:03:57.173970+0000","last_fullsized":"2024-03-14T09:03:57.173970+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192678+0000","last_change":"2024-03-14T09:03:57.184589+0000","last_active":"2024-03-14T09:03:57.192678+0000","last_peered":"2024-03-14T09:03:57.192678+0000","last_clean":"2024-03-14T09:03:57.192678+0000","last_became_active":"2024-03-14T09:03:57.184303+0000","last_became_peered":"2024-03-14T09:03:57.184303+0000","last_unstale":"2024-03-14T09:03:57.192678+0000","last_undegraded":"2024-03-14T09:03:57.192678+0000","last_fullsized":"2024-03-14T09:03:57.192678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"69'3","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187997+0000","last_change":"2024-03-14T09:03:56.766306+0000","last_active":"2024-03-14T09:03:57.187997+0000","last_peered":"2024-03-14T09:03:57.187997+0000","last_clean":"2024-03-14T09:03:57.187997+0000","last_became_active":"2024-03-14T09:03:56.766045+0000","last_became_peered":"2024-03-14T09:03:56.766045+0000","last_unstale":"2024-03-14T09:03:57.187997+0000","last_undegraded":"2024-03-14T09:03:57.187997+0000","last_fullsized":"2024-03-14T09:03:57.187997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191068+0000","last_change":"2024-03-14T09:03:57.189394+0000","last_active":"2024-03-14T09:03:57.191068+0000","last_peered":"2024-03-14T09:03:57.191068+0000","last_clean":"2024-03-14T09:03:57.191068+0000","last_became_active":"2024-03-14T09:03:57.189162+0000","last_became_peered":"2024-03-14T09:03:57.189162+0000","last_unstale":"2024-03-14T09:03:57.191068+0000","last_undegraded":"2024-03-14T09:03:57.191068+0000","last_fullsized":"2024-03-14T09:03:57.191068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"69'6","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.196192+0000","last_change":"2024-03-14T09:03:56.769295+0000","last_active":"2024-03-14T09:03:57.196192+0000","last_peered":"2024-03-14T09:03:57.196192+0000","last_clean":"2024-03-14T09:03:57.196192+0000","last_became_active":"2024-03-14T09:03:56.768952+0000","last_became_peered":"2024-03-14T09:03:56.768952+0000","last_unstale":"2024-03-14T09:03:57.196192+0000","last_undegraded":"2024-03-14T09:03:57.196192+0000","last_fullsized":"2024-03-14T09:03:57.196192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193087+0000","last_change":"2024-03-14T09:03:57.188359+0000","last_active":"2024-03-14T09:03:57.193087+0000","last_peered":"2024-03-14T09:03:57.193087+0000","last_clean":"2024-03-14T09:03:57.193087+0000","last_became_active":"2024-03-14T09:03:57.188093+0000","last_became_peered":"2024-03-14T09:03:57.188093+0000","last_unstale":"2024-03-14T09:03:57.193087+0000","last_undegraded":"2024-03-14T09:03:57.193087+0000","last_fullsized":"2024-03-14T09:03:57.193087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191864+0000","last_change":"2024-03-14T09:03:56.769589+0000","last_active":"2024-03-14T09:03:57.191864+0000","last_peered":"2024-03-14T09:03:57.191864+0000","last_clean":"2024-03-14T09:03:57.191864+0000","last_became_active":"2024-03-14T09:03:56.767656+0000","last_became_peered":"2024-03-14T09:03:56.767656+0000","last_unstale":"2024-03-14T09:03:57.191864+0000","last_undegraded":"2024-03-14T09:03:57.191864+0000","last_fullsized":"2024-03-14T09:03:57.191864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:31.954923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190788+0000","last_change":"2024-03-14T09:03:57.187466+0000","last_active":"2024-03-14T09:03:57.190788+0000","last_peered":"2024-03-14T09:03:57.190788+0000","last_clean":"2024-03-14T09:03:57.190788+0000","last_became_active":"2024-03-14T09:03:57.186590+0000","last_became_peered":"2024-03-14T09:03:57.186590+0000","last_unstale":"2024-03-14T09:03:57.190788+0000","last_undegraded":"2024-03-14T09:03:57.190788+0000","last_fullsized":"2024-03-14T09:03:57.190788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:04.118845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.164091+0000","last_change":"2024-03-14T09:03:56.762545+0000","last_active":"2024-03-14T09:03:57.164091+0000","last_peered":"2024-03-14T09:03:57.164091+0000","last_clean":"2024-03-14T09:03:57.164091+0000","last_became_active":"2024-03-14T09:03:56.762292+0000","last_became_peered":"2024-03-14T09:03:56.762292+0000","last_unstale":"2024-03-14T09:03:57.164091+0000","last_undegraded":"2024-03-14T09:03:57.164091+0000","last_fullsized":"2024-03-14T09:03:57.164091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:03.013048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192002+0000","last_change":"2024-03-14T09:03:57.184740+0000","last_active":"2024-03-14T09:03:57.192002+0000","last_peered":"2024-03-14T09:03:57.192002+0000","last_clean":"2024-03-14T09:03:57.192002+0000","last_became_active":"2024-03-14T09:03:57.184407+0000","last_became_peered":"2024-03-14T09:03:57.184407+0000","last_unstale":"2024-03-14T09:03:57.192002+0000","last_undegraded":"2024-03-14T09:03:57.192002+0000","last_fullsized":"2024-03-14T09:03:57.192002+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:12.744182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.170770+0000","last_change":"2024-03-14T09:03:56.773362+0000","last_active":"2024-03-14T09:03:57.170770+0000","last_peered":"2024-03-14T09:03:57.170770+0000","last_clean":"2024-03-14T09:03:57.170770+0000","last_became_active":"2024-03-14T09:03:56.773012+0000","last_became_peered":"2024-03-14T09:03:56.773012+0000","last_unstale":"2024-03-14T09:03:57.170770+0000","last_undegraded":"2024-03-14T09:03:57.170770+0000","last_fullsized":"2024-03-14T09:03:57.170770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:38.243214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192499+0000","last_change":"2024-03-14T09:03:57.187459+0000","last_active":"2024-03-14T09:03:57.192499+0000","last_peered":"2024-03-14T09:03:57.192499+0000","last_clean":"2024-03-14T09:03:57.192499+0000","last_became_active":"2024-03-14T09:03:57.187221+0000","last_became_peered":"2024-03-14T09:03:57.187221+0000","last_unstale":"2024-03-14T09:03:57.192499+0000","last_undegraded":"2024-03-14T09:03:57.192499+0000","last_fullsized":"2024-03-14T09:03:57.192499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:42.577661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"69'8","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191802+0000","last_change":"2024-03-14T09:03:56.770808+0000","last_active":"2024-03-14T09:03:57.191802+0000","last_peered":"2024-03-14T09:03:57.191802+0000","last_clean":"2024-03-14T09:03:57.191802+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:03:57.191802+0000","last_undegraded":"2024-03-14T09:03:57.191802+0000","last_fullsized":"2024-03-14T09:03:57.191802+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:29.358695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190064+0000","last_change":"2024-03-14T09:03:57.182905+0000","last_active":"2024-03-14T09:03:57.190064+0000","last_peered":"2024-03-14T09:03:57.190064+0000","last_clean":"2024-03-14T09:03:57.190064+0000","last_became_active":"2024-03-14T09:03:57.182631+0000","last_became_peered":"2024-03-14T09:03:57.182631+0000","last_unstale":"2024-03-14T09:03:57.190064+0000","last_undegraded":"2024-03-14T09:03:57.190064+0000","last_fullsized":"2024-03-14T09:03:57.190064+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:05.353470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190608+0000","last_change":"2024-03-14T09:03:56.770834+0000","last_active":"2024-03-14T09:03:57.190608+0000","last_peered":"2024-03-14T09:03:57.190608+0000","last_clean":"2024-03-14T09:03:57.190608+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:03:57.190608+0000","last_undegraded":"2024-03-14T09:03:57.190608+0000","last_fullsized":"2024-03-14T09:03:57.190608+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.183557+0000","last_change":"2024-03-14T09:03:57.180017+0000","last_active":"2024-03-14T09:03:57.183557+0000","last_peered":"2024-03-14T09:03:57.183557+0000","last_clean":"2024-03-14T09:03:57.183557+0000","last_became_active":"2024-03-14T09:03:57.179589+0000","last_became_peered":"2024-03-14T09:03:57.179589+0000","last_unstale":"2024-03-14T09:03:57.183557+0000","last_undegraded":"2024-03-14T09:03:57.183557+0000","last_fullsized":"2024-03-14T09:03:57.183557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190661+0000","last_change":"2024-03-14T09:03:56.769821+0000","last_active":"2024-03-14T09:03:57.190661+0000","last_peered":"2024-03-14T09:03:57.190661+0000","last_clean":"2024-03-14T09:03:57.190661+0000","last_became_active":"2024-03-14T09:03:56.769597+0000","last_became_peered":"2024-03-14T09:03:56.769597+0000","last_unstale":"2024-03-14T09:03:57.190661+0000","last_undegraded":"2024-03-14T09:03:57.190661+0000","last_fullsized":"2024-03-14T09:03:57.190661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:01.018014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190974+0000","last_change":"2024-03-14T09:03:57.187274+0000","last_active":"2024-03-14T09:03:57.190974+0000","last_peered":"2024-03-14T09:03:57.190974+0000","last_clean":"2024-03-14T09:03:57.190974+0000","last_became_active":"2024-03-14T09:03:57.187035+0000","last_became_peered":"2024-03-14T09:03:57.187035+0000","last_unstale":"2024-03-14T09:03:57.190974+0000","last_undegraded":"2024-03-14T09:03:57.190974+0000","last_fullsized":"2024-03-14T09:03:57.190974+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.179257+0000","last_change":"2024-03-14T09:03:56.772665+0000","last_active":"2024-03-14T09:03:57.179257+0000","last_peered":"2024-03-14T09:03:57.179257+0000","last_clean":"2024-03-14T09:03:57.179257+0000","last_became_active":"2024-03-14T09:03:56.772464+0000","last_became_peered":"2024-03-14T09:03:56.772464+0000","last_unstale":"2024-03-14T09:03:57.179257+0000","last_undegraded":"2024-03-14T09:03:57.179257+0000","last_fullsized":"2024-03-14T09:03:57.179257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:22.876530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194060+0000","last_change":"2024-03-14T09:03:57.191774+0000","last_active":"2024-03-14T09:03:57.194060+0000","last_peered":"2024-03-14T09:03:57.194060+0000","last_clean":"2024-03-14T09:03:57.194060+0000","last_became_active":"2024-03-14T09:03:57.191391+0000","last_became_peered":"2024-03-14T09:03:57.191391+0000","last_unstale":"2024-03-14T09:03:57.194060+0000","last_undegraded":"2024-03-14T09:03:57.194060+0000","last_fullsized":"2024-03-14T09:03:57.194060+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:22.069879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180009+0000","last_change":"2024-03-14T09:03:56.759918+0000","last_active":"2024-03-14T09:03:57.180009+0000","last_peered":"2024-03-14T09:03:57.180009+0000","last_clean":"2024-03-14T09:03:57.180009+0000","last_became_active":"2024-03-14T09:03:56.759495+0000","last_became_peered":"2024-03-14T09:03:56.759495+0000","last_unstale":"2024-03-14T09:03:57.180009+0000","last_undegraded":"2024-03-14T09:03:57.180009+0000","last_fullsized":"2024-03-14T09:03:57.180009+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:04.348438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188535+0000","last_change":"2024-03-14T09:03:57.181268+0000","last_active":"2024-03-14T09:03:57.188535+0000","last_peered":"2024-03-14T09:03:57.188535+0000","last_clean":"2024-03-14T09:03:57.188535+0000","last_became_active":"2024-03-14T09:03:57.180967+0000","last_became_peered":"2024-03-14T09:03:57.180967+0000","last_unstale":"2024-03-14T09:03:57.188535+0000","last_undegraded":"2024-03-14T09:03:57.188535+0000","last_fullsized":"2024-03-14T09:03:57.188535+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165514+0000","last_change":"2024-03-14T09:03:56.768053+0000","last_active":"2024-03-14T09:03:57.165514+0000","last_peered":"2024-03-14T09:03:57.165514+0000","last_clean":"2024-03-14T09:03:57.165514+0000","last_became_active":"2024-03-14T09:03:56.767780+0000","last_became_peered":"2024-03-14T09:03:56.767780+0000","last_unstale":"2024-03-14T09:03:57.165514+0000","last_undegraded":"2024-03-14T09:03:57.165514+0000","last_fullsized":"2024-03-14T09:03:57.165514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:00:23.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186832+0000","last_change":"2024-03-14T09:03:57.177141+0000","last_active":"2024-03-14T09:03:57.186832+0000","last_peered":"2024-03-14T09:03:57.186832+0000","last_clean":"2024-03-14T09:03:57.186832+0000","last_became_active":"2024-03-14T09:03:57.176843+0000","last_became_peered":"2024-03-14T09:03:57.176843+0000","last_unstale":"2024-03-14T09:03:57.186832+0000","last_undegraded":"2024-03-14T09:03:57.186832+0000","last_fullsized":"2024-03-14T09:03:57.186832+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:57.185084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165305+0000","last_change":"2024-03-14T09:03:56.764413+0000","last_active":"2024-03-14T09:03:57.165305+0000","last_peered":"2024-03-14T09:03:57.165305+0000","last_clean":"2024-03-14T09:03:57.165305+0000","last_became_active":"2024-03-14T09:03:56.764156+0000","last_became_peered":"2024-03-14T09:03:56.764156+0000","last_unstale":"2024-03-14T09:03:57.165305+0000","last_undegraded":"2024-03-14T09:03:57.165305+0000","last_fullsized":"2024-03-14T09:03:57.165305+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:21:29.624660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190200+0000","last_change":"2024-03-14T09:03:57.187334+0000","last_active":"2024-03-14T09:03:57.190200+0000","last_peered":"2024-03-14T09:03:57.190200+0000","last_clean":"2024-03-14T09:03:57.190200+0000","last_became_active":"2024-03-14T09:03:57.187054+0000","last_became_peered":"2024-03-14T09:03:57.187054+0000","last_unstale":"2024-03-14T09:03:57.190200+0000","last_undegraded":"2024-03-14T09:03:57.190200+0000","last_fullsized":"2024-03-14T09:03:57.190200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.163616+0000","last_change":"2024-03-14T09:03:56.759664+0000","last_active":"2024-03-14T09:03:57.163616+0000","last_peered":"2024-03-14T09:03:57.163616+0000","last_clean":"2024-03-14T09:03:57.163616+0000","last_became_active":"2024-03-14T09:03:56.759378+0000","last_became_peered":"2024-03-14T09:03:56.759378+0000","last_unstale":"2024-03-14T09:03:57.163616+0000","last_undegraded":"2024-03-14T09:03:57.163616+0000","last_fullsized":"2024-03-14T09:03:57.163616+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:03.113813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182669+0000","last_change":"2024-03-14T09:03:57.177315+0000","last_active":"2024-03-14T09:03:57.182669+0000","last_peered":"2024-03-14T09:03:57.182669+0000","last_clean":"2024-03-14T09:03:57.182669+0000","last_became_active":"2024-03-14T09:03:57.177045+0000","last_became_peered":"2024-03-14T09:03:57.177045+0000","last_unstale":"2024-03-14T09:03:57.182669+0000","last_undegraded":"2024-03-14T09:03:57.182669+0000","last_fullsized":"2024-03-14T09:03:57.182669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184947+0000","last_change":"2024-03-14T09:03:56.764563+0000","last_active":"2024-03-14T09:03:57.184947+0000","last_peered":"2024-03-14T09:03:57.184947+0000","last_clean":"2024-03-14T09:03:57.184947+0000","last_became_active":"2024-03-14T09:03:56.764299+0000","last_became_peered":"2024-03-14T09:03:56.764299+0000","last_unstale":"2024-03-14T09:03:57.184947+0000","last_undegraded":"2024-03-14T09:03:57.184947+0000","last_fullsized":"2024-03-14T09:03:57.184947+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:22.457392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190035+0000","last_change":"2024-03-14T09:03:57.182451+0000","last_active":"2024-03-14T09:03:57.190035+0000","last_peered":"2024-03-14T09:03:57.190035+0000","last_clean":"2024-03-14T09:03:57.190035+0000","last_became_active":"2024-03-14T09:03:57.182007+0000","last_became_peered":"2024-03-14T09:03:57.182007+0000","last_unstale":"2024-03-14T09:03:57.190035+0000","last_undegraded":"2024-03-14T09:03:57.190035+0000","last_fullsized":"2024-03-14T09:03:57.190035+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.175176+0000","last_change":"2024-03-14T09:03:56.768285+0000","last_active":"2024-03-14T09:03:57.175176+0000","last_peered":"2024-03-14T09:03:57.175176+0000","last_clean":"2024-03-14T09:03:57.175176+0000","last_became_active":"2024-03-14T09:03:56.768060+0000","last_became_peered":"2024-03-14T09:03:56.768060+0000","last_unstale":"2024-03-14T09:03:57.175176+0000","last_undegraded":"2024-03-14T09:03:57.175176+0000","last_fullsized":"2024-03-14T09:03:57.175176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:56.180051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190668+0000","last_change":"2024-03-14T09:03:57.188009+0000","last_active":"2024-03-14T09:03:57.190668+0000","last_peered":"2024-03-14T09:03:57.190668+0000","last_clean":"2024-03-14T09:03:57.190668+0000","last_became_active":"2024-03-14T09:03:57.187745+0000","last_became_peered":"2024-03-14T09:03:57.187745+0000","last_unstale":"2024-03-14T09:03:57.190668+0000","last_undegraded":"2024-03-14T09:03:57.190668+0000","last_fullsized":"2024-03-14T09:03:57.190668+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.168474+0000","last_change":"2024-03-14T09:03:56.764742+0000","last_active":"2024-03-14T09:03:57.168474+0000","last_peered":"2024-03-14T09:03:57.168474+0000","last_clean":"2024-03-14T09:03:57.168474+0000","last_became_active":"2024-03-14T09:03:56.764467+0000","last_became_peered":"2024-03-14T09:03:56.764467+0000","last_unstale":"2024-03-14T09:03:57.168474+0000","last_undegraded":"2024-03-14T09:03:57.168474+0000","last_fullsized":"2024-03-14T09:03:57.168474+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191050+0000","last_change":"2024-03-14T09:03:57.189309+0000","last_active":"2024-03-14T09:03:57.191050+0000","last_peered":"2024-03-14T09:03:57.191050+0000","last_clean":"2024-03-14T09:03:57.191050+0000","last_became_active":"2024-03-14T09:03:57.189127+0000","last_became_peered":"2024-03-14T09:03:57.189127+0000","last_unstale":"2024-03-14T09:03:57.191050+0000","last_undegraded":"2024-03-14T09:03:57.191050+0000","last_fullsized":"2024-03-14T09:03:57.191050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.174752+0000","last_change":"2024-03-14T09:03:56.770115+0000","last_active":"2024-03-14T09:03:57.174752+0000","last_peered":"2024-03-14T09:03:57.174752+0000","last_clean":"2024-03-14T09:03:57.174752+0000","last_became_active":"2024-03-14T09:03:56.769621+0000","last_became_peered":"2024-03-14T09:03:56.769621+0000","last_unstale":"2024-03-14T09:03:57.174752+0000","last_undegraded":"2024-03-14T09:03:57.174752+0000","last_fullsized":"2024-03-14T09:03:57.174752+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187802+0000","last_change":"2024-03-14T09:03:57.180976+0000","last_active":"2024-03-14T09:03:57.187802+0000","last_peered":"2024-03-14T09:03:57.187802+0000","last_clean":"2024-03-14T09:03:57.187802+0000","last_became_active":"2024-03-14T09:03:57.180689+0000","last_became_peered":"2024-03-14T09:03:57.180689+0000","last_unstale":"2024-03-14T09:03:57.187802+0000","last_undegraded":"2024-03-14T09:03:57.187802+0000","last_fullsized":"2024-03-14T09:03:57.187802+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:02.023046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.179957+0000","last_change":"2024-03-14T09:03:56.770777+0000","last_active":"2024-03-14T09:03:57.179957+0000","last_peered":"2024-03-14T09:03:57.179957+0000","last_clean":"2024-03-14T09:03:57.179957+0000","last_became_active":"2024-03-14T09:03:56.770466+0000","last_became_peered":"2024-03-14T09:03:56.770466+0000","last_unstale":"2024-03-14T09:03:57.179957+0000","last_undegraded":"2024-03-14T09:03:57.179957+0000","last_fullsized":"2024-03-14T09:03:57.179957+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:41.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192729+0000","last_change":"2024-03-14T09:03:57.184709+0000","last_active":"2024-03-14T09:03:57.192729+0000","last_peered":"2024-03-14T09:03:57.192729+0000","last_clean":"2024-03-14T09:03:57.192729+0000","last_became_active":"2024-03-14T09:03:57.184380+0000","last_became_peered":"2024-03-14T09:03:57.184380+0000","last_unstale":"2024-03-14T09:03:57.192729+0000","last_undegraded":"2024-03-14T09:03:57.192729+0000","last_fullsized":"2024-03-14T09:03:57.192729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165302+0000","last_change":"2024-03-14T09:03:56.769939+0000","last_active":"2024-03-14T09:03:57.165302+0000","last_peered":"2024-03-14T09:03:57.165302+0000","last_clean":"2024-03-14T09:03:57.165302+0000","last_became_active":"2024-03-14T09:03:56.769670+0000","last_became_peered":"2024-03-14T09:03:56.769670+0000","last_unstale":"2024-03-14T09:03:57.165302+0000","last_undegraded":"2024-03-14T09:03:57.165302+0000","last_fullsized":"2024-03-14T09:03:57.165302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:50.421546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190782+0000","last_change":"2024-03-14T09:03:57.187259+0000","last_active":"2024-03-14T09:03:57.190782+0000","last_peered":"2024-03-14T09:03:57.190782+0000","last_clean":"2024-03-14T09:03:57.190782+0000","last_became_active":"2024-03-14T09:03:57.186972+0000","last_became_peered":"2024-03-14T09:03:57.186972+0000","last_unstale":"2024-03-14T09:03:57.190782+0000","last_undegraded":"2024-03-14T09:03:57.190782+0000","last_fullsized":"2024-03-14T09:03:57.190782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.168900+0000","last_change":"2024-03-14T09:03:56.771166+0000","last_active":"2024-03-14T09:03:57.168900+0000","last_peered":"2024-03-14T09:03:57.168900+0000","last_clean":"2024-03-14T09:03:57.168900+0000","last_became_active":"2024-03-14T09:03:56.769678+0000","last_became_peered":"2024-03-14T09:03:56.769678+0000","last_unstale":"2024-03-14T09:03:57.168900+0000","last_undegraded":"2024-03-14T09:03:57.168900+0000","last_fullsized":"2024-03-14T09:03:57.168900+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:06.742278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188743+0000","last_change":"2024-03-14T09:03:57.181824+0000","last_active":"2024-03-14T09:03:57.188743+0000","last_peered":"2024-03-14T09:03:57.188743+0000","last_clean":"2024-03-14T09:03:57.188743+0000","last_became_active":"2024-03-14T09:03:57.181573+0000","last_became_peered":"2024-03-14T09:03:57.181573+0000","last_unstale":"2024-03-14T09:03:57.188743+0000","last_undegraded":"2024-03-14T09:03:57.188743+0000","last_fullsized":"2024-03-14T09:03:57.188743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165173+0000","last_change":"2024-03-14T09:03:56.770803+0000","last_active":"2024-03-14T09:03:57.165173+0000","last_peered":"2024-03-14T09:03:57.165173+0000","last_clean":"2024-03-14T09:03:57.165173+0000","last_became_active":"2024-03-14T09:03:56.770311+0000","last_became_peered":"2024-03-14T09:03:56.770311+0000","last_unstale":"2024-03-14T09:03:57.165173+0000","last_undegraded":"2024-03-14T09:03:57.165173+0000","last_fullsized":"2024-03-14T09:03:57.165173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192676+0000","last_change":"2024-03-14T09:03:57.186670+0000","last_active":"2024-03-14T09:03:57.192676+0000","last_peered":"2024-03-14T09:03:57.192676+0000","last_clean":"2024-03-14T09:03:57.192676+0000","last_became_active":"2024-03-14T09:03:57.186398+0000","last_became_peered":"2024-03-14T09:03:57.186398+0000","last_unstale":"2024-03-14T09:03:57.192676+0000","last_undegraded":"2024-03-14T09:03:57.192676+0000","last_fullsized":"2024-03-14T09:03:57.192676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:15.878140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190981+0000","last_change":"2024-03-14T09:03:56.766595+0000","last_active":"2024-03-14T09:03:57.190981+0000","last_peered":"2024-03-14T09:03:57.190981+0000","last_clean":"2024-03-14T09:03:57.190981+0000","last_became_active":"2024-03-14T09:03:56.766269+0000","last_became_peered":"2024-03-14T09:03:56.766269+0000","last_unstale":"2024-03-14T09:03:57.190981+0000","last_undegraded":"2024-03-14T09:03:57.190981+0000","last_fullsized":"2024-03-14T09:03:57.190981+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-14T09:03:56.759931+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:56.759931+0000","last_peered":"2024-03-14T09:03:56.759931+0000","last_clean":"2024-03-14T09:03:56.759931+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:56.759931+0000","last_undegraded":"2024-03-14T09:03:56.759931+0000","last_fullsized":"2024-03-14T09:03:56.759931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28420,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343420,"statfs":{"total":96636764160,"available":96607662080,"internally_reserved":0,"allocated":1429504,"data_stored":952692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.85699999999999998,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":6.5,"15min":6.5},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53100000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68100000000000005,"5min":0.878,"15min":0.878},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":369793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.87,"15min":0.87},"last":0.63},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":362324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33900000000000002,"15min":0.34000000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.45700000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.378},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39400000000000002,"15min":0.39500000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.497,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.307}]},{"osd":4,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.436}]},{"osd":5,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28440,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343400,"statfs":{"total":96636764160,"available":96607641600,"internally_reserved":0,"allocated":1445888,"data_stored":959501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.71599999999999997,"15min":0.78200000000000003},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.41799999999999998,"5min":7.383,"15min":7.383},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.79200000000000004,"15min":0.86599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51800000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.81699999999999995,"15min":0.89200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.48899999999999999,"5min":7.944,"15min":7.944},"last":0.377}]},{"osd":3,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.77200000000000002,"15min":0.83599999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0349999999999999,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.629},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.98399999999999999,"15min":1.0489999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0509999999999999,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80300000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":369681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34200000000000003,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.373},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.40400000000000003,"5min":1.111,"15min":1.111},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.314},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.29699999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.67300000000000004,"15min":0.72999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.41199999999999998,"5min":7.5,"15min":7.5},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.71599999999999997,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.48699999999999999,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":1.151,"15min":1.151},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":1.179,"15min":1.179},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71799999999999997,"5min":0.751,"15min":0.751},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.75900000000000001,"5min":1.165,"15min":1.165},"last":0.436},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64900000000000002,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83199999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":369771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.311,"15min":0.308},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.42799999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.307}]},{"osd":2,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.373,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.45400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.378},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.45800000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75700000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76200000000000001,"5min":1.087,"15min":1.087},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":362324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51100000000000001,"15min":0.51600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.499,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.68999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.751,"15min":0.751},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.89100000000000001,"15min":0.94699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76000000000000001,"5min":7.702,"15min":7.702},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.753,"15min":0.753},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.749}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":370685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.44},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67900000000000005,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.627},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:01.346 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:01.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.469+0000 7f4c2f187700 1 Processor -- start 2024-03-14T09:04:01.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.469+0000 7f4c2f187700 1 -- start start 2024-03-14T09:04:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2f187700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4c2806d3c0 0x7f4c280c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2f187700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c280c1d90 0x7f4c280c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2f187700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c28151860 con 0x7f4c280c1830 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2f187700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c280c50e0 con 0x7f4c2806d3c0 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2f187700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28150f30 con 0x7f4c280c1d90 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2d724700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4c280c1830 0x7f4c280c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:51980/0 (socket says 172.21.15.154:51980) 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2d724700 1 -- 172.21.15.154:0/802438141 learned_addr learned my addr 172.21.15.154:0/802438141 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c27fff700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c280c1d90 0x7f4c280c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/802438141 >> v2:172.21.15.191:3300/0 conn(0x7f4c2806d3c0 0x7f4c280c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c277fe700 1 -- 172.21.15.154:0/802438141 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170105975 0 0) 0x7f4c28151860 con 0x7f4c280c1830 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c277fe700 1 -- 172.21.15.154:0/802438141 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c180035b0 con 0x7f4c280c1830 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2cf23700 1 -- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c280c1d90 msgr2=0x7f4c280c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c280c1d90 0x7f4c280c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2cf23700 1 -- 172.21.15.154:0/802438141 >> v1:172.21.15.154:6789/0 conn(0x7f4c280c1830 legacy=0x7f4c280c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.470+0000 7f4c2cf23700 1 -- 172.21.15.154:0/802438141 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28151210 con 0x7f4c2806d3c0 2024-03-14T09:04:01.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806d3c0 0x7f4c280c12f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009840 tx=0x7f4c1c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f06b47fc68ac4c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c277fe700 1 -- 172.21.15.154:0/802438141 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c011830 con 0x7f4c2806d3c0 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c277fe700 1 -- 172.21.15.154:0/802438141 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c011990 con 0x7f4c2806d3c0 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c277fe700 1 -- 172.21.15.154:0/802438141 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c011d20 con 0x7f4c2806d3c0 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 -- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806d3c0 msgr2=0x7f4c280c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806d3c0 0x7f4c280c12f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009840 tx=0x7f4c1c009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 -- 172.21.15.154:0/802438141 shutdown_connections 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806d3c0 0x7f4c280c12f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 --2- 172.21.15.154:0/802438141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c280c1d90 0x7f4c280c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 -- 172.21.15.154:0/802438141 >> 172.21.15.154:0/802438141 conn(0x7f4c2806a770 msgr2=0x7f4c2806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 -- 172.21.15.154:0/802438141 shutdown_connections 2024-03-14T09:04:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.471+0000 7f4c2f187700 1 -- 172.21.15.154:0/802438141 wait complete. 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.472+0000 7f4c2f187700 1 Processor -- start 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 -- start start 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c2806d3c0 0x7f4c2814c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 0x7f4c280a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2814eb00 con 0x7f4c280c1d90 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2814e800 con 0x7f4c280c1830 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2f187700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2814f610 con 0x7f4c2806d3c0 2024-03-14T09:04:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2d724700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2d724700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39250/0 (socket says 172.21.15.154:39250) 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.473+0000 7f4c2d724700 1 -- 172.21.15.154:0/2678028819 learned_addr learned my addr 172.21.15.154:0/2678028819 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c27fff700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 0x7f4c280a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c2806d3c0 0x7f4c2814c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 -- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c2806d3c0 msgr2=0x7f4c2814c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c2806d3c0 0x7f4c2814c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 -- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 msgr2=0x7f4c280a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 0x7f4c280a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 -- 172.21.15.154:0/2678028819 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0094a0 con 0x7f4c280c1d90 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c27fff700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 0x7f4c280a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2d724700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000cb70 tx=0x7f4c2000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000ab10 con 0x7f4c280c1d90 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2000ac70 con 0x7f4c280c1d90 2024-03-14T09:04:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2814f8f0 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20012400 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.474+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c280ac890 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.475+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2814ecb0 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.476+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4c2001b3b0 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.476+0000 7f4c257fa700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 0x7f4c0c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.476+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c20088aa0 con 0x7f4c280c1d90 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.476+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 0x7f4c0c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.477+0000 7f4c2cf23700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 0x7f4c0c06ccf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.480+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4c2814ecb0 con 0x7f4c280c1d90 2024-03-14T09:04:01.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.669+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c28066550 con 0x7f4c0c06a830 2024-03-14T09:04:01.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.678+0000 7f4c257fa700 1 -- 172.21.15.154:0/2678028819 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289476 (secure 0 0 0) 0x7f4c1c011b00 con 0x7f4c0c06a830 2024-03-14T09:04:01.680 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 msgr2=0x7f4c0c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 0x7f4c0c06ccf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 msgr2=0x7f4c2813f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000cb70 tx=0x7f4c2000cf30 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 shutdown_connections 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4c0c06a830 0x7f4c0c06ccf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c280c1d90 0x7f4c2813f140 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4c2806d3c0 0x7f4c2814c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 --2- 172.21.15.154:0/2678028819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c280c1830 0x7f4c280a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 >> 172.21.15.154:0/2678028819 conn(0x7f4c2806a770 msgr2=0x7f4c281459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:01.684 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 shutdown_connections 2024-03-14T09:04:01.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.681+0000 7f4c2f187700 1 -- 172.21.15.154:0/2678028819 wait complete. 2024-03-14T09:04:01.685 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:01.694 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-14T09:04:01.556386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223996,"kb_used_data":7792,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750724,"statfs":{"total":773094113280,"available":772864741376,"internally_reserved":0,"allocated":7979008,"data_stored":4116771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.811327"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188980+0000","last_change":"2024-03-14T09:03:57.183867+0000","last_active":"2024-03-14T09:03:57.188980+0000","last_peered":"2024-03-14T09:03:57.188980+0000","last_clean":"2024-03-14T09:03:57.188980+0000","last_became_active":"2024-03-14T09:03:57.183371+0000","last_became_peered":"2024-03-14T09:03:57.183371+0000","last_unstale":"2024-03-14T09:03:57.188980+0000","last_undegraded":"2024-03-14T09:03:57.188980+0000","last_fullsized":"2024-03-14T09:03:57.188980+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192502+0000","last_change":"2024-03-14T09:03:57.187973+0000","last_active":"2024-03-14T09:03:57.192502+0000","last_peered":"2024-03-14T09:03:57.192502+0000","last_clean":"2024-03-14T09:03:57.192502+0000","last_became_active":"2024-03-14T09:03:57.187737+0000","last_became_peered":"2024-03-14T09:03:57.187737+0000","last_unstale":"2024-03-14T09:03:57.192502+0000","last_undegraded":"2024-03-14T09:03:57.192502+0000","last_fullsized":"2024-03-14T09:03:57.192502+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:39.248247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184397+0000","last_change":"2024-03-14T09:03:57.180121+0000","last_active":"2024-03-14T09:03:57.184397+0000","last_peered":"2024-03-14T09:03:57.184397+0000","last_clean":"2024-03-14T09:03:57.184397+0000","last_became_active":"2024-03-14T09:03:57.179465+0000","last_became_peered":"2024-03-14T09:03:57.179465+0000","last_unstale":"2024-03-14T09:03:57.184397+0000","last_undegraded":"2024-03-14T09:03:57.184397+0000","last_fullsized":"2024-03-14T09:03:57.184397+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194063+0000","last_change":"2024-03-14T09:03:57.191808+0000","last_active":"2024-03-14T09:03:57.194063+0000","last_peered":"2024-03-14T09:03:57.194063+0000","last_clean":"2024-03-14T09:03:57.194063+0000","last_became_active":"2024-03-14T09:03:57.187786+0000","last_became_peered":"2024-03-14T09:03:57.187786+0000","last_unstale":"2024-03-14T09:03:57.194063+0000","last_undegraded":"2024-03-14T09:03:57.194063+0000","last_fullsized":"2024-03-14T09:03:57.194063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:23.462424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190948+0000","last_change":"2024-03-14T09:03:57.186814+0000","last_active":"2024-03-14T09:03:57.190948+0000","last_peered":"2024-03-14T09:03:57.190948+0000","last_clean":"2024-03-14T09:03:57.190948+0000","last_became_active":"2024-03-14T09:03:57.186480+0000","last_became_peered":"2024-03-14T09:03:57.186480+0000","last_unstale":"2024-03-14T09:03:57.190948+0000","last_undegraded":"2024-03-14T09:03:57.190948+0000","last_fullsized":"2024-03-14T09:03:57.190948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187781+0000","last_change":"2024-03-14T09:03:57.179120+0000","last_active":"2024-03-14T09:03:57.187781+0000","last_peered":"2024-03-14T09:03:57.187781+0000","last_clean":"2024-03-14T09:03:57.187781+0000","last_became_active":"2024-03-14T09:03:57.178824+0000","last_became_peered":"2024-03-14T09:03:57.178824+0000","last_unstale":"2024-03-14T09:03:57.187781+0000","last_undegraded":"2024-03-14T09:03:57.187781+0000","last_fullsized":"2024-03-14T09:03:57.187781+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186258+0000","last_change":"2024-03-14T09:03:57.176658+0000","last_active":"2024-03-14T09:03:57.186258+0000","last_peered":"2024-03-14T09:03:57.186258+0000","last_clean":"2024-03-14T09:03:57.186258+0000","last_became_active":"2024-03-14T09:03:57.176352+0000","last_became_peered":"2024-03-14T09:03:57.176352+0000","last_unstale":"2024-03-14T09:03:57.186258+0000","last_undegraded":"2024-03-14T09:03:57.186258+0000","last_fullsized":"2024-03-14T09:03:57.186258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:51:09.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190195+0000","last_change":"2024-03-14T09:03:57.186734+0000","last_active":"2024-03-14T09:03:57.190195+0000","last_peered":"2024-03-14T09:03:57.190195+0000","last_clean":"2024-03-14T09:03:57.190195+0000","last_became_active":"2024-03-14T09:03:57.186455+0000","last_became_peered":"2024-03-14T09:03:57.186455+0000","last_unstale":"2024-03-14T09:03:57.190195+0000","last_undegraded":"2024-03-14T09:03:57.190195+0000","last_fullsized":"2024-03-14T09:03:57.190195+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190164+0000","last_change":"2024-03-14T09:03:57.186757+0000","last_active":"2024-03-14T09:03:57.190164+0000","last_peered":"2024-03-14T09:03:57.190164+0000","last_clean":"2024-03-14T09:03:57.190164+0000","last_became_active":"2024-03-14T09:03:57.186486+0000","last_became_peered":"2024-03-14T09:03:57.186486+0000","last_unstale":"2024-03-14T09:03:57.190164+0000","last_undegraded":"2024-03-14T09:03:57.190164+0000","last_fullsized":"2024-03-14T09:03:57.190164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184362+0000","last_change":"2024-03-14T09:03:57.180488+0000","last_active":"2024-03-14T09:03:57.184362+0000","last_peered":"2024-03-14T09:03:57.184362+0000","last_clean":"2024-03-14T09:03:57.184362+0000","last_became_active":"2024-03-14T09:03:57.180202+0000","last_became_peered":"2024-03-14T09:03:57.180202+0000","last_unstale":"2024-03-14T09:03:57.184362+0000","last_undegraded":"2024-03-14T09:03:57.184362+0000","last_fullsized":"2024-03-14T09:03:57.184362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192354+0000","last_change":"2024-03-14T09:03:57.185725+0000","last_active":"2024-03-14T09:03:57.192354+0000","last_peered":"2024-03-14T09:03:57.192354+0000","last_clean":"2024-03-14T09:03:57.192354+0000","last_became_active":"2024-03-14T09:03:57.185457+0000","last_became_peered":"2024-03-14T09:03:57.185457+0000","last_unstale":"2024-03-14T09:03:57.192354+0000","last_undegraded":"2024-03-14T09:03:57.192354+0000","last_fullsized":"2024-03-14T09:03:57.192354+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:26.439102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188935+0000","last_change":"2024-03-14T09:03:57.182863+0000","last_active":"2024-03-14T09:03:57.188935+0000","last_peered":"2024-03-14T09:03:57.188935+0000","last_clean":"2024-03-14T09:03:57.188935+0000","last_became_active":"2024-03-14T09:03:57.181034+0000","last_became_peered":"2024-03-14T09:03:57.181034+0000","last_unstale":"2024-03-14T09:03:57.188935+0000","last_undegraded":"2024-03-14T09:03:57.188935+0000","last_fullsized":"2024-03-14T09:03:57.188935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182507+0000","last_change":"2024-03-14T09:03:57.177255+0000","last_active":"2024-03-14T09:03:57.182507+0000","last_peered":"2024-03-14T09:03:57.182507+0000","last_clean":"2024-03-14T09:03:57.182507+0000","last_became_active":"2024-03-14T09:03:57.176972+0000","last_became_peered":"2024-03-14T09:03:57.176972+0000","last_unstale":"2024-03-14T09:03:57.182507+0000","last_undegraded":"2024-03-14T09:03:57.182507+0000","last_fullsized":"2024-03-14T09:03:57.182507+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:41.881683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194078+0000","last_change":"2024-03-14T09:03:57.191926+0000","last_active":"2024-03-14T09:03:57.194078+0000","last_peered":"2024-03-14T09:03:57.194078+0000","last_clean":"2024-03-14T09:03:57.194078+0000","last_became_active":"2024-03-14T09:03:57.191416+0000","last_became_peered":"2024-03-14T09:03:57.191416+0000","last_unstale":"2024-03-14T09:03:57.194078+0000","last_undegraded":"2024-03-14T09:03:57.194078+0000","last_fullsized":"2024-03-14T09:03:57.194078+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:25.082584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180617+0000","last_change":"2024-03-14T09:03:57.175406+0000","last_active":"2024-03-14T09:03:57.180617+0000","last_peered":"2024-03-14T09:03:57.180617+0000","last_clean":"2024-03-14T09:03:57.180617+0000","last_became_active":"2024-03-14T09:03:57.175155+0000","last_became_peered":"2024-03-14T09:03:57.175155+0000","last_unstale":"2024-03-14T09:03:57.180617+0000","last_undegraded":"2024-03-14T09:03:57.180617+0000","last_fullsized":"2024-03-14T09:03:57.180617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:19.270920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190025+0000","last_change":"2024-03-14T09:03:57.181942+0000","last_active":"2024-03-14T09:03:57.190025+0000","last_peered":"2024-03-14T09:03:57.190025+0000","last_clean":"2024-03-14T09:03:57.190025+0000","last_became_active":"2024-03-14T09:03:57.181669+0000","last_became_peered":"2024-03-14T09:03:57.181669+0000","last_unstale":"2024-03-14T09:03:57.190025+0000","last_undegraded":"2024-03-14T09:03:57.190025+0000","last_fullsized":"2024-03-14T09:03:57.190025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182572+0000","last_change":"2024-03-14T09:03:57.177217+0000","last_active":"2024-03-14T09:03:57.182572+0000","last_peered":"2024-03-14T09:03:57.182572+0000","last_clean":"2024-03-14T09:03:57.182572+0000","last_became_active":"2024-03-14T09:03:57.176941+0000","last_became_peered":"2024-03-14T09:03:57.176941+0000","last_unstale":"2024-03-14T09:03:57.182572+0000","last_undegraded":"2024-03-14T09:03:57.182572+0000","last_fullsized":"2024-03-14T09:03:57.182572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186217+0000","last_change":"2024-03-14T09:03:57.176770+0000","last_active":"2024-03-14T09:03:57.186217+0000","last_peered":"2024-03-14T09:03:57.186217+0000","last_clean":"2024-03-14T09:03:57.186217+0000","last_became_active":"2024-03-14T09:03:57.176496+0000","last_became_peered":"2024-03-14T09:03:57.176496+0000","last_unstale":"2024-03-14T09:03:57.186217+0000","last_undegraded":"2024-03-14T09:03:57.186217+0000","last_fullsized":"2024-03-14T09:03:57.186217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:32.959955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193128+0000","last_change":"2024-03-14T09:03:57.188745+0000","last_active":"2024-03-14T09:03:57.193128+0000","last_peered":"2024-03-14T09:03:57.193128+0000","last_clean":"2024-03-14T09:03:57.193128+0000","last_became_active":"2024-03-14T09:03:57.187830+0000","last_became_peered":"2024-03-14T09:03:57.187830+0000","last_unstale":"2024-03-14T09:03:57.193128+0000","last_undegraded":"2024-03-14T09:03:57.193128+0000","last_fullsized":"2024-03-14T09:03:57.193128+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192997+0000","last_change":"2024-03-14T09:03:57.186877+0000","last_active":"2024-03-14T09:03:57.192997+0000","last_peered":"2024-03-14T09:03:57.192997+0000","last_clean":"2024-03-14T09:03:57.192997+0000","last_became_active":"2024-03-14T09:03:57.186581+0000","last_became_peered":"2024-03-14T09:03:57.186581+0000","last_unstale":"2024-03-14T09:03:57.192997+0000","last_undegraded":"2024-03-14T09:03:57.192997+0000","last_fullsized":"2024-03-14T09:03:57.192997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:51.324739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193011+0000","last_change":"2024-03-14T09:03:57.187026+0000","last_active":"2024-03-14T09:03:57.193011+0000","last_peered":"2024-03-14T09:03:57.193011+0000","last_clean":"2024-03-14T09:03:57.193011+0000","last_became_active":"2024-03-14T09:03:57.186774+0000","last_became_peered":"2024-03-14T09:03:57.186774+0000","last_unstale":"2024-03-14T09:03:57.193011+0000","last_undegraded":"2024-03-14T09:03:57.193011+0000","last_fullsized":"2024-03-14T09:03:57.193011+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:55:25.363869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192570+0000","last_change":"2024-03-14T09:03:57.189350+0000","last_active":"2024-03-14T09:03:57.192570+0000","last_peered":"2024-03-14T09:03:57.192570+0000","last_clean":"2024-03-14T09:03:57.192570+0000","last_became_active":"2024-03-14T09:03:57.189109+0000","last_became_peered":"2024-03-14T09:03:57.189109+0000","last_unstale":"2024-03-14T09:03:57.192570+0000","last_undegraded":"2024-03-14T09:03:57.192570+0000","last_fullsized":"2024-03-14T09:03:57.192570+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:18.878301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190524+0000","last_change":"2024-03-14T09:03:57.186278+0000","last_active":"2024-03-14T09:03:57.190524+0000","last_peered":"2024-03-14T09:03:57.190524+0000","last_clean":"2024-03-14T09:03:57.190524+0000","last_became_active":"2024-03-14T09:03:57.186014+0000","last_became_peered":"2024-03-14T09:03:57.186014+0000","last_unstale":"2024-03-14T09:03:57.190524+0000","last_undegraded":"2024-03-14T09:03:57.190524+0000","last_fullsized":"2024-03-14T09:03:57.190524+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193008+0000","last_change":"2024-03-14T09:03:57.186749+0000","last_active":"2024-03-14T09:03:57.193008+0000","last_peered":"2024-03-14T09:03:57.193008+0000","last_clean":"2024-03-14T09:03:57.193008+0000","last_became_active":"2024-03-14T09:03:57.186502+0000","last_became_peered":"2024-03-14T09:03:57.186502+0000","last_unstale":"2024-03-14T09:03:57.193008+0000","last_undegraded":"2024-03-14T09:03:57.193008+0000","last_fullsized":"2024-03-14T09:03:57.193008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:10.996899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192643+0000","last_change":"2024-03-14T09:03:57.185209+0000","last_active":"2024-03-14T09:03:57.192643+0000","last_peered":"2024-03-14T09:03:57.192643+0000","last_clean":"2024-03-14T09:03:57.192643+0000","last_became_active":"2024-03-14T09:03:57.184942+0000","last_became_peered":"2024-03-14T09:03:57.184942+0000","last_unstale":"2024-03-14T09:03:57.192643+0000","last_undegraded":"2024-03-14T09:03:57.192643+0000","last_fullsized":"2024-03-14T09:03:57.192643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:37.052225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190868+0000","last_change":"2024-03-14T09:03:57.189881+0000","last_active":"2024-03-14T09:03:57.190868+0000","last_peered":"2024-03-14T09:03:57.190868+0000","last_clean":"2024-03-14T09:03:57.190868+0000","last_became_active":"2024-03-14T09:03:57.189632+0000","last_became_peered":"2024-03-14T09:03:57.189632+0000","last_unstale":"2024-03-14T09:03:57.190868+0000","last_undegraded":"2024-03-14T09:03:57.190868+0000","last_fullsized":"2024-03-14T09:03:57.190868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.170931+0000","last_change":"2024-03-14T09:03:57.168978+0000","last_active":"2024-03-14T09:03:57.170931+0000","last_peered":"2024-03-14T09:03:57.170931+0000","last_clean":"2024-03-14T09:03:57.170931+0000","last_became_active":"2024-03-14T09:03:57.167802+0000","last_became_peered":"2024-03-14T09:03:57.167802+0000","last_unstale":"2024-03-14T09:03:57.170931+0000","last_undegraded":"2024-03-14T09:03:57.170931+0000","last_fullsized":"2024-03-14T09:03:57.170931+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188955+0000","last_change":"2024-03-14T09:03:57.183439+0000","last_active":"2024-03-14T09:03:57.188955+0000","last_peered":"2024-03-14T09:03:57.188955+0000","last_clean":"2024-03-14T09:03:57.188955+0000","last_became_active":"2024-03-14T09:03:57.183150+0000","last_became_peered":"2024-03-14T09:03:57.183150+0000","last_unstale":"2024-03-14T09:03:57.188955+0000","last_undegraded":"2024-03-14T09:03:57.188955+0000","last_fullsized":"2024-03-14T09:03:57.188955+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192237+0000","last_change":"2024-03-14T09:03:57.184782+0000","last_active":"2024-03-14T09:03:57.192237+0000","last_peered":"2024-03-14T09:03:57.192237+0000","last_clean":"2024-03-14T09:03:57.192237+0000","last_became_active":"2024-03-14T09:03:57.184507+0000","last_became_peered":"2024-03-14T09:03:57.184507+0000","last_unstale":"2024-03-14T09:03:57.192237+0000","last_undegraded":"2024-03-14T09:03:57.192237+0000","last_fullsized":"2024-03-14T09:03:57.192237+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:48.425158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192552+0000","last_change":"2024-03-14T09:03:57.185432+0000","last_active":"2024-03-14T09:03:57.192552+0000","last_peered":"2024-03-14T09:03:57.192552+0000","last_clean":"2024-03-14T09:03:57.192552+0000","last_became_active":"2024-03-14T09:03:57.184879+0000","last_became_peered":"2024-03-14T09:03:57.184879+0000","last_unstale":"2024-03-14T09:03:57.192552+0000","last_undegraded":"2024-03-14T09:03:57.192552+0000","last_fullsized":"2024-03-14T09:03:57.192552+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:22.429275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190892+0000","last_change":"2024-03-14T09:03:57.186461+0000","last_active":"2024-03-14T09:03:57.190892+0000","last_peered":"2024-03-14T09:03:57.190892+0000","last_clean":"2024-03-14T09:03:57.190892+0000","last_became_active":"2024-03-14T09:03:57.186176+0000","last_became_peered":"2024-03-14T09:03:57.186176+0000","last_unstale":"2024-03-14T09:03:57.190892+0000","last_undegraded":"2024-03-14T09:03:57.190892+0000","last_fullsized":"2024-03-14T09:03:57.190892+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190459+0000","last_change":"2024-03-14T09:03:57.185583+0000","last_active":"2024-03-14T09:03:57.190459+0000","last_peered":"2024-03-14T09:03:57.190459+0000","last_clean":"2024-03-14T09:03:57.190459+0000","last_became_active":"2024-03-14T09:03:57.185292+0000","last_became_peered":"2024-03-14T09:03:57.185292+0000","last_unstale":"2024-03-14T09:03:57.190459+0000","last_undegraded":"2024-03-14T09:03:57.190459+0000","last_fullsized":"2024-03-14T09:03:57.190459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190124+0000","last_change":"2024-03-14T09:03:57.182704+0000","last_active":"2024-03-14T09:03:57.190124+0000","last_peered":"2024-03-14T09:03:57.190124+0000","last_clean":"2024-03-14T09:03:57.190124+0000","last_became_active":"2024-03-14T09:03:57.182438+0000","last_became_peered":"2024-03-14T09:03:57.182438+0000","last_unstale":"2024-03-14T09:03:57.190124+0000","last_undegraded":"2024-03-14T09:03:57.190124+0000","last_fullsized":"2024-03-14T09:03:57.190124+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"69'4","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191382+0000","last_change":"2024-03-14T09:03:56.771973+0000","last_active":"2024-03-14T09:03:57.191382+0000","last_peered":"2024-03-14T09:03:57.191382+0000","last_clean":"2024-03-14T09:03:57.191382+0000","last_became_active":"2024-03-14T09:03:56.771714+0000","last_became_peered":"2024-03-14T09:03:56.771714+0000","last_unstale":"2024-03-14T09:03:57.191382+0000","last_undegraded":"2024-03-14T09:03:57.191382+0000","last_fullsized":"2024-03-14T09:03:57.191382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:17.873268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190487+0000","last_change":"2024-03-14T09:03:57.185961+0000","last_active":"2024-03-14T09:03:57.190487+0000","last_peered":"2024-03-14T09:03:57.190487+0000","last_clean":"2024-03-14T09:03:57.190487+0000","last_became_active":"2024-03-14T09:03:57.185658+0000","last_became_peered":"2024-03-14T09:03:57.185658+0000","last_unstale":"2024-03-14T09:03:57.190487+0000","last_undegraded":"2024-03-14T09:03:57.190487+0000","last_fullsized":"2024-03-14T09:03:57.190487+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182170+0000","last_change":"2024-03-14T09:03:56.769740+0000","last_active":"2024-03-14T09:03:57.182170+0000","last_peered":"2024-03-14T09:03:57.182170+0000","last_clean":"2024-03-14T09:03:57.182170+0000","last_became_active":"2024-03-14T09:03:56.769429+0000","last_became_peered":"2024-03-14T09:03:56.769429+0000","last_unstale":"2024-03-14T09:03:57.182170+0000","last_undegraded":"2024-03-14T09:03:57.182170+0000","last_fullsized":"2024-03-14T09:03:57.182170+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190683+0000","last_change":"2024-03-14T09:03:57.188083+0000","last_active":"2024-03-14T09:03:57.190683+0000","last_peered":"2024-03-14T09:03:57.190683+0000","last_clean":"2024-03-14T09:03:57.190683+0000","last_became_active":"2024-03-14T09:03:57.187832+0000","last_became_peered":"2024-03-14T09:03:57.187832+0000","last_unstale":"2024-03-14T09:03:57.190683+0000","last_undegraded":"2024-03-14T09:03:57.190683+0000","last_fullsized":"2024-03-14T09:03:57.190683+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192826+0000","last_change":"2024-03-14T09:03:56.771927+0000","last_active":"2024-03-14T09:03:57.192826+0000","last_peered":"2024-03-14T09:03:57.192826+0000","last_clean":"2024-03-14T09:03:57.192826+0000","last_became_active":"2024-03-14T09:03:56.771684+0000","last_became_peered":"2024-03-14T09:03:56.771684+0000","last_unstale":"2024-03-14T09:03:57.192826+0000","last_undegraded":"2024-03-14T09:03:57.192826+0000","last_fullsized":"2024-03-14T09:03:57.192826+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184425+0000","last_change":"2024-03-14T09:03:57.180285+0000","last_active":"2024-03-14T09:03:57.184425+0000","last_peered":"2024-03-14T09:03:57.184425+0000","last_clean":"2024-03-14T09:03:57.184425+0000","last_became_active":"2024-03-14T09:03:57.179838+0000","last_became_peered":"2024-03-14T09:03:57.179838+0000","last_unstale":"2024-03-14T09:03:57.184425+0000","last_undegraded":"2024-03-14T09:03:57.184425+0000","last_fullsized":"2024-03-14T09:03:57.184425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190644+0000","last_change":"2024-03-14T09:03:56.772021+0000","last_active":"2024-03-14T09:03:57.190644+0000","last_peered":"2024-03-14T09:03:57.190644+0000","last_clean":"2024-03-14T09:03:57.190644+0000","last_became_active":"2024-03-14T09:03:56.771764+0000","last_became_peered":"2024-03-14T09:03:56.771764+0000","last_unstale":"2024-03-14T09:03:57.190644+0000","last_undegraded":"2024-03-14T09:03:57.190644+0000","last_fullsized":"2024-03-14T09:03:57.190644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:30.727586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192528+0000","last_change":"2024-03-14T09:03:57.185267+0000","last_active":"2024-03-14T09:03:57.192528+0000","last_peered":"2024-03-14T09:03:57.192528+0000","last_clean":"2024-03-14T09:03:57.192528+0000","last_became_active":"2024-03-14T09:03:57.184614+0000","last_became_peered":"2024-03-14T09:03:57.184614+0000","last_unstale":"2024-03-14T09:03:57.192528+0000","last_undegraded":"2024-03-14T09:03:57.192528+0000","last_fullsized":"2024-03-14T09:03:57.192528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:25.314389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186165+0000","last_change":"2024-03-14T09:03:56.770949+0000","last_active":"2024-03-14T09:03:57.186165+0000","last_peered":"2024-03-14T09:03:57.186165+0000","last_clean":"2024-03-14T09:03:57.186165+0000","last_became_active":"2024-03-14T09:03:56.770678+0000","last_became_peered":"2024-03-14T09:03:56.770678+0000","last_unstale":"2024-03-14T09:03:57.186165+0000","last_undegraded":"2024-03-14T09:03:57.186165+0000","last_fullsized":"2024-03-14T09:03:57.186165+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182189+0000","last_change":"2024-03-14T09:03:57.178070+0000","last_active":"2024-03-14T09:03:57.182189+0000","last_peered":"2024-03-14T09:03:57.182189+0000","last_clean":"2024-03-14T09:03:57.182189+0000","last_became_active":"2024-03-14T09:03:57.177621+0000","last_became_peered":"2024-03-14T09:03:57.177621+0000","last_unstale":"2024-03-14T09:03:57.182189+0000","last_undegraded":"2024-03-14T09:03:57.182189+0000","last_fullsized":"2024-03-14T09:03:57.182189+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:55.259556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.181107+0000","last_change":"2024-03-14T09:03:56.768242+0000","last_active":"2024-03-14T09:03:57.181107+0000","last_peered":"2024-03-14T09:03:57.181107+0000","last_clean":"2024-03-14T09:03:57.181107+0000","last_became_active":"2024-03-14T09:03:56.767960+0000","last_became_peered":"2024-03-14T09:03:56.767960+0000","last_unstale":"2024-03-14T09:03:57.181107+0000","last_undegraded":"2024-03-14T09:03:57.181107+0000","last_fullsized":"2024-03-14T09:03:57.181107+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190804+0000","last_change":"2024-03-14T09:03:57.187305+0000","last_active":"2024-03-14T09:03:57.190804+0000","last_peered":"2024-03-14T09:03:57.190804+0000","last_clean":"2024-03-14T09:03:57.190804+0000","last_became_active":"2024-03-14T09:03:57.186346+0000","last_became_peered":"2024-03-14T09:03:57.186346+0000","last_unstale":"2024-03-14T09:03:57.190804+0000","last_undegraded":"2024-03-14T09:03:57.190804+0000","last_fullsized":"2024-03-14T09:03:57.190804+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:04.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.178742+0000","last_change":"2024-03-14T09:03:56.773036+0000","last_active":"2024-03-14T09:03:57.178742+0000","last_peered":"2024-03-14T09:03:57.178742+0000","last_clean":"2024-03-14T09:03:57.178742+0000","last_became_active":"2024-03-14T09:03:56.772740+0000","last_became_peered":"2024-03-14T09:03:56.772740+0000","last_unstale":"2024-03-14T09:03:57.178742+0000","last_undegraded":"2024-03-14T09:03:57.178742+0000","last_fullsized":"2024-03-14T09:03:57.178742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:44.982623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180219+0000","last_change":"2024-03-14T09:03:57.172128+0000","last_active":"2024-03-14T09:03:57.180219+0000","last_peered":"2024-03-14T09:03:57.180219+0000","last_clean":"2024-03-14T09:03:57.180219+0000","last_became_active":"2024-03-14T09:03:57.171877+0000","last_became_peered":"2024-03-14T09:03:57.171877+0000","last_unstale":"2024-03-14T09:03:57.180219+0000","last_undegraded":"2024-03-14T09:03:57.180219+0000","last_fullsized":"2024-03-14T09:03:57.180219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:25.355300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.171531+0000","last_change":"2024-03-14T09:03:56.769464+0000","last_active":"2024-03-14T09:03:57.171531+0000","last_peered":"2024-03-14T09:03:57.171531+0000","last_clean":"2024-03-14T09:03:57.171531+0000","last_became_active":"2024-03-14T09:03:56.769231+0000","last_became_peered":"2024-03-14T09:03:56.769231+0000","last_unstale":"2024-03-14T09:03:57.171531+0000","last_undegraded":"2024-03-14T09:03:57.171531+0000","last_fullsized":"2024-03-14T09:03:57.171531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:50.319706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191239+0000","last_change":"2024-03-14T09:03:57.188282+0000","last_active":"2024-03-14T09:03:57.191239+0000","last_peered":"2024-03-14T09:03:57.191239+0000","last_clean":"2024-03-14T09:03:57.191239+0000","last_became_active":"2024-03-14T09:03:57.188018+0000","last_became_peered":"2024-03-14T09:03:57.188018+0000","last_unstale":"2024-03-14T09:03:57.191239+0000","last_undegraded":"2024-03-14T09:03:57.191239+0000","last_fullsized":"2024-03-14T09:03:57.191239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:22.289306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.174983+0000","last_change":"2024-03-14T09:03:56.768124+0000","last_active":"2024-03-14T09:03:57.174983+0000","last_peered":"2024-03-14T09:03:57.174983+0000","last_clean":"2024-03-14T09:03:57.174983+0000","last_became_active":"2024-03-14T09:03:56.766669+0000","last_became_peered":"2024-03-14T09:03:56.766669+0000","last_unstale":"2024-03-14T09:03:57.174983+0000","last_undegraded":"2024-03-14T09:03:57.174983+0000","last_fullsized":"2024-03-14T09:03:57.174983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:11.739150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187522+0000","last_change":"2024-03-14T09:03:57.179078+0000","last_active":"2024-03-14T09:03:57.187522+0000","last_peered":"2024-03-14T09:03:57.187522+0000","last_clean":"2024-03-14T09:03:57.187522+0000","last_became_active":"2024-03-14T09:03:57.178782+0000","last_became_peered":"2024-03-14T09:03:57.178782+0000","last_unstale":"2024-03-14T09:03:57.187522+0000","last_undegraded":"2024-03-14T09:03:57.187522+0000","last_fullsized":"2024-03-14T09:03:57.187522+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.173970+0000","last_change":"2024-03-14T09:03:56.769911+0000","last_active":"2024-03-14T09:03:57.173970+0000","last_peered":"2024-03-14T09:03:57.173970+0000","last_clean":"2024-03-14T09:03:57.173970+0000","last_became_active":"2024-03-14T09:03:56.769598+0000","last_became_peered":"2024-03-14T09:03:56.769598+0000","last_unstale":"2024-03-14T09:03:57.173970+0000","last_undegraded":"2024-03-14T09:03:57.173970+0000","last_fullsized":"2024-03-14T09:03:57.173970+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192678+0000","last_change":"2024-03-14T09:03:57.184589+0000","last_active":"2024-03-14T09:03:57.192678+0000","last_peered":"2024-03-14T09:03:57.192678+0000","last_clean":"2024-03-14T09:03:57.192678+0000","last_became_active":"2024-03-14T09:03:57.184303+0000","last_became_peered":"2024-03-14T09:03:57.184303+0000","last_unstale":"2024-03-14T09:03:57.192678+0000","last_undegraded":"2024-03-14T09:03:57.192678+0000","last_fullsized":"2024-03-14T09:03:57.192678+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"69'3","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187997+0000","last_change":"2024-03-14T09:03:56.766306+0000","last_active":"2024-03-14T09:03:57.187997+0000","last_peered":"2024-03-14T09:03:57.187997+0000","last_clean":"2024-03-14T09:03:57.187997+0000","last_became_active":"2024-03-14T09:03:56.766045+0000","last_became_peered":"2024-03-14T09:03:56.766045+0000","last_unstale":"2024-03-14T09:03:57.187997+0000","last_undegraded":"2024-03-14T09:03:57.187997+0000","last_fullsized":"2024-03-14T09:03:57.187997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191068+0000","last_change":"2024-03-14T09:03:57.189394+0000","last_active":"2024-03-14T09:03:57.191068+0000","last_peered":"2024-03-14T09:03:57.191068+0000","last_clean":"2024-03-14T09:03:57.191068+0000","last_became_active":"2024-03-14T09:03:57.189162+0000","last_became_peered":"2024-03-14T09:03:57.189162+0000","last_unstale":"2024-03-14T09:03:57.191068+0000","last_undegraded":"2024-03-14T09:03:57.191068+0000","last_fullsized":"2024-03-14T09:03:57.191068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"69'6","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.196192+0000","last_change":"2024-03-14T09:03:56.769295+0000","last_active":"2024-03-14T09:03:57.196192+0000","last_peered":"2024-03-14T09:03:57.196192+0000","last_clean":"2024-03-14T09:03:57.196192+0000","last_became_active":"2024-03-14T09:03:56.768952+0000","last_became_peered":"2024-03-14T09:03:56.768952+0000","last_unstale":"2024-03-14T09:03:57.196192+0000","last_undegraded":"2024-03-14T09:03:57.196192+0000","last_fullsized":"2024-03-14T09:03:57.196192+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.193087+0000","last_change":"2024-03-14T09:03:57.188359+0000","last_active":"2024-03-14T09:03:57.193087+0000","last_peered":"2024-03-14T09:03:57.193087+0000","last_clean":"2024-03-14T09:03:57.193087+0000","last_became_active":"2024-03-14T09:03:57.188093+0000","last_became_peered":"2024-03-14T09:03:57.188093+0000","last_unstale":"2024-03-14T09:03:57.193087+0000","last_undegraded":"2024-03-14T09:03:57.193087+0000","last_fullsized":"2024-03-14T09:03:57.193087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191864+0000","last_change":"2024-03-14T09:03:56.769589+0000","last_active":"2024-03-14T09:03:57.191864+0000","last_peered":"2024-03-14T09:03:57.191864+0000","last_clean":"2024-03-14T09:03:57.191864+0000","last_became_active":"2024-03-14T09:03:56.767656+0000","last_became_peered":"2024-03-14T09:03:56.767656+0000","last_unstale":"2024-03-14T09:03:57.191864+0000","last_undegraded":"2024-03-14T09:03:57.191864+0000","last_fullsized":"2024-03-14T09:03:57.191864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:31.954923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190788+0000","last_change":"2024-03-14T09:03:57.187466+0000","last_active":"2024-03-14T09:03:57.190788+0000","last_peered":"2024-03-14T09:03:57.190788+0000","last_clean":"2024-03-14T09:03:57.190788+0000","last_became_active":"2024-03-14T09:03:57.186590+0000","last_became_peered":"2024-03-14T09:03:57.186590+0000","last_unstale":"2024-03-14T09:03:57.190788+0000","last_undegraded":"2024-03-14T09:03:57.190788+0000","last_fullsized":"2024-03-14T09:03:57.190788+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:04.118845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.164091+0000","last_change":"2024-03-14T09:03:56.762545+0000","last_active":"2024-03-14T09:03:57.164091+0000","last_peered":"2024-03-14T09:03:57.164091+0000","last_clean":"2024-03-14T09:03:57.164091+0000","last_became_active":"2024-03-14T09:03:56.762292+0000","last_became_peered":"2024-03-14T09:03:56.762292+0000","last_unstale":"2024-03-14T09:03:57.164091+0000","last_undegraded":"2024-03-14T09:03:57.164091+0000","last_fullsized":"2024-03-14T09:03:57.164091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:03.013048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192002+0000","last_change":"2024-03-14T09:03:57.184740+0000","last_active":"2024-03-14T09:03:57.192002+0000","last_peered":"2024-03-14T09:03:57.192002+0000","last_clean":"2024-03-14T09:03:57.192002+0000","last_became_active":"2024-03-14T09:03:57.184407+0000","last_became_peered":"2024-03-14T09:03:57.184407+0000","last_unstale":"2024-03-14T09:03:57.192002+0000","last_undegraded":"2024-03-14T09:03:57.192002+0000","last_fullsized":"2024-03-14T09:03:57.192002+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:12.744182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.170770+0000","last_change":"2024-03-14T09:03:56.773362+0000","last_active":"2024-03-14T09:03:57.170770+0000","last_peered":"2024-03-14T09:03:57.170770+0000","last_clean":"2024-03-14T09:03:57.170770+0000","last_became_active":"2024-03-14T09:03:56.773012+0000","last_became_peered":"2024-03-14T09:03:56.773012+0000","last_unstale":"2024-03-14T09:03:57.170770+0000","last_undegraded":"2024-03-14T09:03:57.170770+0000","last_fullsized":"2024-03-14T09:03:57.170770+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:38.243214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192499+0000","last_change":"2024-03-14T09:03:57.187459+0000","last_active":"2024-03-14T09:03:57.192499+0000","last_peered":"2024-03-14T09:03:57.192499+0000","last_clean":"2024-03-14T09:03:57.192499+0000","last_became_active":"2024-03-14T09:03:57.187221+0000","last_became_peered":"2024-03-14T09:03:57.187221+0000","last_unstale":"2024-03-14T09:03:57.192499+0000","last_undegraded":"2024-03-14T09:03:57.192499+0000","last_fullsized":"2024-03-14T09:03:57.192499+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:42.577661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"69'8","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191802+0000","last_change":"2024-03-14T09:03:56.770808+0000","last_active":"2024-03-14T09:03:57.191802+0000","last_peered":"2024-03-14T09:03:57.191802+0000","last_clean":"2024-03-14T09:03:57.191802+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:03:57.191802+0000","last_undegraded":"2024-03-14T09:03:57.191802+0000","last_fullsized":"2024-03-14T09:03:57.191802+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:29.358695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190064+0000","last_change":"2024-03-14T09:03:57.182905+0000","last_active":"2024-03-14T09:03:57.190064+0000","last_peered":"2024-03-14T09:03:57.190064+0000","last_clean":"2024-03-14T09:03:57.190064+0000","last_became_active":"2024-03-14T09:03:57.182631+0000","last_became_peered":"2024-03-14T09:03:57.182631+0000","last_unstale":"2024-03-14T09:03:57.190064+0000","last_undegraded":"2024-03-14T09:03:57.190064+0000","last_fullsized":"2024-03-14T09:03:57.190064+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:05.353470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190608+0000","last_change":"2024-03-14T09:03:56.770834+0000","last_active":"2024-03-14T09:03:57.190608+0000","last_peered":"2024-03-14T09:03:57.190608+0000","last_clean":"2024-03-14T09:03:57.190608+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:03:57.190608+0000","last_undegraded":"2024-03-14T09:03:57.190608+0000","last_fullsized":"2024-03-14T09:03:57.190608+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.183557+0000","last_change":"2024-03-14T09:03:57.180017+0000","last_active":"2024-03-14T09:03:57.183557+0000","last_peered":"2024-03-14T09:03:57.183557+0000","last_clean":"2024-03-14T09:03:57.183557+0000","last_became_active":"2024-03-14T09:03:57.179589+0000","last_became_peered":"2024-03-14T09:03:57.179589+0000","last_unstale":"2024-03-14T09:03:57.183557+0000","last_undegraded":"2024-03-14T09:03:57.183557+0000","last_fullsized":"2024-03-14T09:03:57.183557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190661+0000","last_change":"2024-03-14T09:03:56.769821+0000","last_active":"2024-03-14T09:03:57.190661+0000","last_peered":"2024-03-14T09:03:57.190661+0000","last_clean":"2024-03-14T09:03:57.190661+0000","last_became_active":"2024-03-14T09:03:56.769597+0000","last_became_peered":"2024-03-14T09:03:56.769597+0000","last_unstale":"2024-03-14T09:03:57.190661+0000","last_undegraded":"2024-03-14T09:03:57.190661+0000","last_fullsized":"2024-03-14T09:03:57.190661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:01.018014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190974+0000","last_change":"2024-03-14T09:03:57.187274+0000","last_active":"2024-03-14T09:03:57.190974+0000","last_peered":"2024-03-14T09:03:57.190974+0000","last_clean":"2024-03-14T09:03:57.190974+0000","last_became_active":"2024-03-14T09:03:57.187035+0000","last_became_peered":"2024-03-14T09:03:57.187035+0000","last_unstale":"2024-03-14T09:03:57.190974+0000","last_undegraded":"2024-03-14T09:03:57.190974+0000","last_fullsized":"2024-03-14T09:03:57.190974+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.179257+0000","last_change":"2024-03-14T09:03:56.772665+0000","last_active":"2024-03-14T09:03:57.179257+0000","last_peered":"2024-03-14T09:03:57.179257+0000","last_clean":"2024-03-14T09:03:57.179257+0000","last_became_active":"2024-03-14T09:03:56.772464+0000","last_became_peered":"2024-03-14T09:03:56.772464+0000","last_unstale":"2024-03-14T09:03:57.179257+0000","last_undegraded":"2024-03-14T09:03:57.179257+0000","last_fullsized":"2024-03-14T09:03:57.179257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:22.876530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.194060+0000","last_change":"2024-03-14T09:03:57.191774+0000","last_active":"2024-03-14T09:03:57.194060+0000","last_peered":"2024-03-14T09:03:57.194060+0000","last_clean":"2024-03-14T09:03:57.194060+0000","last_became_active":"2024-03-14T09:03:57.191391+0000","last_became_peered":"2024-03-14T09:03:57.191391+0000","last_unstale":"2024-03-14T09:03:57.194060+0000","last_undegraded":"2024-03-14T09:03:57.194060+0000","last_fullsized":"2024-03-14T09:03:57.194060+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:22.069879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.180009+0000","last_change":"2024-03-14T09:03:56.759918+0000","last_active":"2024-03-14T09:03:57.180009+0000","last_peered":"2024-03-14T09:03:57.180009+0000","last_clean":"2024-03-14T09:03:57.180009+0000","last_became_active":"2024-03-14T09:03:56.759495+0000","last_became_peered":"2024-03-14T09:03:56.759495+0000","last_unstale":"2024-03-14T09:03:57.180009+0000","last_undegraded":"2024-03-14T09:03:57.180009+0000","last_fullsized":"2024-03-14T09:03:57.180009+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:04.348438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188535+0000","last_change":"2024-03-14T09:03:57.181268+0000","last_active":"2024-03-14T09:03:57.188535+0000","last_peered":"2024-03-14T09:03:57.188535+0000","last_clean":"2024-03-14T09:03:57.188535+0000","last_became_active":"2024-03-14T09:03:57.180967+0000","last_became_peered":"2024-03-14T09:03:57.180967+0000","last_unstale":"2024-03-14T09:03:57.188535+0000","last_undegraded":"2024-03-14T09:03:57.188535+0000","last_fullsized":"2024-03-14T09:03:57.188535+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165514+0000","last_change":"2024-03-14T09:03:56.768053+0000","last_active":"2024-03-14T09:03:57.165514+0000","last_peered":"2024-03-14T09:03:57.165514+0000","last_clean":"2024-03-14T09:03:57.165514+0000","last_became_active":"2024-03-14T09:03:56.767780+0000","last_became_peered":"2024-03-14T09:03:56.767780+0000","last_unstale":"2024-03-14T09:03:57.165514+0000","last_undegraded":"2024-03-14T09:03:57.165514+0000","last_fullsized":"2024-03-14T09:03:57.165514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:00:23.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.186832+0000","last_change":"2024-03-14T09:03:57.177141+0000","last_active":"2024-03-14T09:03:57.186832+0000","last_peered":"2024-03-14T09:03:57.186832+0000","last_clean":"2024-03-14T09:03:57.186832+0000","last_became_active":"2024-03-14T09:03:57.176843+0000","last_became_peered":"2024-03-14T09:03:57.176843+0000","last_unstale":"2024-03-14T09:03:57.186832+0000","last_undegraded":"2024-03-14T09:03:57.186832+0000","last_fullsized":"2024-03-14T09:03:57.186832+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:57.185084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165305+0000","last_change":"2024-03-14T09:03:56.764413+0000","last_active":"2024-03-14T09:03:57.165305+0000","last_peered":"2024-03-14T09:03:57.165305+0000","last_clean":"2024-03-14T09:03:57.165305+0000","last_became_active":"2024-03-14T09:03:56.764156+0000","last_became_peered":"2024-03-14T09:03:56.764156+0000","last_unstale":"2024-03-14T09:03:57.165305+0000","last_undegraded":"2024-03-14T09:03:57.165305+0000","last_fullsized":"2024-03-14T09:03:57.165305+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:21:29.624660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190200+0000","last_change":"2024-03-14T09:03:57.187334+0000","last_active":"2024-03-14T09:03:57.190200+0000","last_peered":"2024-03-14T09:03:57.190200+0000","last_clean":"2024-03-14T09:03:57.190200+0000","last_became_active":"2024-03-14T09:03:57.187054+0000","last_became_peered":"2024-03-14T09:03:57.187054+0000","last_unstale":"2024-03-14T09:03:57.190200+0000","last_undegraded":"2024-03-14T09:03:57.190200+0000","last_fullsized":"2024-03-14T09:03:57.190200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.163616+0000","last_change":"2024-03-14T09:03:56.759664+0000","last_active":"2024-03-14T09:03:57.163616+0000","last_peered":"2024-03-14T09:03:57.163616+0000","last_clean":"2024-03-14T09:03:57.163616+0000","last_became_active":"2024-03-14T09:03:56.759378+0000","last_became_peered":"2024-03-14T09:03:56.759378+0000","last_unstale":"2024-03-14T09:03:57.163616+0000","last_undegraded":"2024-03-14T09:03:57.163616+0000","last_fullsized":"2024-03-14T09:03:57.163616+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:03.113813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.182669+0000","last_change":"2024-03-14T09:03:57.177315+0000","last_active":"2024-03-14T09:03:57.182669+0000","last_peered":"2024-03-14T09:03:57.182669+0000","last_clean":"2024-03-14T09:03:57.182669+0000","last_became_active":"2024-03-14T09:03:57.177045+0000","last_became_peered":"2024-03-14T09:03:57.177045+0000","last_unstale":"2024-03-14T09:03:57.182669+0000","last_undegraded":"2024-03-14T09:03:57.182669+0000","last_fullsized":"2024-03-14T09:03:57.182669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"69'2","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.184947+0000","last_change":"2024-03-14T09:03:56.764563+0000","last_active":"2024-03-14T09:03:57.184947+0000","last_peered":"2024-03-14T09:03:57.184947+0000","last_clean":"2024-03-14T09:03:57.184947+0000","last_became_active":"2024-03-14T09:03:56.764299+0000","last_became_peered":"2024-03-14T09:03:56.764299+0000","last_unstale":"2024-03-14T09:03:57.184947+0000","last_undegraded":"2024-03-14T09:03:57.184947+0000","last_fullsized":"2024-03-14T09:03:57.184947+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:22.457392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190035+0000","last_change":"2024-03-14T09:03:57.182451+0000","last_active":"2024-03-14T09:03:57.190035+0000","last_peered":"2024-03-14T09:03:57.190035+0000","last_clean":"2024-03-14T09:03:57.190035+0000","last_became_active":"2024-03-14T09:03:57.182007+0000","last_became_peered":"2024-03-14T09:03:57.182007+0000","last_unstale":"2024-03-14T09:03:57.190035+0000","last_undegraded":"2024-03-14T09:03:57.190035+0000","last_fullsized":"2024-03-14T09:03:57.190035+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.175176+0000","last_change":"2024-03-14T09:03:56.768285+0000","last_active":"2024-03-14T09:03:57.175176+0000","last_peered":"2024-03-14T09:03:57.175176+0000","last_clean":"2024-03-14T09:03:57.175176+0000","last_became_active":"2024-03-14T09:03:56.768060+0000","last_became_peered":"2024-03-14T09:03:56.768060+0000","last_unstale":"2024-03-14T09:03:57.175176+0000","last_undegraded":"2024-03-14T09:03:57.175176+0000","last_fullsized":"2024-03-14T09:03:57.175176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:56.180051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190668+0000","last_change":"2024-03-14T09:03:57.188009+0000","last_active":"2024-03-14T09:03:57.190668+0000","last_peered":"2024-03-14T09:03:57.190668+0000","last_clean":"2024-03-14T09:03:57.190668+0000","last_became_active":"2024-03-14T09:03:57.187745+0000","last_became_peered":"2024-03-14T09:03:57.187745+0000","last_unstale":"2024-03-14T09:03:57.190668+0000","last_undegraded":"2024-03-14T09:03:57.190668+0000","last_fullsized":"2024-03-14T09:03:57.190668+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.168474+0000","last_change":"2024-03-14T09:03:56.764742+0000","last_active":"2024-03-14T09:03:57.168474+0000","last_peered":"2024-03-14T09:03:57.168474+0000","last_clean":"2024-03-14T09:03:57.168474+0000","last_became_active":"2024-03-14T09:03:56.764467+0000","last_became_peered":"2024-03-14T09:03:56.764467+0000","last_unstale":"2024-03-14T09:03:57.168474+0000","last_undegraded":"2024-03-14T09:03:57.168474+0000","last_fullsized":"2024-03-14T09:03:57.168474+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.191050+0000","last_change":"2024-03-14T09:03:57.189309+0000","last_active":"2024-03-14T09:03:57.191050+0000","last_peered":"2024-03-14T09:03:57.191050+0000","last_clean":"2024-03-14T09:03:57.191050+0000","last_became_active":"2024-03-14T09:03:57.189127+0000","last_became_peered":"2024-03-14T09:03:57.189127+0000","last_unstale":"2024-03-14T09:03:57.191050+0000","last_undegraded":"2024-03-14T09:03:57.191050+0000","last_fullsized":"2024-03-14T09:03:57.191050+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.174752+0000","last_change":"2024-03-14T09:03:56.770115+0000","last_active":"2024-03-14T09:03:57.174752+0000","last_peered":"2024-03-14T09:03:57.174752+0000","last_clean":"2024-03-14T09:03:57.174752+0000","last_became_active":"2024-03-14T09:03:56.769621+0000","last_became_peered":"2024-03-14T09:03:56.769621+0000","last_unstale":"2024-03-14T09:03:57.174752+0000","last_undegraded":"2024-03-14T09:03:57.174752+0000","last_fullsized":"2024-03-14T09:03:57.174752+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.187802+0000","last_change":"2024-03-14T09:03:57.180976+0000","last_active":"2024-03-14T09:03:57.187802+0000","last_peered":"2024-03-14T09:03:57.187802+0000","last_clean":"2024-03-14T09:03:57.187802+0000","last_became_active":"2024-03-14T09:03:57.180689+0000","last_became_peered":"2024-03-14T09:03:57.180689+0000","last_unstale":"2024-03-14T09:03:57.187802+0000","last_undegraded":"2024-03-14T09:03:57.187802+0000","last_fullsized":"2024-03-14T09:03:57.187802+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:02.023046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.179957+0000","last_change":"2024-03-14T09:03:56.770777+0000","last_active":"2024-03-14T09:03:57.179957+0000","last_peered":"2024-03-14T09:03:57.179957+0000","last_clean":"2024-03-14T09:03:57.179957+0000","last_became_active":"2024-03-14T09:03:56.770466+0000","last_became_peered":"2024-03-14T09:03:56.770466+0000","last_unstale":"2024-03-14T09:03:57.179957+0000","last_undegraded":"2024-03-14T09:03:57.179957+0000","last_fullsized":"2024-03-14T09:03:57.179957+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:41.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192729+0000","last_change":"2024-03-14T09:03:57.184709+0000","last_active":"2024-03-14T09:03:57.192729+0000","last_peered":"2024-03-14T09:03:57.192729+0000","last_clean":"2024-03-14T09:03:57.192729+0000","last_became_active":"2024-03-14T09:03:57.184380+0000","last_became_peered":"2024-03-14T09:03:57.184380+0000","last_unstale":"2024-03-14T09:03:57.192729+0000","last_undegraded":"2024-03-14T09:03:57.192729+0000","last_fullsized":"2024-03-14T09:03:57.192729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165302+0000","last_change":"2024-03-14T09:03:56.769939+0000","last_active":"2024-03-14T09:03:57.165302+0000","last_peered":"2024-03-14T09:03:57.165302+0000","last_clean":"2024-03-14T09:03:57.165302+0000","last_became_active":"2024-03-14T09:03:56.769670+0000","last_became_peered":"2024-03-14T09:03:56.769670+0000","last_unstale":"2024-03-14T09:03:57.165302+0000","last_undegraded":"2024-03-14T09:03:57.165302+0000","last_fullsized":"2024-03-14T09:03:57.165302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:50.421546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190782+0000","last_change":"2024-03-14T09:03:57.187259+0000","last_active":"2024-03-14T09:03:57.190782+0000","last_peered":"2024-03-14T09:03:57.190782+0000","last_clean":"2024-03-14T09:03:57.190782+0000","last_became_active":"2024-03-14T09:03:57.186972+0000","last_became_peered":"2024-03-14T09:03:57.186972+0000","last_unstale":"2024-03-14T09:03:57.190782+0000","last_undegraded":"2024-03-14T09:03:57.190782+0000","last_fullsized":"2024-03-14T09:03:57.190782+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.168900+0000","last_change":"2024-03-14T09:03:56.771166+0000","last_active":"2024-03-14T09:03:57.168900+0000","last_peered":"2024-03-14T09:03:57.168900+0000","last_clean":"2024-03-14T09:03:57.168900+0000","last_became_active":"2024-03-14T09:03:56.769678+0000","last_became_peered":"2024-03-14T09:03:56.769678+0000","last_unstale":"2024-03-14T09:03:57.168900+0000","last_undegraded":"2024-03-14T09:03:57.168900+0000","last_fullsized":"2024-03-14T09:03:57.168900+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:06.742278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.188743+0000","last_change":"2024-03-14T09:03:57.181824+0000","last_active":"2024-03-14T09:03:57.188743+0000","last_peered":"2024-03-14T09:03:57.188743+0000","last_clean":"2024-03-14T09:03:57.188743+0000","last_became_active":"2024-03-14T09:03:57.181573+0000","last_became_peered":"2024-03-14T09:03:57.181573+0000","last_unstale":"2024-03-14T09:03:57.188743+0000","last_undegraded":"2024-03-14T09:03:57.188743+0000","last_fullsized":"2024-03-14T09:03:57.188743+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.165173+0000","last_change":"2024-03-14T09:03:56.770803+0000","last_active":"2024-03-14T09:03:57.165173+0000","last_peered":"2024-03-14T09:03:57.165173+0000","last_clean":"2024-03-14T09:03:57.165173+0000","last_became_active":"2024-03-14T09:03:56.770311+0000","last_became_peered":"2024-03-14T09:03:56.770311+0000","last_unstale":"2024-03-14T09:03:57.165173+0000","last_undegraded":"2024-03-14T09:03:57.165173+0000","last_fullsized":"2024-03-14T09:03:57.165173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.192676+0000","last_change":"2024-03-14T09:03:57.186670+0000","last_active":"2024-03-14T09:03:57.192676+0000","last_peered":"2024-03-14T09:03:57.192676+0000","last_clean":"2024-03-14T09:03:57.192676+0000","last_became_active":"2024-03-14T09:03:57.186398+0000","last_became_peered":"2024-03-14T09:03:57.186398+0000","last_unstale":"2024-03-14T09:03:57.192676+0000","last_undegraded":"2024-03-14T09:03:57.192676+0000","last_fullsized":"2024-03-14T09:03:57.192676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:15.878140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"69'1","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-14T09:03:57.190981+0000","last_change":"2024-03-14T09:03:56.766595+0000","last_active":"2024-03-14T09:03:57.190981+0000","last_peered":"2024-03-14T09:03:57.190981+0000","last_clean":"2024-03-14T09:03:57.190981+0000","last_became_active":"2024-03-14T09:03:56.766269+0000","last_became_peered":"2024-03-14T09:03:56.766269+0000","last_unstale":"2024-03-14T09:03:57.190981+0000","last_undegraded":"2024-03-14T09:03:57.190981+0000","last_fullsized":"2024-03-14T09:03:57.190981+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-03-14T09:03:56.759931+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:03:56.759931+0000","last_peered":"2024-03-14T09:03:56.759931+0000","last_clean":"2024-03-14T09:03:56.759931+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:03:56.759931+0000","last_undegraded":"2024-03-14T09:03:56.759931+0000","last_fullsized":"2024-03-14T09:03:56.759931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28420,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343420,"statfs":{"total":96636764160,"available":96607662080,"internally_reserved":0,"allocated":1429504,"data_stored":952692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.85699999999999998,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":6.5,"15min":6.5},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53100000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68100000000000005,"5min":0.878,"15min":0.878},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":369793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.87,"15min":0.87},"last":0.63},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":362324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33900000000000002,"15min":0.34000000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.45700000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.378},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39400000000000002,"15min":0.39500000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.497,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.307}]},{"osd":4,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.436}]},{"osd":5,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49099999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28440,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343400,"statfs":{"total":96636764160,"available":96607641600,"internally_reserved":0,"allocated":1445888,"data_stored":959501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.71599999999999997,"15min":0.78200000000000003},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.41799999999999998,"5min":7.383,"15min":7.383},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.79200000000000004,"15min":0.86599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51800000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.81699999999999995,"15min":0.89200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.48899999999999999,"5min":7.944,"15min":7.944},"last":0.377}]},{"osd":3,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.77200000000000002,"15min":0.83599999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0349999999999999,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.629},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.98399999999999999,"15min":1.0489999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0509999999999999,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80300000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":369681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34200000000000003,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.373},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.40400000000000003,"5min":1.111,"15min":1.111},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.314},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.29699999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.67300000000000004,"15min":0.72999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.41199999999999998,"5min":7.5,"15min":7.5},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.71599999999999997,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.48699999999999999,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":1.151,"15min":1.151},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":1.179,"15min":1.179},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71799999999999997,"5min":0.751,"15min":0.751},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.75900000000000001,"5min":1.165,"15min":1.165},"last":0.436},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64900000000000002,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83199999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":369771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.311,"15min":0.308},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.42799999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.307}]},{"osd":2,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.373,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.45400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.378},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.45800000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75700000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76200000000000001,"5min":1.087,"15min":1.087},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":362324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51100000000000001,"15min":0.51600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.5},{"interface":"front","average":{"1min":0.499,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.68999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.751,"15min":0.751},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.89100000000000001,"15min":0.94699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76000000000000001,"5min":7.702,"15min":7.702},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.753,"15min":0.753},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.749}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":370685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.44},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67900000000000005,"5min":0.876,"15min":0.876},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.627},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:01.699 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:04:01.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-14T09:04:01.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.864+0000 7f94c6d1f700 1 Processor -- start 2024-03-14T09:04:01.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.864+0000 7f94c6d1f700 1 -- start start 2024-03-14T09:04:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c6d1f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f94c0155df0 0x7f94c0156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c6d1f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94c015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c6d1f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c00ac8e0 con 0x7f94c014eaf0 2024-03-14T09:04:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c6d1f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006b310 con 0x7f94c0155df0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c6d1f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c006de30 con 0x7f94c015d270 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c4abb700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f94c014eaf0 0x7f94c006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:51986/0 (socket says 172.21.15.154:51986) 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c4abb700 1 -- 172.21.15.154:0/2123560318 learned_addr learned my addr 172.21.15.154:0/2123560318 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94c52bc700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94c015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.865+0000 7f94bffff700 1 --2- 172.21.15.154:0/2123560318 >> v2:172.21.15.191:3300/0 conn(0x7f94c0155df0 0x7f94c0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731736205 0 0) 0x7f94c00ac8e0 con 0x7f94c014eaf0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b00035b0 con 0x7f94c014eaf0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889745717 0 0) 0x7f94b00035b0 con 0x7f94c014eaf0 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 msgr2=0x7f94c015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94c015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 >> v2:172.21.15.191:3300/0 conn(0x7f94c0155df0 msgr2=0x7f94c0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 --2- 172.21.15.154:0/2123560318 >> v2:172.21.15.191:3300/0 conn(0x7f94c0155df0 0x7f94c0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00acb30 con 0x7f94c014eaf0 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f94b4001670 con 0x7f94c014eaf0 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 >> v1:172.21.15.154:6789/0 conn(0x7f94c014eaf0 legacy=0x7f94c006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b0004850 0x7f94b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c006b310 con 0x7f94b0003a00 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c006de30 con 0x7f94b0004850 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b000ba90 con 0x7f94c015d270 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94bffff700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b0004850 0x7f94b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c52bc700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 -- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 msgr2=0x7f94b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c015d270 0x7f94b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 -- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b0004850 msgr2=0x7f94b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b0004850 0x7f94b0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.866+0000 7f94c4abb700 1 -- 172.21.15.154:0/2123560318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b00035b0 con 0x7f94b0003a00 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c4abb700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94b0003e20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c370 tx=0x7f94b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d287e84d672c3758 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4007c60 con 0x7f94b0003a00 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4005960 con 0x7f94b0003a00 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94bf7fe700 1 -- 172.21.15.154:0/2123560318 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4005d10 con 0x7f94b0003a00 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 -- 172.21.15.154:0/2123560318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 msgr2=0x7f94b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/2123560318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94b0003e20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c370 tx=0x7f94b0005880 comp rx=0 tx=0).stop 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c52bc700 1 -- 172.21.15.154:0/2123560318 reap_dead start 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 -- 172.21.15.154:0/2123560318 shutdown_connections 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 -- 172.21.15.154:0/2123560318 >> 172.21.15.154:0/2123560318 conn(0x7f94c00c3880 msgr2=0x7f94c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 -- 172.21.15.154:0/2123560318 shutdown_connections 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.867+0000 7f94c6d1f700 1 -- 172.21.15.154:0/2123560318 wait complete. 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.868+0000 7f94c6d1f700 1 Processor -- start 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 -- start start 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c01ee4f0 0x7f94c0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c0153460 0x7f94c013c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00c1bb0 con 0x7f94b0003a00 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01ee0d0 con 0x7f94c01ee4f0 2024-03-14T09:04:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c6d1f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00c18d0 con 0x7f94c0153460 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c4abb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c4abb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39276/0 (socket says 172.21.15.154:39276) 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.869+0000 7f94c4abb700 1 -- 172.21.15.154:0/1609206747 learned_addr learned my addr 172.21.15.154:0/1609206747 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c52bc700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c0153460 0x7f94c013c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 -- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c0153460 msgr2=0x7f94c013c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94bffff700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c01ee4f0 0x7f94c0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c0153460 0x7f94c013c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 -- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c01ee4f0 msgr2=0x7f94c0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c01ee4f0 0x7f94c0152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 -- 172.21.15.154:0/1609206747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c01eda10 con 0x7f94b0003a00 2024-03-14T09:04:01.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c4abb700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c960 tx=0x7f94b4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4004870 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b40049f0 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c01edcf0 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4004da0 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.870+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0145120 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.871+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c01ed870 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.872+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f94b40070d0 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.872+0000 7f94bd7fa700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 0x7f94a406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.872+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94b408a340 con 0x7f94b0003a00 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.872+0000 7f94bffff700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 0x7f94a406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.872+0000 7f94bffff700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 0x7f94a406cd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94c013d510 tx=0x7f94ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:01.879 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:01.876+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f94c01ed870 con 0x7f94b0003a00 2024-03-14T09:04:02.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.148+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f94c01ee210 con 0x7f94b0003a00 2024-03-14T09:04:02.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.152+0000 7f94bd7fa700 1 -- 172.21.15.154:0/1609206747 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f94c01ee210 con 0x7f94b0003a00 2024-03-14T09:04:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 msgr2=0x7f94a406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 0x7f94a406cd20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f94c013d510 tx=0x7f94ac008040 comp rx=0 tx=0).stop 2024-03-14T09:04:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 msgr2=0x7f94c00c2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c960 tx=0x7f94b4000fe0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 shutdown_connections 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94c0153460 0x7f94c013c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94a406a860 0x7f94a406cd20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94b0003a00 0x7f94c00c2420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 --2- 172.21.15.154:0/1609206747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c01ee4f0 0x7f94c0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 >> 172.21.15.154:0/1609206747 conn(0x7f94c00c3880 msgr2=0x7f94c006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 shutdown_connections 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.154+0000 7f94c6d1f700 1 -- 172.21.15.154:0/1609206747 wait complete. 2024-03-14T09:04:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2024-03-14T09:04:02.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:02.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.288+0000 7f678f208700 1 Processor -- start 2024-03-14T09:04:02.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.288+0000 7f678f208700 1 -- start start 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678f208700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 0x7f678814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678f208700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67880c5d60 0x7f67880c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678f208700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67880acd10 con 0x7f67880702f0 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678f208700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6788150110 con 0x7f67880c5d60 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678f208700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67880ac7b0 con 0x7f67880c4380 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67880c5d60 0x7f67880c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67880c5d60 0x7f67880c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39282/0 (socket says 172.21.15.154:39282) 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 -- 172.21.15.154:0/1502823686 learned_addr learned my addr 172.21.15.154:0/1502823686 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f6787fff700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 0x7f678814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 -- 172.21.15.154:0/1502823686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 msgr2=0x7f678814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 0x7f678814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 -- 172.21.15.154:0/1502823686 >> v1:172.21.15.191:6789/0 conn(0x7f67880702f0 legacy=0x7f67880c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:02.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 -- 172.21.15.154:0/1502823686 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880ac9c0 con 0x7f67880c5d60 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f6787fff700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 0x7f678814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f678d7a5700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c5d60 0x7f67880c61d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6780009e50 tx=0x7f678000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998a99434e80410d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f67877fe700 1 -- 172.21.15.154:0/1502823686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000e070 con 0x7f67880c5d60 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f67877fe700 1 -- 172.21.15.154:0/1502823686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780003a20 con 0x7f67880c5d60 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.289+0000 7f67877fe700 1 -- 172.21.15.154:0/1502823686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780003dd0 con 0x7f67880c5d60 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 -- 172.21.15.154:0/1502823686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c5d60 msgr2=0x7f67880c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c5d60 0x7f67880c61d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6780009e50 tx=0x7f678000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 -- 172.21.15.154:0/1502823686 shutdown_connections 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c5d60 0x7f67880c61d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 --2- 172.21.15.154:0/1502823686 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c4380 0x7f678814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 -- 172.21.15.154:0/1502823686 >> 172.21.15.154:0/1502823686 conn(0x7f67880a9bf0 msgr2=0x7f67880a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 -- 172.21.15.154:0/1502823686 shutdown_connections 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.290+0000 7f678f208700 1 -- 172.21.15.154:0/1502823686 wait complete. 2024-03-14T09:04:02.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.291+0000 7f678f208700 1 Processor -- start 2024-03-14T09:04:02.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.291+0000 7f678f208700 1 -- start start 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67880702f0 0x7f67881513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c5d60 0x7f67881ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678814b200 con 0x7f67880c4380 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67881ec030 con 0x7f67880702f0 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678f208700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6788150960 con 0x7f67880c5d60 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39292/0 (socket says 172.21.15.154:39292) 2024-03-14T09:04:02.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 -- 172.21.15.154:0/275474077 learned_addr learned my addr 172.21.15.154:0/275474077 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678cfa4700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67880702f0 0x7f67881513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f678d7a5700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c5d60 0x7f67881ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 -- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c5d60 msgr2=0x7f67881ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c5d60 0x7f67881ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 -- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67880702f0 msgr2=0x7f67881513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67880702f0 0x7f67881513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.292+0000 7f6787fff700 1 -- 172.21.15.154:0/275474077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780009b30 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f6787fff700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f677400c750 tx=0x7f677400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774011800 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774011960 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678814ab40 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774011cd0 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.293+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678814a850 con 0x7f67880c4380 2024-03-14T09:04:02.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.294+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678814b340 con 0x7f67880c4380 2024-03-14T09:04:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.294+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6774021350 con 0x7f67880c4380 2024-03-14T09:04:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.295+0000 7f67857fa700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 0x7f676c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.295+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6774088ae0 con 0x7f67880c4380 2024-03-14T09:04:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.295+0000 7f678cfa4700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 0x7f676c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:02.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.295+0000 7f678cfa4700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 0x7f676c06ccf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f677c007fd0 tx=0x7f677c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:02.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.299+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f678814b340 con 0x7f67880c4380 2024-03-14T09:04:02.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.536+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f678814b340 con 0x7f67880c4380 2024-03-14T09:04:02.536 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.537+0000 7f67857fa700 1 -- 172.21.15.154:0/275474077 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4748 (secure 0 0 0) 0x7f678814b340 con 0x7f67880c4380 2024-03-14T09:04:02.537 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:02.537 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":45,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":6438,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 msgr2=0x7f676c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 0x7f676c06ccf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f677c007fd0 tx=0x7f677c007ec0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 msgr2=0x7f67881eb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f677400c750 tx=0x7f677400ca60 comp rx=0 tx=0).stop 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 shutdown_connections 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67880c5d60 0x7f67881ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67880702f0 0x7f67881513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f676c06a830 0x7f676c06ccf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.539 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 --2- 172.21.15.154:0/275474077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67880c4380 0x7f67881eb5b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:02.540 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 >> 172.21.15.154:0/275474077 conn(0x7f67880a9bf0 msgr2=0x7f6788148530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:02.540 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 shutdown_connections 2024-03-14T09:04:02.540 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:02.539+0000 7f678f208700 1 -- 172.21.15.154:0/275474077 wait complete. 2024-03-14T09:04:02.540 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 45 2024-03-14T09:04:02.670 INFO:tasks.ceph.mds.a.smithi154.stderr:2024-03-14T09:04:02.670+0000 7fa034edb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:04:02.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-14T09:03:57.159417+0000', 'modified': '2024-03-14T09:04:00.171013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6441}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6441': {'gid': 6441, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6835/1688485831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6833', 'nonce': 1688485831}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 1688485831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-14T09:04:02.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-14T09:04:02.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-14T09:04:03.078 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.078+0000 7f7054ba1700 1 Processor -- start 2024-03-14T09:04:03.079 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.078+0000 7f7054ba1700 1 -- start start 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f7054ba1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f705014eaf0 0x7f705006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f7054ba1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f7050155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f7054ba1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70500ac8e0 con 0x7f7050156330 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f7054ba1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705006b310 con 0x7f705014eaf0 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f7054ba1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705006de30 con 0x7f705006acb0 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.079+0000 7f704e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f705014eaf0 0x7f705006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704ed9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7050156330 0x7f7050157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:51996/0 (socket says 172.21.15.154:51996) 2024-03-14T09:04:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704ed9d700 1 -- 172.21.15.154:0/3163974966 learned_addr learned my addr 172.21.15.154:0/3163974966 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704dd9b700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f7050155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704e59c700 1 -- 172.21.15.154:0/3163974966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 msgr2=0x7f7050155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704e59c700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f7050155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704e59c700 1 -- 172.21.15.154:0/3163974966 >> v1:172.21.15.154:6789/0 conn(0x7f7050156330 legacy=0x7f7050157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704e59c700 1 -- 172.21.15.154:0/3163974966 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500acb30 con 0x7f705014eaf0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704d59a700 1 -- 172.21.15.154:0/3163974966 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829557321 0 0) 0x7f70500ac8e0 con 0x7f7050156330 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704dd9b700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f7050155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704e59c700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f705006a770 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7044009840 tx=0x7f7044009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918a8139722c27f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704d59a700 1 -- 172.21.15.154:0/3163974966 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044011830 con 0x7f705014eaf0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704d59a700 1 -- 172.21.15.154:0/3163974966 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044011990 con 0x7f705014eaf0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.080+0000 7f704d59a700 1 -- 172.21.15.154:0/3163974966 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044011d20 con 0x7f705014eaf0 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 -- 172.21.15.154:0/3163974966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 msgr2=0x7f705006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f705006a770 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7044009840 tx=0x7f7044009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 -- 172.21.15.154:0/3163974966 shutdown_connections 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f705006a770 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 --2- 172.21.15.154:0/3163974966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f7050155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 -- 172.21.15.154:0/3163974966 >> 172.21.15.154:0/3163974966 conn(0x7f70500c3880 msgr2=0x7f70500c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 -- 172.21.15.154:0/3163974966 shutdown_connections 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.081+0000 7f7054ba1700 1 -- 172.21.15.154:0/3163974966 wait complete. 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.082+0000 7f7054ba1700 1 Processor -- start 2024-03-14T09:04:03.084 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.082+0000 7f7054ba1700 1 -- start start 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f705014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050156330 0x7f70501ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050148d90 con 0x7f705014eaf0 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050152190 con 0x7f7050156330 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7050153f70 con 0x7f705006acb0 2024-03-14T09:04:03.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39310/0 (socket says 172.21.15.154:39310) 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 -- 172.21.15.154:0/2746159852 learned_addr learned my addr 172.21.15.154:0/2746159852 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704ed9d700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050156330 0x7f70501ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704e59c700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f705014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 -- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 msgr2=0x7f705014a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f705014a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 -- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050156330 msgr2=0x7f70501ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050156330 0x7f70501ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 -- 172.21.15.154:0/2746159852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440094a0 con 0x7f705014eaf0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f704000c7f0 tx=0x7f704000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f703effd700 1 -- 172.21.15.154:0/2746159852 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040011800 con 0x7f705014eaf0 2024-03-14T09:04:03.088 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f703effd700 1 -- 172.21.15.154:0/2746159852 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040011960 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f703effd700 1 -- 172.21.15.154:0/2746159852 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040011d10 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7050154250 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.083+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501ee340 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.085+0000 7f703effd700 1 -- 172.21.15.154:0/2746159852 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7040018360 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.085+0000 7f703effd700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 0x7f703006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.085+0000 7f703effd700 1 -- 172.21.15.154:0/2746159852 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7040088cc0 con 0x7f705014eaf0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.085+0000 7f704e59c700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 0x7f703006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.086+0000 7f704e59c700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 0x7f703006ccf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f70440090b0 tx=0x7f704401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.087+0000 7f7060d5ab80 1 Processor -- start 2024-03-14T09:04:03.089 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.087+0000 7f7060d5ab80 1 -- start start 2024-03-14T09:04:03.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d5fe0 0x557b9d0f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d10f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d02f940 con 0x557b9d0d63c0 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d02d300 con 0x557b9d0d5fe0 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b9d028bf0 con 0x557b9d10f180 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39322/0 (socket says 172.21.15.154:39322) 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 -- 172.21.15.154:0/2409127364 learned_addr learned my addr 172.21.15.154:0/2409127364 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704e59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d5fe0 0x557b9d0f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704ed9d700 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d10f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 -- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 msgr2=0x557b9d10f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d10f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 -- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d5fe0 msgr2=0x557b9d0f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d5fe0 0x557b9d0f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 -- 172.21.15.154:0/2409127364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9ceaa360 con 0x557b9d0d63c0 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.088+0000 7f704dd9b700 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f704005fcf0 tx=0x7f7040058600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb22a6eb5392e61 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f702ffff700 1 -- 172.21.15.154:0/2409127364 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704005bc60 con 0x557b9d0d63c0 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f702ffff700 1 -- 172.21.15.154:0/2409127364 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040060220 con 0x557b9d0d63c0 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f702ffff700 1 -- 172.21.15.154:0/2409127364 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040057320 con 0x557b9d0d63c0 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 -- 172.21.15.154:0/2409127364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 msgr2=0x557b9d0f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f704005fcf0 tx=0x7f7040058600 comp rx=0 tx=0).stop 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 -- 172.21.15.154:0/2409127364 shutdown_connections 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d10f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d5fe0 0x557b9d0f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/2409127364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d0f5920 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 -- 172.21.15.154:0/2409127364 >> 172.21.15.154:0/2409127364 conn(0x557b9d04c1d0 msgr2=0x557b9d0d4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 -- 172.21.15.154:0/2409127364 shutdown_connections 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.089+0000 7f7060d5ab80 1 -- 172.21.15.154:0/2409127364 wait complete. 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.090+0000 7f7060d5ab80 1 -- start start 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.090+0000 7f7060d5ab80 1 Processor -- start 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d0f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d0ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9cfc4d30 con 0x557b9d0d5fe0 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9cfc49d0 con 0x557b9d0d63c0 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b9ceac650 con 0x557b9d10f180 2024-03-14T09:04:03.099 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f7060d5ab80 10 client.0 _ll_register_callbacks cb 0x557b9cfce420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39328/0 (socket says 172.21.15.154:39328) 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 -- 172.21.15.154:0/1182102052 learned_addr learned my addr 172.21.15.154:0/1182102052 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704ed9d700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d0ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d0f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 -- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 msgr2=0x557b9d0ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d0ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 -- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 msgr2=0x557b9d0f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d0f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 -- 172.21.15.154:0/1182102052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7040011e70 con 0x557b9d0d5fe0 2024-03-14T09:04:03.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d0f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f704e59c700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7044013000 tx=0x7f704400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12be977a729bcdca server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.091+0000 7f702d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704402c070 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7044003d80 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b9d0c9450 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7044005a20 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 10 client.6588 fetch_fsmap learned FSMap version 45 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x557b9ce56b10 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b9d0c9450 con 0x557b9d0d5fe0 2024-03-14T09:04:03.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7044005b80 con 0x557b9d0d5fe0 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 10 client.6588 fetch_fsmap finished waiting for FSMap version 45 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 10 client.6588 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 20 client.6588 populate_metadata read hostname 'smithi154' 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d1248a0 0x557b9d111fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 4 client.6588 mds_command: new command op to 6441 tid=0[{"prefix": "get_command_descriptions"}] 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 --> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b9d0216b0 con 0x557b9d1248a0 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d1248a0 0x557b9d111fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.092+0000 7f702d7fa700 10 client.6588 ms_handle_connect on v2:172.21.15.154:6833/1688485831 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d1248a0 0x557b9d111fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469c65397102fe26 server_cookie=503a11bb67dd6b97 in_seq=0 out_seq=0 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f702d7fa700 1 -- 172.21.15.154:0/1182102052 <== mds.0 v2:172.21.15.154:6833/1688485831 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557b9d0216b0 con 0x557b9d1248a0 2024-03-14T09:04:03.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f702d7fa700 10 client.6588 handle_command_reply: tid=0 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 client.6588 shutdown 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 msgr2=0x557b9d0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7044013000 tx=0x7f704400e000 comp rx=0 tx=0).stop 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 shutdown_connections 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d1248a0 0x557b9d111fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d10f180 0x557b9d0ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d5fe0 0x557b9d0d5210 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f702d7fa700 0 client.6588 ms_handle_reset on v2:172.21.15.154:6833/1688485831 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1182102052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d0f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 >> 172.21.15.154:0/1182102052 conn(0x557b9d04c1d0 msgr2=0x557b9d0d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 shutdown_connections 2024-03-14T09:04:03.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.093+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1182102052 wait complete. 2024-03-14T09:04:03.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.094+0000 7f7060d5ab80 20 client.6588 trim_cache size 0 max 16384 2024-03-14T09:04:03.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.111+0000 7f7060d5ab80 1 Processor -- start 2024-03-14T09:04:03.111 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.111+0000 7f7060d5ab80 1 -- start start 2024-03-14T09:04:03.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.111+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d0d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.111+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d0d2250 con 0x557b9d10f180 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d0f5cd0 con 0x557b9d0d63c0 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b9d0d1dd0 con 0x557b9d0d5fe0 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39342/0 (socket says 172.21.15.154:39342) 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 -- 172.21.15.154:0/3840541019 learned_addr learned my addr 172.21.15.154:0/3840541019 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704dd9b700 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704e59c700 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d0d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 -- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 msgr2=0x557b9d0d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d0d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 -- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 msgr2=0x557b9d10e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d10e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 -- 172.21.15.154:0/3840541019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9cfeee50 con 0x557b9d10f180 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f704ed9d700 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f70480071e0 tx=0x7f70480074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bebc5bd1868df5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f703cff9700 1 -- 172.21.15.154:0/3840541019 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048016630 con 0x557b9d10f180 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f703cff9700 1 -- 172.21.15.154:0/3840541019 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048016790 con 0x557b9d10f180 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.112+0000 7f703cff9700 1 -- 172.21.15.154:0/3840541019 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048016b20 con 0x557b9d10f180 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- 172.21.15.154:0/3840541019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 msgr2=0x557b9d10e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f70480071e0 tx=0x7f70480074f0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- 172.21.15.154:0/3840541019 shutdown_connections 2024-03-14T09:04:03.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d10f180 0x557b9d10e900 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d0d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/3840541019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d0d63c0 0x557b9d10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- 172.21.15.154:0/3840541019 >> 172.21.15.154:0/3840541019 conn(0x557b9d04c1d0 msgr2=0x557b9d0cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- 172.21.15.154:0/3840541019 shutdown_connections 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- 172.21.15.154:0/3840541019 wait complete. 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 -- start start 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.113+0000 7f7060d5ab80 1 Processor -- start 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d11a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 0x557b9d10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d17bd30 con 0x557b9d0d63c0 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9d119820 con 0x557b9d10f180 2024-03-14T09:04:03.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b9d178ac0 con 0x557b9d0d5fe0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f7060d5ab80 10 client.0 _ll_register_callbacks cb 0x557b9cfce420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f704dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39348/0 (socket says 172.21.15.154:39348) 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f704dd9b700 1 -- 172.21.15.154:0/1866675982 learned_addr learned my addr 172.21.15.154:0/1866675982 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f704e59c700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d11a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.114+0000 7f704ed9d700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 0x557b9d10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 -- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 msgr2=0x557b9d11a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d11a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 -- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 msgr2=0x557b9d10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 0x557b9d10e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 -- 172.21.15.154:0/1866675982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704800f040 con 0x557b9d0d63c0 2024-03-14T09:04:03.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704ed9d700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 0x557b9d10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704dd9b700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7040087220 tx=0x7f704005e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6363572090510fa server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040057520 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7040058e40 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7040060220 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7040055ff0 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 10 client.6594 fetch_fsmap learned FSMap version 45 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x557b9ceaec80 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040060220 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 45) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f704005fd90 con 0x557b9d0d63c0 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 10 client.6594 fetch_fsmap finished waiting for FSMap version 45 2024-03-14T09:04:03.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 10 client.6594 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 20 client.6594 populate_metadata read hostname 'smithi154' 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d175540 0x557b9d175920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 4 client.6594 mds_command: new command op to 6441 tid=0[{"prefix": "status"}] 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 --> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557b9ce77ca0 con 0x557b9d175540 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.115+0000 7f704e59c700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d175540 0x557b9d175920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f702d7fa700 10 client.6594 ms_handle_connect on v2:172.21.15.154:6833/1688485831 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f704e59c700 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d175540 0x557b9d175920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2e373ceba2ec70 server_cookie=8039884701600898 in_seq=0 out_seq=0 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f702d7fa700 1 -- 172.21.15.154:0/1866675982 <== mds.0 v2:172.21.15.154:6833/1688485831 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557b9ce77ca0 con 0x557b9d175540 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f702d7fa700 10 client.6594 handle_command_reply: tid=0 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 client.6594 shutdown 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 msgr2=0x557b9d1264f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7040087220 tx=0x7f704005e040 comp rx=0 tx=0).stop 2024-03-14T09:04:03.123 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 shutdown_connections 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:6833/1688485831,v1:172.21.15.154:6835/1688485831] conn(0x557b9d175540 0x557b9d175920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b9d10f180 0x557b9d10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x557b9d0d5fe0 0x557b9d11a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f702d7fa700 0 client.6594 ms_handle_reset on v2:172.21.15.154:6833/1688485831 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 --2- 172.21.15.154:0/1866675982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b9d0d63c0 0x557b9d1264f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 >> 172.21.15.154:0/1866675982 conn(0x557b9d04c1d0 msgr2=0x557b9d0d3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 shutdown_connections 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 1 -- 172.21.15.154:0/1866675982 wait complete. 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.116+0000 7f7060d5ab80 20 client.6594 trim_cache size 0 max 16384 2024-03-14T09:04:03.124 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "661f6023-01e4-4d6e-b4b3-5b5701ce7cc5", 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 6441, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 5.9559706989999999, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 45, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 70, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 70, 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 11.426882129999999 2024-03-14T09:04:03.125 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:03.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 msgr2=0x7f703006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 0x7f703006ccf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f70440090b0 tx=0x7f704401c040 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 msgr2=0x7f70501ed8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f704000c7f0 tx=0x7f704000cbb0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 shutdown_connections 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7050156330 0x7f70501ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f705006acb0 0x7f705014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f703006a830 0x7f703006ccf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 --2- 172.21.15.154:0/2746159852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f705014eaf0 0x7f70501ed8c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 >> 172.21.15.154:0/2746159852 conn(0x7f70500c3880 msgr2=0x7f70500c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 shutdown_connections 2024-03-14T09:04:03.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.117+0000 7f7054ba1700 1 -- 172.21.15.154:0/2746159852 wait complete. 2024-03-14T09:04:03.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:04:03.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.250+0000 7f0370921700 1 Processor -- start 2024-03-14T09:04:03.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.250+0000 7f0370921700 1 -- start start 2024-03-14T09:04:03.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0370921700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f036c06ebf0 0x7f036c0c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0370921700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f036c151fc0 0x7f036c158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0370921700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c070e80 con 0x7f036c0c22a0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0370921700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0c0670 con 0x7f036c06ebf0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0370921700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c071e70 con 0x7f036c151fc0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f036c06ebf0 0x7f036c0c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0369d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f036c0c22a0 0x7f036c157e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52000/0 (socket says 172.21.15.154:52000) 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f0369d9b700 1 -- 172.21.15.154:0/1424272316 learned_addr learned my addr 172.21.15.154:0/1424272316 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036ad9d700 1 --2- 172.21.15.154:0/1424272316 >> v2:172.21.15.191:3300/0 conn(0x7f036c151fc0 0x7f036c158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 -- 172.21.15.154:0/1424272316 >> v2:172.21.15.191:3300/0 conn(0x7f036c151fc0 msgr2=0x7f036c158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 --2- 172.21.15.154:0/1424272316 >> v2:172.21.15.191:3300/0 conn(0x7f036c151fc0 0x7f036c158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 -- 172.21.15.154:0/1424272316 >> v1:172.21.15.154:6789/0 conn(0x7f036c0c22a0 legacy=0x7f036c157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 -- 172.21.15.154:0/1424272316 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0710b0 con 0x7f036c06ebf0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036959a700 1 -- 172.21.15.154:0/1424272316 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500651039 0 0) 0x7f036c070e80 con 0x7f036c0c22a0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036a59c700 1 --2- 172.21.15.154:0/1424272316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c0c1d60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f036c070330 tx=0x7f0360009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507555abccc320d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036959a700 1 -- 172.21.15.154:0/1424272316 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360011830 con 0x7f036c06ebf0 2024-03-14T09:04:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036959a700 1 -- 172.21.15.154:0/1424272316 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360011990 con 0x7f036c06ebf0 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.251+0000 7f036959a700 1 -- 172.21.15.154:0/1424272316 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360011d20 con 0x7f036c06ebf0 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 -- 172.21.15.154:0/1424272316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 msgr2=0x7f036c0c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 --2- 172.21.15.154:0/1424272316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c0c1d60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f036c070330 tx=0x7f0360009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 -- 172.21.15.154:0/1424272316 shutdown_connections 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 --2- 172.21.15.154:0/1424272316 >> v2:172.21.15.191:3300/0 conn(0x7f036c151fc0 0x7f036c158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 --2- 172.21.15.154:0/1424272316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c0c1d60 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 -- 172.21.15.154:0/1424272316 >> 172.21.15.154:0/1424272316 conn(0x7f036c0c6130 msgr2=0x7f036c0c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 -- 172.21.15.154:0/1424272316 shutdown_connections 2024-03-14T09:04:03.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.252+0000 7f0370921700 1 -- 172.21.15.154:0/1424272316 wait complete. 2024-03-14T09:04:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.253+0000 7f0370921700 1 Processor -- start 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.253+0000 7f0370921700 1 -- start start 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f036c0c22a0 0x7f036c1dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f036c151fc0 0x7f036c1deaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1e19c0 con 0x7f036c06ebf0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c158730 con 0x7f036c151fc0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0370921700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c06ced0 con 0x7f036c0c22a0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39378/0 (socket says 172.21.15.154:39378) 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 -- 172.21.15.154:0/4156259027 learned_addr learned my addr 172.21.15.154:0/4156259027 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f0369d9b700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f036c0c22a0 0x7f036c1dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036ad9d700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f036c151fc0 0x7f036c1deaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 -- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f036c0c22a0 msgr2=0x7f036c1dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f036c0c22a0 0x7f036c1dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 -- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f036c151fc0 msgr2=0x7f036c1deaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f036c151fc0 0x7f036c1deaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.254+0000 7f036a59c700 1 -- 172.21.15.154:0/4156259027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600094a0 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f036a59c700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f03600090b0 tx=0x7f0360004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600038a0 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036001f070 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360003c20 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c158980 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.255+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c06d170 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.256+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c06deb0 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.256+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f036001a430 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.257+0000 7f035affd700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 0x7f034c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.257+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0360021070 con 0x7f036c06ebf0 2024-03-14T09:04:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.257+0000 7f0369d9b700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 0x7f034c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.257+0000 7f0369d9b700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 0x7f034c06ccf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f035c007fd0 tx=0x7f035c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.261+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0360055d00 con 0x7f036c06ebf0 2024-03-14T09:04:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.500+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f036c0697d0 con 0x7f036c06ebf0 2024-03-14T09:04:03.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.501+0000 7f035affd700 1 -- 172.21.15.154:0/4156259027 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f036002b070 con 0x7f036c06ebf0 2024-03-14T09:04:03.501 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 msgr2=0x7f034c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 0x7f034c06ccf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f035c007fd0 tx=0x7f035c009040 comp rx=0 tx=0).stop 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 msgr2=0x7f036c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f03600090b0 tx=0x7f0360004750 comp rx=0 tx=0).stop 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 shutdown_connections 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f036c151fc0 0x7f036c1deaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f034c06a830 0x7f034c06ccf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f036c06ebf0 0x7f036c06d970 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 --2- 172.21.15.154:0/4156259027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f036c0c22a0 0x7f036c1dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:03.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 >> 172.21.15.154:0/4156259027 conn(0x7f036c0c6130 msgr2=0x7f036c150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:03.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 shutdown_connections 2024-03-14T09:04:03.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:03.503+0000 7f0370921700 1 -- 172.21.15.154:0/4156259027 wait complete. 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:04:03.515 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:04:03.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.516 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:04:03.533 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:04:03.533 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:04:03.533 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:04:03.534 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42216sec preferred_lft 42216sec 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:04:03.535 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.536 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:04:03.536 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:04:03.536 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:04:03.536 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:03.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.536 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:04:03.599 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:04:03.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.600 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:04:03.659 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:04:03.659 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:04:03.659 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:04:03.659 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:04:03.660 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:04:03.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:04:03.660 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:04:03.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.765 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:04:03.782 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:04:03.782 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:04:03.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.782 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:04:03.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:03.850 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:03.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:03.906 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:03.963 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-14T09:04:03.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:03.963 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:04.048 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:04.049 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:04.068 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:04.086 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:04.086 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:04.086 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: 0@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:04:04.087 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:04:04.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:04:04.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:04.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.116 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:04:04.204 INFO:teuthology.orchestra.run.smithi154.stdout:client6606 2024-03-14T09:04:04.204 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:04.204 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:04.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:04:04.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.386+0000 7fa697981700 1 Processor -- start 2024-03-14T09:04:04.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.387+0000 7fa697981700 1 -- start start 2024-03-14T09:04:04.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.387+0000 7fa697981700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6900a8540 0x7fa69013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:04.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.387+0000 7fa697981700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69013ade0 con 0x7fa6900c1780 2024-03-14T09:04:04.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.387+0000 7fa697981700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69006fb30 con 0x7fa6900c3250 2024-03-14T09:04:04.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.387+0000 7fa697981700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69006ec30 con 0x7fa6900a8540 2024-03-14T09:04:04.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa69571d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6900a8540 0x7fa69013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa695f1e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa6900c1780 0x7fa6900c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52006/0 (socket says 172.21.15.154:52006) 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa695f1e700 1 -- 172.21.15.154:0/1555968218 learned_addr learned my addr 172.21.15.154:0/1555968218 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa69571d700 1 -- 172.21.15.154:0/1555968218 >> v1:172.21.15.191:6789/0 conn(0x7fa6900c3250 legacy=0x7fa690154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa69571d700 1 -- 172.21.15.154:0/1555968218 >> v1:172.21.15.154:6789/0 conn(0x7fa6900c1780 legacy=0x7fa6900c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa69571d700 1 -- 172.21.15.154:0/1555968218 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69006ef10 con 0x7fa6900a8540 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa687fff700 1 -- 172.21.15.154:0/1555968218 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117784477 0 0) 0x7fa688001490 con 0x7fa6900c1780 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa69571d700 1 --2- 172.21.15.154:0/1555968218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900a8540 0x7fa69013a8a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa68000bae0 tx=0x7fa68000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377c48f42342f064 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa687fff700 1 -- 172.21.15.154:0/1555968218 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68000dcf0 con 0x7fa6900a8540 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa687fff700 1 -- 172.21.15.154:0/1555968218 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68000de50 con 0x7fa6900a8540 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.388+0000 7fa687fff700 1 -- 172.21.15.154:0/1555968218 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680004250 con 0x7fa6900a8540 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 -- 172.21.15.154:0/1555968218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900a8540 msgr2=0x7fa69013a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 --2- 172.21.15.154:0/1555968218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900a8540 0x7fa69013a8a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa68000bae0 tx=0x7fa68000bdf0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 -- 172.21.15.154:0/1555968218 shutdown_connections 2024-03-14T09:04:04.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 --2- 172.21.15.154:0/1555968218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900a8540 0x7fa69013a8a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 -- 172.21.15.154:0/1555968218 >> 172.21.15.154:0/1555968218 conn(0x7fa6900c3880 msgr2=0x7fa6900c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:04.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 -- 172.21.15.154:0/1555968218 shutdown_connections 2024-03-14T09:04:04.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.389+0000 7fa697981700 1 -- 172.21.15.154:0/1555968218 wait complete. 2024-03-14T09:04:04.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.390+0000 7fa697981700 1 Processor -- start 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.390+0000 7fa697981700 1 -- start start 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6900a8540 0x7fa6901342f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6900c1780 0x7fa6900c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690134830 con 0x7fa6900c3250 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6901e7c60 con 0x7fa6900c1780 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa690150a30 con 0x7fa6900a8540 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39392/0 (socket says 172.21.15.154:39392) 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 -- 172.21.15.154:0/1210447750 learned_addr learned my addr 172.21.15.154:0/1210447750 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa694f1c700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6900c1780 0x7fa6900c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa69571d700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6900a8540 0x7fa6901342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:04.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 -- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6900a8540 msgr2=0x7fa6901342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6900a8540 0x7fa6901342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 -- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6900c1780 msgr2=0x7fa6900c02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6900c1780 0x7fa6900c02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 -- 172.21.15.154:0/1210447750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68000b740 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa695f1e700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa68800cb90 tx=0x7fa68800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800aad0 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800ac30 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6901edca0 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.391+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880070d0 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.392+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa690151950 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.392+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6901ede30 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.393+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa688007230 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.393+0000 7fa685ffb700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 0x7fa67406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.393+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa688088940 con 0x7fa6900c3250 2024-03-14T09:04:04.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.393+0000 7fa69571d700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 0x7fa67406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:04.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.394+0000 7fa69571d700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 0x7fa67406ccf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b2c0 tx=0x7fa680008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:04.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.398+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa688053df0 con 0x7fa6900c3250 2024-03-14T09:04:04.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.635+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa69013ade0 con 0x7fa6900c3250 2024-03-14T09:04:04.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.636+0000 7fa685ffb700 1 -- 172.21.15.154:0/1210447750 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa69013ade0 con 0x7fa6900c3250 2024-03-14T09:04:04.636 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 msgr2=0x7fa67406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 0x7fa67406ccf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b2c0 tx=0x7fa680008040 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 msgr2=0x7fa690134cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa68800cb90 tx=0x7fa68800cf50 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 shutdown_connections 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa67406a830 0x7fa67406ccf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6900c3250 0x7fa690134cc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa6900a8540 0x7fa6901342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 --2- 172.21.15.154:0/1210447750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6900c1780 0x7fa6900c02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:04.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 >> 172.21.15.154:0/1210447750 conn(0x7fa6900c3880 msgr2=0x7fa69006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:04.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 shutdown_connections 2024-03-14T09:04:04.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:04.638+0000 7fa697981700 1 -- 172.21.15.154:0/1210447750 wait complete. 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:04:04.650 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:04.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.651 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:04.672 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42220sec preferred_lft 42220sec 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:04:04.673 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:04.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.675 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:04:04.742 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:04:04.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.744 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:04:04.804 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:04:04.804 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:04:04.804 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:04:04.804 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:04:04.805 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:04:04.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:04:04.805 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:04:04.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.914 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:04:04.931 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:04:04.931 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:04.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:04.932 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:04:04.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:04.999 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:05.055 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:05.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:05.056 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:05.112 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-14T09:04:05.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:05.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:05.198 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:05.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:05.218 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:05.236 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:05.237 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:05.237 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:04:05.237 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:04:05.237 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:04:05.239 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:04:05.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:05.239 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:05.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:05.265 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:04:05.338 INFO:teuthology.orchestra.run.smithi191.stdout:client6040 2024-03-14T09:04:05.339 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:05.339 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:05.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth rm client.testuser 2024-03-14T09:04:05.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.520+0000 7fd1e06f7700 1 Processor -- start 2024-03-14T09:04:05.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.520+0000 7fd1e06f7700 1 -- start start 2024-03-14T09:04:05.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1e06f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d8156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1e06f7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1d815d270 0x7fd1d815f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1e06f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d80ac8e0 con 0x7fd1d814eaf0 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1e06f7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d806b310 con 0x7fd1d815d270 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1e06f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d806de30 con 0x7fd1d8155df0 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1dec94700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1d815d270 0x7fd1d815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1dec94700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1d815d270 0x7fd1d815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39396/0 (socket says 172.21.15.154:39396) 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.521+0000 7fd1dec94700 1 -- 172.21.15.154:0/2114760837 learned_addr learned my addr 172.21.15.154:0/2114760837 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1ddc92700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dec94700 1 -- 172.21.15.154:0/2114760837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 msgr2=0x7fd1d8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dec94700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dec94700 1 -- 172.21.15.154:0/2114760837 >> v1:172.21.15.191:6789/0 conn(0x7fd1d814eaf0 legacy=0x7fd1d806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dec94700 1 -- 172.21.15.154:0/2114760837 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80acb30 con 0x7fd1d815d270 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1ddc92700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dec94700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d815d270 0x7fd1d815f660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009e50 tx=0x7fd1d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38f14861ef2c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dd491700 1 -- 172.21.15.154:0/2114760837 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000e070 con 0x7fd1d815d270 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dd491700 1 -- 172.21.15.154:0/2114760837 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d000b9c0 con 0x7fd1d815d270 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.522+0000 7fd1dd491700 1 -- 172.21.15.154:0/2114760837 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000bd70 con 0x7fd1d815d270 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2114760837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d815d270 msgr2=0x7fd1d815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d815d270 0x7fd1d815f660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009e50 tx=0x7fd1d000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2114760837 shutdown_connections 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d815d270 0x7fd1d815f660 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2114760837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d8156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2114760837 >> 172.21.15.154:0/2114760837 conn(0x7fd1d80c3880 msgr2=0x7fd1d80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2114760837 shutdown_connections 2024-03-14T09:04:05.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.523+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2114760837 wait complete. 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 Processor -- start 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 -- start start 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1d815d270 0x7fd1d8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d814bea0 con 0x7fd1d814eaf0 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d814ad20 con 0x7fd1d815d270 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.524+0000 7fd1e06f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d814b530 con 0x7fd1d8155df0 2024-03-14T09:04:05.527 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39410/0 (socket says 172.21.15.154:39410) 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 -- 172.21.15.154:0/2829494166 learned_addr learned my addr 172.21.15.154:0/2829494166 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1ddc92700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1dec94700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1d815d270 0x7fd1d8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 -- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 msgr2=0x7fd1d815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d815cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 -- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1d815d270 msgr2=0x7fd1d8150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1d815d270 0x7fd1d8150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 -- 172.21.15.154:0/2829494166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0009b30 con 0x7fd1d814eaf0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1de493700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c4e0 tx=0x7fd1c800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80052d0 con 0x7fd1d814eaf0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8005430 con 0x7fd1d814eaf0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d814af70 con 0x7fd1d814eaf0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80038c0 con 0x7fd1d814eaf0 2024-03-14T09:04:05.528 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.525+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d81ea6f0 con 0x7fd1d814eaf0 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.526+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d81ea550 con 0x7fd1d814eaf0 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.527+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd1c8003a20 con 0x7fd1d814eaf0 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.527+0000 7fd1ceffd700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 0x7fd1bc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.527+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1c8088070 con 0x7fd1d814eaf0 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.527+0000 7fd1ddc92700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 0x7fd1bc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.529 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.527+0000 7fd1ddc92700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 0x7fd1bc06ccf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80681a0 tx=0x7fd1d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.531 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.531+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd1d81ea550 con 0x7fd1d814eaf0 2024-03-14T09:04:05.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.807+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth rm", "entity": "client.testuser"} v 0) v1 -- 0x7fd1d81ea550 con 0x7fd1d814eaf0 2024-03-14T09:04:05.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.808+0000 7fd1ceffd700 1 -- 172.21.15.154:0/2829494166 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth rm", "entity": "client.testuser"}]=0 v23) v1 ==== 84+0+0 (secure 0 0 0) 0x7fd1d81ea550 con 0x7fd1d814eaf0 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 msgr2=0x7fd1bc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 0x7fd1bc06ccf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80681a0 tx=0x7fd1d4008040 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 msgr2=0x7fd1d8144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c4e0 tx=0x7fd1c800c7f0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 shutdown_connections 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1d815d270 0x7fd1d8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd1bc06a830 0x7fd1bc06ccf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1d814eaf0 0x7fd1d8144a60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 --2- 172.21.15.154:0/2829494166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1d8155df0 0x7fd1d815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 >> 172.21.15.154:0/2829494166 conn(0x7fd1d80c3880 msgr2=0x7fd1d8156830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:05.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 shutdown_connections 2024-03-14T09:04:05.811 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.810+0000 7fd1e06f7700 1 -- 172.21.15.154:0/2829494166 wait complete. 2024-03-14T09:04:05.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:05.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.943+0000 7fca5308e700 1 Processor -- start 2024-03-14T09:04:05.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.943+0000 7fca5308e700 1 -- start start 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5308e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 0x7fca4c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5308e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 0x7fca4c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5308e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c13ade0 con 0x7fca4c0c3250 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5308e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c06fb30 con 0x7fca4c0c1780 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5308e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c06ec30 con 0x7fca4c0a8540 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca4bfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fca4c0c3250 0x7fca4c154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52018/0 (socket says 172.21.15.154:52018) 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca4bfff700 1 -- 172.21.15.154:0/620318553 learned_addr learned my addr 172.21.15.154:0/620318553 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca50e2a700 1 --2- 172.21.15.154:0/620318553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 0x7fca4c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca5162b700 1 --2- 172.21.15.154:0/620318553 >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 0x7fca4c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759423495 0 0) 0x7fca38001110 con 0x7fca4c0c3250 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.944+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c0035b0 con 0x7fca4c0c3250 2024-03-14T09:04:05.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca50e2a700 1 -- 172.21.15.154:0/620318553 >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 msgr2=0x7fca4c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca50e2a700 1 --2- 172.21.15.154:0/620318553 >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 0x7fca4c0c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca50e2a700 1 -- 172.21.15.154:0/620318553 >> v1:172.21.15.154:6789/0 conn(0x7fca4c0c3250 legacy=0x7fca4c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951472564 0 0) 0x7fca3c0035b0 con 0x7fca4c0c3250 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca50e2a700 1 -- 172.21.15.154:0/620318553 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c06ef10 con 0x7fca4c0a8540 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca5162b700 1 --2- 172.21.15.154:0/620318553 >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 0x7fca4c0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca50e2a700 1 --2- 172.21.15.154:0/620318553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 0x7fca4c13a8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca40009840 tx=0x7fca40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5feae290ec0ae6f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38001800 con 0x7fca4c0a8540 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40004030 con 0x7fca4c0a8540 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca4b7fe700 1 -- 172.21.15.154:0/620318553 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400043c0 con 0x7fca4c0a8540 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca5308e700 1 -- 172.21.15.154:0/620318553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 msgr2=0x7fca4c13a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.945+0000 7fca5308e700 1 --2- 172.21.15.154:0/620318553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 0x7fca4c13a8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca40009840 tx=0x7fca40009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 -- 172.21.15.154:0/620318553 shutdown_connections 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 --2- 172.21.15.154:0/620318553 >> v2:172.21.15.191:3300/0 conn(0x7fca4c0c1780 0x7fca4c0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 --2- 172.21.15.154:0/620318553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0a8540 0x7fca4c13a8a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 -- 172.21.15.154:0/620318553 >> 172.21.15.154:0/620318553 conn(0x7fca4c0c3880 msgr2=0x7fca4c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:05.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 -- 172.21.15.154:0/620318553 shutdown_connections 2024-03-14T09:04:05.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.946+0000 7fca5308e700 1 -- 172.21.15.154:0/620318553 wait complete. 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 Processor -- start 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 -- start start 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 0x7fca4c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 0x7fca4c1e7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1eb620 con 0x7fca4c0a8540 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1e73a0 con 0x7fca4c0c1780 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.947+0000 7fca5308e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1eb0a0 con 0x7fca4c0c3250 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39424/0 (socket says 172.21.15.154:39424) 2024-03-14T09:04:05.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 -- 172.21.15.154:0/4159428145 learned_addr learned my addr 172.21.15.154:0/4159428145 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca4bfff700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 0x7fca4c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca5162b700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 0x7fca4c1e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 -- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 msgr2=0x7fca4c1e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 0x7fca4c1e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 -- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 msgr2=0x7fca4c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 0x7fca4c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 -- 172.21.15.154:0/4159428145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44009a70 con 0x7fca4c0a8540 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca4bfff700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 0x7fca4c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca5162b700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 0x7fca4c1e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca50e2a700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fca400090b0 tx=0x7fca40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4001f070 con 0x7fca4c0a8540 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40003b90 con 0x7fca4c0a8540 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca400094a0 con 0x7fca4c0a8540 2024-03-14T09:04:05.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40011630 con 0x7fca4c0a8540 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.948+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1eb340 con 0x7fca4c0a8540 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.949+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c0697d0 con 0x7fca4c0a8540 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.950+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fca40011790 con 0x7fca4c0a8540 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.950+0000 7fca497fa700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 0x7fca3006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.950+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca4008b4e0 con 0x7fca4c0a8540 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.950+0000 7fca4bfff700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 0x7fca3006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:05.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.951+0000 7fca4bfff700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 0x7fca3006ccf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca38001370 tx=0x7fca38001240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:05.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:05.954+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fca40056990 con 0x7fca4c0a8540 2024-03-14T09:04:06.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.148+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca4c1e7170 con 0x7fca4c0a8540 2024-03-14T09:04:06.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.149+0000 7fca497fa700 1 -- 172.21.15.154:0/4159428145 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16140 (secure 0 0 0) 0x7fca40059fb0 con 0x7fca4c0a8540 2024-03-14T09:04:06.149 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:06.149 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":70,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:03:58.161285+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:06.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 msgr2=0x7fca3006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 0x7fca3006ccf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca38001370 tx=0x7fca38001240 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 msgr2=0x7fca4c12fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fca400090b0 tx=0x7fca40004750 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 shutdown_connections 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c0c3250 0x7fca4c1e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fca3006a830 0x7fca3006ccf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca4c0a8540 0x7fca4c12fb40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 --2- 172.21.15.154:0/4159428145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c0c1780 0x7fca4c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 >> 172.21.15.154:0/4159428145 conn(0x7fca4c0c3880 msgr2=0x7fca4c0c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 shutdown_connections 2024-03-14T09:04:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.151+0000 7fca5308e700 1 -- 172.21.15.154:0/4159428145 wait complete. 2024-03-14T09:04:06.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:06.164 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:06.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.288+0000 7f8ca93af700 1 Processor -- start 2024-03-14T09:04:06.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.288+0000 7f8ca93af700 1 -- start start 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca93af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca93af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 0x7f8ca40c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca93af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca414ed60 con 0x7f8ca40c4a80 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca93af700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca40c6b10 con 0x7f8ca4158a80 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca93af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4155820 con 0x7f8ca414e980 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 0x7f8ca40c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:34616/0 (socket says 172.21.15.154:34616) 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 0x7f8ca40c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:49666/0 (socket says 172.21.15.154:49666) 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca27fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8ca40c4a80 0x7f8ca4158370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:45732/0 (socket says 172.21.15.154:45732) 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.289+0000 7f8ca2ffd700 1 -- 172.21.15.154:0/4022752527 learned_addr learned my addr 172.21.15.154:0/4022752527 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca2ffd700 1 -- 172.21.15.154:0/4022752527 >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 msgr2=0x7f8ca40c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/4022752527 >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 0x7f8ca40c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca2ffd700 1 -- 172.21.15.154:0/4022752527 >> v1:172.21.15.191:6789/0 conn(0x7f8ca40c4a80 legacy=0x7f8ca4158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca2ffd700 1 -- 172.21.15.154:0/4022752527 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40c6cf0 con 0x7f8ca414e980 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/4022752527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098f0 tx=0x7f8c98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2950ce1ccad4d76a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca1ffb700 1 -- 172.21.15.154:0/4022752527 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98011830 con 0x7f8ca414e980 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca1ffb700 1 -- 172.21.15.154:0/4022752527 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98011990 con 0x7f8ca414e980 2024-03-14T09:04:06.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.290+0000 7f8ca1ffb700 1 -- 172.21.15.154:0/4022752527 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98011d20 con 0x7f8ca414e980 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 -- 172.21.15.154:0/4022752527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 msgr2=0x7f8ca41552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 --2- 172.21.15.154:0/4022752527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098f0 tx=0x7f8c98009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 -- 172.21.15.154:0/4022752527 shutdown_connections 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 --2- 172.21.15.154:0/4022752527 >> v2:172.21.15.191:3300/0 conn(0x7f8ca4158a80 0x7f8ca40c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 --2- 172.21.15.154:0/4022752527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca414e980 0x7f8ca41552e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 -- 172.21.15.154:0/4022752527 >> 172.21.15.154:0/4022752527 conn(0x7f8ca40abcf0 msgr2=0x7f8ca40ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 -- 172.21.15.154:0/4022752527 shutdown_connections 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.291+0000 7f8ca93af700 1 -- 172.21.15.154:0/4022752527 wait complete. 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.292+0000 7f8ca93af700 1 Processor -- start 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.292+0000 7f8ca93af700 1 -- start start 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 0x7f8ca40c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 0x7f8ca41edb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40aad80 con 0x7f8ca414e980 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40ab060 con 0x7f8ca40c4a80 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40ab340 con 0x7f8ca4158a80 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39436/0 (socket says 172.21.15.154:39436) 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 -- 172.21.15.154:0/2544841032 learned_addr learned my addr 172.21.15.154:0/2544841032 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca37fe700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 0x7f8ca41edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 0x7f8ca40c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 -- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 msgr2=0x7f8ca41edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 0x7f8ca41edb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 -- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 msgr2=0x7f8ca40c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 0x7f8ca40c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 -- 172.21.15.154:0/2544841032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98009550 con 0x7f8ca414e980 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 0x7f8ca40c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca37fe700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 0x7f8ca41edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca27fc700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c520 tx=0x7f8c9400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94005cb0 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94005e10 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca40ab620 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940072d0 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.293+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca40c3ac0 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.294+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca40abb40 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.295+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8c94007430 con 0x7f8ca414e980 2024-03-14T09:04:06.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.295+0000 7f8c8b7fe700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 0x7f8c9006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.296+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c94088150 con 0x7f8ca414e980 2024-03-14T09:04:06.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.296+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 0x7f8c9006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.296+0000 7f8ca2ffd700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 0x7f8c9006cd20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4157b20 tx=0x7f8c98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.300+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8c94053620 con 0x7f8ca414e980 2024-03-14T09:04:06.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.492+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ca40aaec0 con 0x7f8ca414e980 2024-03-14T09:04:06.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.493+0000 7f8c8b7fe700 1 -- 172.21.15.154:0/2544841032 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ca40aaec0 con 0x7f8ca414e980 2024-03-14T09:04:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 msgr2=0x7f8c9006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 0x7f8c9006cd20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4157b20 tx=0x7f8c98008030 comp rx=0 tx=0).stop 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 msgr2=0x7f8ca40c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c520 tx=0x7f8c9400c8e0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 shutdown_connections 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ca4158a80 0x7f8ca41edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ca40c4a80 0x7f8ca40c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8c9006a860 0x7f8c9006cd20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 --2- 172.21.15.154:0/2544841032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca414e980 0x7f8ca40c1dc0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 >> 172.21.15.154:0/2544841032 conn(0x7f8ca40abcf0 msgr2=0x7f8ca406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 shutdown_connections 2024-03-14T09:04:06.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.495+0000 7f8ca93af700 1 -- 172.21.15.154:0/2544841032 wait complete. 2024-03-14T09:04:06.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:06.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.632+0000 7f714a5bf700 1 Processor -- start 2024-03-14T09:04:06.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.632+0000 7f714a5bf700 1 -- start start 2024-03-14T09:04:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f714a5bf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f714414eaf0 0x7f714406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f714a5bf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f714406acb0 0x7f7144155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f714a5bf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71440ac8e0 con 0x7f7144156330 2024-03-14T09:04:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f714a5bf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714406b310 con 0x7f714414eaf0 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f714a5bf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714406de30 con 0x7f714406acb0 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f7143fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f714414eaf0 0x7f714406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f7148b5c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7144156330 0x7f7144157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52034/0 (socket says 172.21.15.154:52034) 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f7148b5c700 1 -- 172.21.15.154:0/943073029 learned_addr learned my addr 172.21.15.154:0/943073029 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f7143fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f714414eaf0 0x7f714406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39438/0 (socket says 172.21.15.154:39438) 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.633+0000 7f71437fe700 1 --2- 172.21.15.154:0/943073029 >> v2:172.21.15.191:3300/0 conn(0x7f714406acb0 0x7f7144155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7143fff700 1 -- 172.21.15.154:0/943073029 >> v2:172.21.15.191:3300/0 conn(0x7f714406acb0 msgr2=0x7f7144155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7143fff700 1 --2- 172.21.15.154:0/943073029 >> v2:172.21.15.191:3300/0 conn(0x7f714406acb0 0x7f7144155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7143fff700 1 -- 172.21.15.154:0/943073029 >> v1:172.21.15.154:6789/0 conn(0x7f7144156330 legacy=0x7f7144157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7143fff700 1 -- 172.21.15.154:0/943073029 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440acb30 con 0x7f714414eaf0 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7142ffd700 1 -- 172.21.15.154:0/943073029 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843703523 0 0) 0x7f71440ac8e0 con 0x7f7144156330 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7143fff700 1 --2- 172.21.15.154:0/943073029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714414eaf0 0x7f714406a770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f714414f1c0 tx=0x7f7138009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b939e19b293094a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7142ffd700 1 -- 172.21.15.154:0/943073029 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138011830 con 0x7f714414eaf0 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7142ffd700 1 -- 172.21.15.154:0/943073029 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138011990 con 0x7f714414eaf0 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f7142ffd700 1 -- 172.21.15.154:0/943073029 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138011d20 con 0x7f714414eaf0 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 -- 172.21.15.154:0/943073029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714414eaf0 msgr2=0x7f714406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 --2- 172.21.15.154:0/943073029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714414eaf0 0x7f714406a770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f714414f1c0 tx=0x7f7138009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 -- 172.21.15.154:0/943073029 shutdown_connections 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 --2- 172.21.15.154:0/943073029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f714414eaf0 0x7f714406a770 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 --2- 172.21.15.154:0/943073029 >> v2:172.21.15.191:3300/0 conn(0x7f714406acb0 0x7f7144155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 -- 172.21.15.154:0/943073029 >> 172.21.15.154:0/943073029 conn(0x7f71440c3880 msgr2=0x7f71440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.634+0000 7f714a5bf700 1 -- 172.21.15.154:0/943073029 shutdown_connections 2024-03-14T09:04:06.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.635+0000 7f714a5bf700 1 -- 172.21.15.154:0/943073029 wait complete. 2024-03-14T09:04:06.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 Processor -- start 2024-03-14T09:04:06.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 -- start start 2024-03-14T09:04:06.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714406acb0 0x7f7144154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714414eaf0 0x7f7144143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71441eabe0 con 0x7f7144156330 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71441549c0 con 0x7f714414eaf0 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.636+0000 7f714a5bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7144141100 con 0x7f714406acb0 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39444/0 (socket says 172.21.15.154:39444) 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 -- 172.21.15.154:0/3573788488 learned_addr learned my addr 172.21.15.154:0/3573788488 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7143fff700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714406acb0 0x7f7144154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f71437fe700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714414eaf0 0x7f7144143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 -- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714406acb0 msgr2=0x7f7144154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714406acb0 0x7f7144154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 -- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714414eaf0 msgr2=0x7f7144143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714414eaf0 0x7f7144143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 -- 172.21.15.154:0/3573788488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380094a0 con 0x7f7144156330 2024-03-14T09:04:06.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7148b5c700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f713c002230 tx=0x7f713c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00ac20 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00ada0 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71441413e0 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c007310 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.637+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7144152860 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.638+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71440697d0 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.639+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f713c007470 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.639+0000 7f7140ff9700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 0x7f713006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.639+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f713c088ba0 con 0x7f7144156330 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.639+0000 7f7143fff700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 0x7f713006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.640+0000 7f7143fff700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 0x7f713006ccf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f71380090b0 tx=0x7f7138008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.644+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f713c054050 con 0x7f7144156330 2024-03-14T09:04:06.835 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.835+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f714406b470 con 0x7f7144156330 2024-03-14T09:04:06.836 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.836+0000 7f7140ff9700 1 -- 172.21.15.154:0/3573788488 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f713c0180a0 con 0x7f7144156330 2024-03-14T09:04:06.836 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:06.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 msgr2=0x7f713006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 0x7f713006ccf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f71380090b0 tx=0x7f7138008030 comp rx=0 tx=0).stop 2024-03-14T09:04:06.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 msgr2=0x7f714413b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f713c002230 tx=0x7f713c00a060 comp rx=0 tx=0).stop 2024-03-14T09:04:06.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 shutdown_connections 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f713006a830 0x7f713006ccf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7144156330 0x7f714413b0f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714406acb0 0x7f7144154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 --2- 172.21.15.154:0/3573788488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714414eaf0 0x7f7144143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 >> 172.21.15.154:0/3573788488 conn(0x7f71440c3880 msgr2=0x7f7144148e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 shutdown_connections 2024-03-14T09:04:06.839 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.838+0000 7f714a5bf700 1 -- 172.21.15.154:0/3573788488 wait complete. 2024-03-14T09:04:06.850 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:06.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.974+0000 7f8f00b16700 1 Processor -- start 2024-03-14T09:04:06.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.974+0000 7f8f00b16700 1 -- start start 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8f00b16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8f00b16700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8efc0c3250 0x7f8efc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8f00b16700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc13ade0 con 0x7f8efc1550b0 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8f00b16700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc06fb30 con 0x7f8efc0c3250 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8f00b16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc06ec30 con 0x7f8efc0a8540 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.975+0000 7f8ef9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8efc0c3250 0x7f8efc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8efad9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8efc1550b0 0x7f8efc0c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52044/0 (socket says 172.21.15.154:52044) 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8efc0c3250 0x7f8efc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39454/0 (socket says 172.21.15.154:39454) 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8efad9d700 1 -- 172.21.15.154:0/3629673626 learned_addr learned my addr 172.21.15.154:0/3629673626 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:06.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8efa59c700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 -- 172.21.15.154:0/3629673626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 msgr2=0x7f8efc13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 -- 172.21.15.154:0/3629673626 >> v1:172.21.15.154:6789/0 conn(0x7f8efc1550b0 legacy=0x7f8efc0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef959a700 1 -- 172.21.15.154:0/3629673626 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207600368 0 0) 0x7f8ef4001490 con 0x7f8efc1550b0 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 -- 172.21.15.154:0/3629673626 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc06ef10 con 0x7f8efc0c3250 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8efa59c700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef9d9b700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc0c3250 0x7f8efc154b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bae0 tx=0x7f8eec00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c75a365cc9f80dd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef959a700 1 -- 172.21.15.154:0/3629673626 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00dd30 con 0x7f8efc0c3250 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef959a700 1 -- 172.21.15.154:0/3629673626 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec00de90 con 0x7f8efc0c3250 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.976+0000 7f8ef959a700 1 -- 172.21.15.154:0/3629673626 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec010690 con 0x7f8efc0c3250 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 -- 172.21.15.154:0/3629673626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc0c3250 msgr2=0x7f8efc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc0c3250 0x7f8efc154b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00bae0 tx=0x7f8eec00bdf0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 -- 172.21.15.154:0/3629673626 shutdown_connections 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 --2- 172.21.15.154:0/3629673626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc0c3250 0x7f8efc154b70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 -- 172.21.15.154:0/3629673626 >> 172.21.15.154:0/3629673626 conn(0x7f8efc0c3880 msgr2=0x7f8efc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:06.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 -- 172.21.15.154:0/3629673626 shutdown_connections 2024-03-14T09:04:06.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.977+0000 7f8f00b16700 1 -- 172.21.15.154:0/3629673626 wait complete. 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.978+0000 7f8f00b16700 1 Processor -- start 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.978+0000 7f8f00b16700 1 -- start start 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc0c3250 0x7f8efc14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1db580 con 0x7f8efc1550b0 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc14f010 con 0x7f8efc0c3250 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc134220 con 0x7f8efc0a8540 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39458/0 (socket says 172.21.15.154:39458) 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 -- 172.21.15.154:0/2320337431 learned_addr learned my addr 172.21.15.154:0/2320337431 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efa59c700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.982 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8ef9d9b700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc0c3250 0x7f8efc14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 -- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 msgr2=0x7f8efc0c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc0c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 -- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc0c3250 msgr2=0x7f8efc14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc0c3250 0x7f8efc14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 -- 172.21.15.154:0/2320337431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec00b740 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8efad9d700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400cba0 tx=0x7f8ef400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400ab80 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400ad00 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4007310 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.979+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc0ac0d0 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.980+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc0c0a60 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.980+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc1db6c0 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.981+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8ef4007470 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.981+0000 7f8ef2ffd700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 0x7f8edc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.981+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ef40886d0 con 0x7f8efc1550b0 2024-03-14T09:04:06.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.982+0000 7f8efa59c700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 0x7f8edc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:06.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.982+0000 7f8efa59c700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 0x7f8edc06ccf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:06.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:06.986+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8efc1db6c0 con 0x7f8efc1550b0 2024-03-14T09:04:07.178 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.178+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8efc13ade0 con 0x7f8efc1550b0 2024-03-14T09:04:07.179 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.179+0000 7f8ef2ffd700 1 -- 172.21.15.154:0/2320337431 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8efc13ade0 con 0x7f8efc1550b0 2024-03-14T09:04:07.179 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 msgr2=0x7f8edc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 0x7f8edc06ccf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4006fd0 tx=0x7f8ee4008040 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 msgr2=0x7f8efc14ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400cba0 tx=0x7f8ef400cf60 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 shutdown_connections 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8edc06a830 0x7f8edc06ccf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8efc1550b0 0x7f8efc14ead0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8efc0a8540 0x7f8efc0c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 --2- 172.21.15.154:0/2320337431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8efc0c3250 0x7f8efc14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 >> 172.21.15.154:0/2320337431 conn(0x7f8efc0c3880 msgr2=0x7f8efc138b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:07.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 shutdown_connections 2024-03-14T09:04:07.182 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.181+0000 7f8f00b16700 1 -- 172.21.15.154:0/2320337431 wait complete. 2024-03-14T09:04:07.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-14T09:04:07.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.314+0000 7eff963fe700 1 Processor -- start 2024-03-14T09:04:07.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.314+0000 7eff963fe700 1 -- start start 2024-03-14T09:04:07.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff963fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff9014eaf0 0x7eff9006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff963fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7eff9006acb0 0x7eff90155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff963fe700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff900ac8e0 con 0x7eff90156330 2024-03-14T09:04:07.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff963fe700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9006b310 con 0x7eff9014eaf0 2024-03-14T09:04:07.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff963fe700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9006de30 con 0x7eff9006acb0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff9014eaf0 0x7eff9006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff9499b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7eff90156330 0x7eff90157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52048/0 (socket says 172.21.15.154:52048) 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff9499b700 1 -- 172.21.15.154:0/1563413605 learned_addr learned my addr 172.21.15.154:0/1563413605 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8f7fe700 1 --2- 172.21.15.154:0/1563413605 >> v2:172.21.15.191:3300/0 conn(0x7eff9006acb0 0x7eff90155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 -- 172.21.15.154:0/1563413605 >> v2:172.21.15.191:3300/0 conn(0x7eff9006acb0 msgr2=0x7eff90155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 --2- 172.21.15.154:0/1563413605 >> v2:172.21.15.191:3300/0 conn(0x7eff9006acb0 0x7eff90155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 -- 172.21.15.154:0/1563413605 >> v1:172.21.15.154:6789/0 conn(0x7eff90156330 legacy=0x7eff90157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 -- 172.21.15.154:0/1563413605 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900acb30 con 0x7eff9014eaf0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8effd700 1 -- 172.21.15.154:0/1563413605 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986691821 0 0) 0x7eff900ac8e0 con 0x7eff90156330 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8ffff700 1 --2- 172.21.15.154:0/1563413605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9014eaf0 0x7eff9006a770 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7eff9014f1c0 tx=0x7eff84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42883b4cf1df41 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8effd700 1 -- 172.21.15.154:0/1563413605 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84011830 con 0x7eff9014eaf0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8effd700 1 -- 172.21.15.154:0/1563413605 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff84011990 con 0x7eff9014eaf0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.315+0000 7eff8effd700 1 -- 172.21.15.154:0/1563413605 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84011d20 con 0x7eff9014eaf0 2024-03-14T09:04:07.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 -- 172.21.15.154:0/1563413605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9014eaf0 msgr2=0x7eff9006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 --2- 172.21.15.154:0/1563413605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9014eaf0 0x7eff9006a770 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7eff9014f1c0 tx=0x7eff84009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 -- 172.21.15.154:0/1563413605 shutdown_connections 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 --2- 172.21.15.154:0/1563413605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9014eaf0 0x7eff9006a770 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 --2- 172.21.15.154:0/1563413605 >> v2:172.21.15.191:3300/0 conn(0x7eff9006acb0 0x7eff90155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 -- 172.21.15.154:0/1563413605 >> 172.21.15.154:0/1563413605 conn(0x7eff900c3880 msgr2=0x7eff900c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 -- 172.21.15.154:0/1563413605 shutdown_connections 2024-03-14T09:04:07.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.316+0000 7eff963fe700 1 -- 172.21.15.154:0/1563413605 wait complete. 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.317+0000 7eff963fe700 1 Processor -- start 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.317+0000 7eff963fe700 1 -- start start 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 0x7eff9013b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90156330 0x7eff9013b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff901ee270 con 0x7eff9006acb0 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff901ed660 con 0x7eff90156330 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff963fe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff901ed940 con 0x7eff9014eaf0 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39474/0 (socket says 172.21.15.154:39474) 2024-03-14T09:04:07.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 -- 172.21.15.154:0/3142428810 learned_addr learned my addr 172.21.15.154:0/3142428810 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8f7fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 0x7eff9013b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff9499b700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90156330 0x7eff9013b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 -- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 msgr2=0x7eff9013b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 0x7eff9013b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 -- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90156330 msgr2=0x7eff9013b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90156330 0x7eff9013b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8ffff700 1 -- 172.21.15.154:0/3142428810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff840094a0 con 0x7eff9006acb0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.318+0000 7eff8f7fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 0x7eff9013b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff8ffff700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7eff840090b0 tx=0x7eff84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff840038a0 con 0x7eff9006acb0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8401f070 con 0x7eff9006acb0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84003c20 con 0x7eff9006acb0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff901edc20 con 0x7eff9006acb0 2024-03-14T09:04:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.319+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9013bec0 con 0x7eff9006acb0 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.320+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff900697d0 con 0x7eff9006acb0 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.320+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7eff8401a430 con 0x7eff9006acb0 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.321+0000 7eff8cff9700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 0x7eff7c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.321+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff84021070 con 0x7eff9006acb0 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.321+0000 7eff8f7fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 0x7eff7c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.321+0000 7eff8f7fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 0x7eff7c06ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7eff80009ac0 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.325+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7eff84055a60 con 0x7eff9006acb0 2024-03-14T09:04:07.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.522+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff9006b470 con 0x7eff9006acb0 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.522+0000 7eff8cff9700 1 -- 172.21.15.154:0/3142428810 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+209 (secure 0 0 0) 0x7eff840150f0 con 0x7eff9006acb0 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2024-03-14T09:04:07.522 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 12, 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_metadata" 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 13, 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "cephfs_data" 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-03-14T09:04:07.523 INFO:teuthology.orchestra.run.smithi154.stdout:] 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.524+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 msgr2=0x7eff7c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.524+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 0x7eff7c06ccf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7eff80009ac0 tx=0x7eff80008040 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.524+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 msgr2=0x7eff90144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.524+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7eff840090b0 tx=0x7eff84004750 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 shutdown_connections 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff90156330 0x7eff9013b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7eff7c06a830 0x7eff7c06ccf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff9006acb0 0x7eff90144320 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 --2- 172.21.15.154:0/3142428810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff9014eaf0 0x7eff9013b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 >> 172.21.15.154:0/3142428810 conn(0x7eff900c3880 msgr2=0x7eff901e11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 shutdown_connections 2024-03-14T09:04:07.525 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.525+0000 7eff963fe700 1 -- 172.21.15.154:0/3142428810 wait complete. 2024-03-14T09:04:07.537 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-14T09:04:07.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-14T09:04:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.662+0000 7f76b3708700 1 Processor -- start 2024-03-14T09:04:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 -- start start 2024-03-14T09:04:07.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f76ac0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f76ac0c2d20 0x7f76ac0c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac156950 con 0x7f76ac0ada40 2024-03-14T09:04:07.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0c4820 con 0x7f76ac0c2d20 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b3708700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0c7800 con 0x7f76ac0c4440 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b0ca3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f76ac0ada40 0x7f76ac06e1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52050/0 (socket says 172.21.15.154:52050) 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.663+0000 7f76b0ca3700 1 -- 172.21.15.154:0/982421138 learned_addr learned my addr 172.21.15.154:0/982421138 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> v2:172.21.15.191:3300/0 conn(0x7f76ac0c2d20 0x7f76ac0c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b14a4700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f76ac0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945239553 0 0) 0x7f76ac156950 con 0x7f76ac0ada40 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76980035b0 con 0x7f76ac0ada40 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2411445685 0 0) 0x7f76980035b0 con 0x7f76ac0ada40 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 msgr2=0x7f76ac0705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f76ac0705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 >> v2:172.21.15.191:3300/0 conn(0x7f76ac0c2d20 msgr2=0x7f76ac0c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 --2- 172.21.15.154:0/982421138 >> v2:172.21.15.191:3300/0 conn(0x7f76ac0c2d20 0x7f76ac0c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac070af0 con 0x7f76ac0ada40 2024-03-14T09:04:07.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f76a8001670 con 0x7f76ac0ada40 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> v2:172.21.15.191:3300/0 conn(0x7f76ac0c2d20 0x7f76ac0c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 >> v1:172.21.15.154:6789/0 conn(0x7f76ac0ada40 legacy=0x7f76ac06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7698003a30 0x7f7698003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f7698004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7698004d90 0x7f7698005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0c4820 con 0x7f7698004d90 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769800ba10 con 0x7f7698003a30 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769800bb50 con 0x7f76ac0c4440 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7698004d90 0x7f7698005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b14a4700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7698003a30 0x7f7698003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b0ca3700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f7698004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 -- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 msgr2=0x7f7698004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac0c4440 0x7f7698004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 -- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7698003a30 msgr2=0x7f7698003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7698003a30 0x7f7698003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.664+0000 7f76b1ca5700 1 -- 172.21.15.154:0/982421138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980035b0 con 0x7f7698004d90 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7698004d90 0x7f7698005210 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f76a400e380 tx=0x7f7698005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff45da94890f460e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a400ee20 con 0x7f7698004d90 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a400f040 con 0x7f7698004d90 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76a3fff700 1 -- 172.21.15.154:0/982421138 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a40146a0 con 0x7f7698004d90 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 -- 172.21.15.154:0/982421138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7698004d90 msgr2=0x7f7698005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 --2- 172.21.15.154:0/982421138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7698004d90 0x7f7698005210 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f76a400e380 tx=0x7f7698005880 comp rx=0 tx=0).stop 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b1ca5700 1 -- 172.21.15.154:0/982421138 reap_dead start 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 -- 172.21.15.154:0/982421138 shutdown_connections 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 -- 172.21.15.154:0/982421138 >> 172.21.15.154:0/982421138 conn(0x7f76ac0a9d00 msgr2=0x7f76ac0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 -- 172.21.15.154:0/982421138 shutdown_connections 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.665+0000 7f76b3708700 1 -- 172.21.15.154:0/982421138 wait complete. 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 Processor -- start 2024-03-14T09:04:07.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 -- start start 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 0x7f76ac1eb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76ac0c21d0 0x7f76ac144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac152d40 con 0x7f76ac1ebb90 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac155d60 con 0x7f76ac0c21d0 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b3708700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac153510 con 0x7f76ac1eb270 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b0ca3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b0ca3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39500/0 (socket says 172.21.15.154:39500) 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b0ca3700 1 -- 172.21.15.154:0/2096092802 learned_addr learned my addr 172.21.15.154:0/2096092802 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b1ca5700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76ac0c21d0 0x7f76ac144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.667+0000 7f76b14a4700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 0x7f76ac1eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 -- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 msgr2=0x7f76ac1eb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 0x7f76ac1eb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 -- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76ac0c21d0 msgr2=0x7f76ac144570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76ac0c21d0 0x7f76ac144570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 -- 172.21.15.154:0/2096092802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac152f50 con 0x7f76ac1ebb90 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b14a4700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 0x7f76ac1eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b0ca3700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f76a80011e0 tx=0x7f76a8007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800f660 con 0x7f76ac1ebb90 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a800f800 con 0x7f76ac1ebb90 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac153230 con 0x7f76ac1ebb90 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800fbb0 con 0x7f76ac1ebb90 2024-03-14T09:04:07.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.668+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac144ab0 con 0x7f76ac1ebb90 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.669+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac0697d0 con 0x7f76ac1ebb90 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.669+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f76a800fd10 con 0x7f76ac1ebb90 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.670+0000 7f76a1ffb700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 0x7f769006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.670+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76a8088db0 con 0x7f76ac1ebb90 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.670+0000 7f76b14a4700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 0x7f769006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.670+0000 7f76b14a4700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 0x7f769006ccf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f769c009c60 tx=0x7f769c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:07.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.674+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f76a80541d0 con 0x7f76ac1ebb90 2024-03-14T09:04:07.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:07.866+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76ac0c2b00 con 0x7f76ac1ebb90 2024-03-14T09:04:08.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.356+0000 7f76a1ffb700 1 -- 172.21.15.154:0/2096092802 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v71) v1 ==== 140+0+0 (secure 0 0 0) 0x7f76a80577f0 con 0x7f76ac1ebb90 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.360+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 msgr2=0x7f769006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.360+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 0x7f769006ccf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f769c009c60 tx=0x7f769c008040 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.360+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 msgr2=0x7f76ac144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.360+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f76a80011e0 tx=0x7f76a8007570 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 shutdown_connections 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76ac0c21d0 0x7f76ac144570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76ac1eb270 0x7f76ac1eb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f769006a830 0x7f769006ccf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 --2- 172.21.15.154:0/2096092802 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76ac1ebb90 0x7f76ac144030 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 >> 172.21.15.154:0/2096092802 conn(0x7f76ac0a9d00 msgr2=0x7f76ac14e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 shutdown_connections 2024-03-14T09:04:08.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.361+0000 7f76b3708700 1 -- 172.21.15.154:0/2096092802 wait complete. 2024-03-14T09:04:08.362 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' created 2024-03-14T09:04:08.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-14T09:04:08.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.510+0000 7f285b95d700 1 Processor -- start 2024-03-14T09:04:08.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.511+0000 7f285b95d700 1 -- start start 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.511+0000 7f285b95d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f28540c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.511+0000 7f285b95d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540acd10 con 0x7f28540c5d60 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.511+0000 7f285b95d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2854150050 con 0x7f285414fbd0 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.511+0000 7f285b95d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540ac7b0 con 0x7f28540702f0 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f2859efa700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f28540c5d60 0x7f285415cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52066/0 (socket says 172.21.15.154:52066) 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f2859efa700 1 -- 172.21.15.154:0/3257841510 learned_addr learned my addr 172.21.15.154:0/3257841510 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713246539 0 0) 0x7f28540acd10 con 0x7f28540c5d60 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28400035b0 con 0x7f28540c5d60 2024-03-14T09:04:08.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f28596f9700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f28540c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2955386115 0 0) 0x7f28400035b0 con 0x7f28540c5d60 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 msgr2=0x7f28540c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f28540c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 >> v1:172.21.15.191:6789/0 conn(0x7f285414fbd0 legacy=0x7f28540c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540ac9c0 con 0x7f28540c5d60 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f284c0019f0 con 0x7f28540c5d60 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 >> v1:172.21.15.154:6789/0 conn(0x7f28540c5d60 legacy=0x7f285415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2840003a30 0x7f2840003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840004850 0x7f2840004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f2840005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540ac7b0 con 0x7f2840003a30 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284000bac0 con 0x7f2840004850 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284000bc80 con 0x7f28540702f0 2024-03-14T09:04:08.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f28596f9700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2840003a30 0x7f2840003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f2859efa700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f2840005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.512+0000 7f2858ef8700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840004850 0x7f2840004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 -- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 msgr2=0x7f2840005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28540702f0 0x7f2840005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 -- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840004850 msgr2=0x7f2840004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840004850 0x7f2840004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 -- 172.21.15.154:0/3257841510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400035b0 con 0x7f2840003a30 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f28596f9700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2840003a30 0x7f2840003e50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2844009b30 tx=0x7f2840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5bfb6dd0416b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400e6c0 con 0x7f2840003a30 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400e820 con 0x7f2840003a30 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f284bfff700 1 -- 172.21.15.154:0/3257841510 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400eb90 con 0x7f2840003a30 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f285b95d700 1 -- 172.21.15.154:0/3257841510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2840003a30 msgr2=0x7f2840003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f285b95d700 1 --2- 172.21.15.154:0/3257841510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2840003a30 0x7f2840003e50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2844009b30 tx=0x7f2840005880 comp rx=0 tx=0).stop 2024-03-14T09:04:08.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.513+0000 7f2859efa700 1 -- 172.21.15.154:0/3257841510 reap_dead start 2024-03-14T09:04:08.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.514+0000 7f285b95d700 1 -- 172.21.15.154:0/3257841510 shutdown_connections 2024-03-14T09:04:08.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.514+0000 7f285b95d700 1 -- 172.21.15.154:0/3257841510 >> 172.21.15.154:0/3257841510 conn(0x7f28540a9bf0 msgr2=0x7f28540a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:08.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.514+0000 7f285b95d700 1 -- 172.21.15.154:0/3257841510 shutdown_connections 2024-03-14T09:04:08.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.514+0000 7f285b95d700 1 -- 172.21.15.154:0/3257841510 wait complete. 2024-03-14T09:04:08.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 Processor -- start 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 -- start start 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840003a30 0x7f28541dd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 0x7f285414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854144c60 con 0x7f285414cc30 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28541dcc90 con 0x7f2840003a30 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.515+0000 7f285b95d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28541dd660 con 0x7f2854145190 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:39516/0 (socket says 172.21.15.154:39516) 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 -- 172.21.15.154:0/1196336505 learned_addr learned my addr 172.21.15.154:0/1196336505 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2858ef8700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 0x7f285414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f28596f9700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840003a30 0x7f28541dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 -- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 msgr2=0x7f285414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 0x7f285414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 -- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840003a30 msgr2=0x7f28541dd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840003a30 0x7f28541dd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 -- 172.21.15.154:0/1196336505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854144eb0 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2858ef8700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 0x7f285414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2859efa700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f28540aac90 tx=0x7f284c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c00ae50 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c0133f0 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28541dc640 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c0137c0 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.516+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285414a8e0 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.517+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28541dce00 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.518+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f284c013920 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.518+0000 7f2849ffb700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 0x7f283806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.518+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f284c089b50 con 0x7f285414cc30 2024-03-14T09:04:08.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.518+0000 7f28596f9700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 0x7f283806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:08.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.519+0000 7f28596f9700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 0x7f283806ccf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f28440090b0 tx=0x7f2844008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:08.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.523+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f284c054e70 con 0x7f285414cc30 2024-03-14T09:04:08.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:08.721+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f28540697d0 con 0x7f285414cc30 2024-03-14T09:04:09.358 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.359+0000 7f2849ffb700 1 -- 172.21.15.154:0/1196336505 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v72) v1 ==== 172+0+0 (secure 0 0 0) 0x7f284c058490 con 0x7f285414cc30 2024-03-14T09:04:09.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 msgr2=0x7f283806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 0x7f283806ccf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f28440090b0 tx=0x7f2844008030 comp rx=0 tx=0).stop 2024-03-14T09:04:09.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 msgr2=0x7f28541de190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f28540aac90 tx=0x7f284c00a620 comp rx=0 tx=0).stop 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 shutdown_connections 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f283806a830 0x7f283806ccf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f285414cc30 0x7f28541de190 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2840003a30 0x7f28541dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 --2- 172.21.15.154:0/1196336505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2854145190 0x7f285414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.367+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 >> 172.21.15.154:0/1196336505 conn(0x7f28540a9bf0 msgr2=0x7f28540c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:09.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.368+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 shutdown_connections 2024-03-14T09:04:09.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.368+0000 7f285b95d700 1 -- 172.21.15.154:0/1196336505 wait complete. 2024-03-14T09:04:09.373 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' created 2024-03-14T09:04:09.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-14T09:04:09.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.515+0000 7f00cc293700 1 Processor -- start 2024-03-14T09:04:09.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.515+0000 7f00cc293700 1 -- start start 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00cc293700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c414b290 0x7f00c414b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00cc293700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00c4152d80 0x7f00c4155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00cc293700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c4153210 con 0x7f00c4152900 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00cc293700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c414bbb0 con 0x7f00c4152d80 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00cc293700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c406cff0 con 0x7f00c414b290 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00c4152d80 0x7f00c4155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00c982e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f00c4152900 0x7f00c4154c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43418/0 (socket says 172.21.15.154:43418) 2024-03-14T09:04:09.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00c982e700 1 -- 172.21.15.154:0/3224122941 learned_addr learned my addr 172.21.15.154:0/3224122941 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca02f700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c414b290 0x7f00c414b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 -- 172.21.15.154:0/3224122941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c414b290 msgr2=0x7f00c414b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c414b290 0x7f00c414b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 -- 172.21.15.154:0/3224122941 >> v1:172.21.15.154:6789/0 conn(0x7f00c4152900 legacy=0x7f00c4154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 -- 172.21.15.154:0/3224122941 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c406d2d0 con 0x7f00c4152d80 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00c902d700 1 -- 172.21.15.154:0/3224122941 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351679317 0 0) 0x7f00c4153210 con 0x7f00c4152900 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00ca830700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152d80 0x7f00c4155350 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009d90 tx=0x7f00bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c250dd818cbca3be server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.516+0000 7f00c902d700 1 -- 172.21.15.154:0/3224122941 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00e070 con 0x7f00c4152d80 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00c902d700 1 -- 172.21.15.154:0/3224122941 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc00bbd0 con 0x7f00c4152d80 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00c902d700 1 -- 172.21.15.154:0/3224122941 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc012430 con 0x7f00c4152d80 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 -- 172.21.15.154:0/3224122941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152d80 msgr2=0x7f00c4155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152d80 0x7f00c4155350 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009d90 tx=0x7f00bc00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 -- 172.21.15.154:0/3224122941 shutdown_connections 2024-03-14T09:04:09.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152d80 0x7f00c4155350 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 --2- 172.21.15.154:0/3224122941 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c414b290 0x7f00c414b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 -- 172.21.15.154:0/3224122941 >> 172.21.15.154:0/3224122941 conn(0x7f00c40c5e70 msgr2=0x7f00c40c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:09.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 -- 172.21.15.154:0/3224122941 shutdown_connections 2024-03-14T09:04:09.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.517+0000 7f00cc293700 1 -- 172.21.15.154:0/3224122941 wait complete. 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.518+0000 7f00cc293700 1 Processor -- start 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.518+0000 7f00cc293700 1 -- start start 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00c414b290 0x7f00c40c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c4152d80 0x7f00c4140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4149100 con 0x7f00c4152900 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4143d90 con 0x7f00c414b290 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00cc293700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c414a500 con 0x7f00c4152d80 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43452/0 (socket says 172.21.15.154:43452) 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 -- 172.21.15.154:0/2880775976 learned_addr learned my addr 172.21.15.154:0/2880775976 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00ca830700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c4152d80 0x7f00c4140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00ca02f700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00c414b290 0x7f00c40c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 -- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c4152d80 msgr2=0x7f00c4140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c4152d80 0x7f00c4140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 -- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00c414b290 msgr2=0x7f00c40c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00c414b290 0x7f00c40c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.519+0000 7f00c982e700 1 -- 172.21.15.154:0/2880775976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009a70 con 0x7f00c4152900 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00c982e700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f00b40011e0 tx=0x7f00b400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004c80 con 0x7f00c4152900 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4004e00 con 0x7f00c4152900 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c4143fe0 con 0x7f00c4152900 2024-03-14T09:04:09.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4007310 con 0x7f00c4152900 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.520+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c40c3650 con 0x7f00c4152900 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.521+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c4067bd0 con 0x7f00c4152900 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.521+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f00b4007470 con 0x7f00c4152900 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.522+0000 7f00baffd700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 0x7f00a806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.522+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6492+0+0 (secure 0 0 0) 0x7f00b4088be0 con 0x7f00c4152900 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.522+0000 7f00ca02f700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 0x7f00a806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.522+0000 7f00ca02f700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 0x7f00a806ccf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f00c0006b20 tx=0x7f00c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.526 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.526+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f00b4053d80 con 0x7f00c4152900 2024-03-14T09:04:09.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.766+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f00c4073f10 con 0x7f00c4152900 2024-03-14T09:04:09.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.772+0000 7f00baffd700 1 -- 172.21.15.154:0/2880775976 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 14 and data pool 15 v46) v1 ==== 177+0+0 (secure 0 0 0) 0x7f00b40573a0 con 0x7f00c4152900 2024-03-14T09:04:09.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 msgr2=0x7f00a806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 0x7f00a806ccf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f00c0006b20 tx=0x7f00c0008040 comp rx=0 tx=0).stop 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 msgr2=0x7f00c41406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f00b40011e0 tx=0x7f00b400ca30 comp rx=0 tx=0).stop 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 shutdown_connections 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00c4152d80 0x7f00c4140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00c414b290 0x7f00c40c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f00a806a830 0x7f00a806ccf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 --2- 172.21.15.154:0/2880775976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00c4152900 0x7f00c41406a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 >> 172.21.15.154:0/2880775976 conn(0x7f00c40c5e70 msgr2=0x7f00c40c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:09.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 shutdown_connections 2024-03-14T09:04:09.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.783+0000 7f00cc293700 1 -- 172.21.15.154:0/2880775976 wait complete. 2024-03-14T09:04:09.793 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 14 and data pool 15 2024-03-14T09:04:09.805 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:09.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.926+0000 7f94fe00d700 1 Processor -- start 2024-03-14T09:04:09.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.926+0000 7f94fe00d700 1 -- start start 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94fe00d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f94f8155df0 0x7f94f8156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94fe00d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94f815f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94fe00d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f80ac8e0 con 0x7f94f814eaf0 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94fe00d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f806b310 con 0x7f94f8155df0 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94fe00d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f806de30 con 0x7f94f815d270 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94f77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f94f814eaf0 0x7f94f806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43432/0 (socket says 172.21.15.154:43432) 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.927+0000 7f94f77fe700 1 -- 172.21.15.154:0/1236872665 learned_addr learned my addr 172.21.15.154:0/1236872665 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692072017 0 0) 0x7f94f80ac8e0 con 0x7f94f814eaf0 2024-03-14T09:04:09.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e40035b0 con 0x7f94f814eaf0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f7fff700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94f815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f6ffd700 1 --2- 172.21.15.154:0/1236872665 >> v2:172.21.15.191:3300/0 conn(0x7f94f8155df0 0x7f94f8156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584986258 0 0) 0x7f94e40035b0 con 0x7f94f814eaf0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 msgr2=0x7f94f815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94f815f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 >> v2:172.21.15.191:3300/0 conn(0x7f94f8155df0 msgr2=0x7f94f8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 --2- 172.21.15.154:0/1236872665 >> v2:172.21.15.191:3300/0 conn(0x7f94f8155df0 0x7f94f8156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80acb30 con 0x7f94f814eaf0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f94e8001670 con 0x7f94f814eaf0 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 >> v1:172.21.15.154:6789/0 conn(0x7f94f814eaf0 legacy=0x7f94f806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94e4004850 0x7f94e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f806b310 con 0x7f94e4003a00 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f806de30 con 0x7f94e4004850 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e400ba10 con 0x7f94f815d270 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f6ffd700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94e4004850 0x7f94e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f7fff700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 -- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 msgr2=0x7f94e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815d270 0x7f94e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 -- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94e4004850 msgr2=0x7f94e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94e4004850 0x7f94e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.928+0000 7f94f77fe700 1 -- 172.21.15.154:0/1236872665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40035b0 con 0x7f94e4003a00 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94f77fe700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94e4003de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001eb0 tx=0x7f94e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e26a1cddc2fa8c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8005b60 con 0x7f94e4003a00 2024-03-14T09:04:09.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8005cc0 con 0x7f94e4003a00 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94f67fc700 1 -- 172.21.15.154:0/1236872665 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8008310 con 0x7f94e4003a00 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 -- 172.21.15.154:0/1236872665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 msgr2=0x7f94e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 --2- 172.21.15.154:0/1236872665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94e4003de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f94e8001eb0 tx=0x7f94e4005880 comp rx=0 tx=0).stop 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94f7fff700 1 -- 172.21.15.154:0/1236872665 reap_dead start 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 -- 172.21.15.154:0/1236872665 shutdown_connections 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 -- 172.21.15.154:0/1236872665 >> 172.21.15.154:0/1236872665 conn(0x7f94f80c3880 msgr2=0x7f94f80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 -- 172.21.15.154:0/1236872665 shutdown_connections 2024-03-14T09:04:09.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.929+0000 7f94fe00d700 1 -- 172.21.15.154:0/1236872665 wait complete. 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.931+0000 7f94fe00d700 1 Processor -- start 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.931+0000 7f94fe00d700 1 -- start start 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f8154450 0x7f94f80c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 0x7f94f813b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f813c950 con 0x7f94e4003a00 2024-03-14T09:04:09.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81ee2d0 con 0x7f94f8154450 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94fe00d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f815b650 con 0x7f94f815be50 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43468/0 (socket says 172.21.15.154:43468) 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 -- 172.21.15.154:0/2137822510 learned_addr learned my addr 172.21.15.154:0/2137822510 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f6ffd700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f8154450 0x7f94f80c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f7fff700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 0x7f94f813b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 -- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 msgr2=0x7f94f813b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 0x7f94f813b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 -- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f8154450 msgr2=0x7f94f80c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f8154450 0x7f94f80c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 -- 172.21.15.154:0/2137822510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f815b930 con 0x7f94e4003a00 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f7fff700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 0x7f94f813b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94f77fe700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f94e4004750 tx=0x7f94e8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.932+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8025070 con 0x7f94e4003a00 2024-03-14T09:04:09.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.933+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8005e30 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.933+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f81ee5b0 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.933+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8001640 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.933+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f80c19b0 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.933+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f8067970 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.934+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f94e80080d0 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.934+0000 7f94effff700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 0x7f94d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.934+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f94e8089e90 con 0x7f94e4003a00 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.935+0000 7f94f6ffd700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 0x7f94d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.935+0000 7f94f6ffd700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 0x7f94d806ccf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f806d640 tx=0x7f94e0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:09.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:09.939+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f94e8055070 con 0x7f94e4003a00 2024-03-14T09:04:10.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.135+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f80697d0 con 0x7f94e4003a00 2024-03-14T09:04:10.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.136+0000 7f94effff700 1 -- 172.21.15.154:0/2137822510 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+19857 (secure 0 0 0) 0x7f94e8058690 con 0x7f94e4003a00 2024-03-14T09:04:10.136 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:10.136 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":73,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:09.768099+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:10.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 msgr2=0x7f94d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 0x7f94d806ccf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f94f806d640 tx=0x7f94e0009210 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 msgr2=0x7f94f81eda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f94e4004750 tx=0x7f94e8004a90 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 shutdown_connections 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94f815be50 0x7f94f813b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f94d806a830 0x7f94d806ccf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f94e4003a00 0x7f94f81eda00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 --2- 172.21.15.154:0/2137822510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94f8154450 0x7f94f80c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 >> 172.21.15.154:0/2137822510 conn(0x7f94f80c3880 msgr2=0x7f94f813f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.138+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 shutdown_connections 2024-03-14T09:04:10.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.139+0000 7f94fe00d700 1 -- 172.21.15.154:0/2137822510 wait complete. 2024-03-14T09:04:10.151 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:10.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.273+0000 7fa583e5b700 1 Processor -- start 2024-03-14T09:04:10.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.273+0000 7fa583e5b700 1 -- start start 2024-03-14T09:04:10.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa583e5b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa57c0c7f70 0x7fa57c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa583e5b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa57c0c4f30 0x7fa57c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa583e5b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c157e80 con 0x7fa57c0c8350 2024-03-14T09:04:10.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa583e5b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c158ac0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa583e5b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c158120 con 0x7fa57c0c7f70 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa57c0c4f30 0x7fa57c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa57c0c4f30 0x7fa57c1547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43474/0 (socket says 172.21.15.154:43474) 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 -- 172.21.15.154:0/2635394976 learned_addr learned my addr 172.21.15.154:0/2635394976 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa581bf7700 1 --2- 172.21.15.154:0/2635394976 >> v2:172.21.15.191:3300/0 conn(0x7fa57c0c7f70 0x7fa57c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 -- 172.21.15.154:0/2635394976 >> v2:172.21.15.191:3300/0 conn(0x7fa57c0c7f70 msgr2=0x7fa57c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2635394976 >> v2:172.21.15.191:3300/0 conn(0x7fa57c0c7f70 0x7fa57c0c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 -- 172.21.15.154:0/2635394976 >> v1:172.21.15.191:6789/0 conn(0x7fa57c0c8350 legacy=0x7fa57c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 -- 172.21.15.154:0/2635394976 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c158400 con 0x7fa57c0c4f30 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2635394976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c1547f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ca350 tx=0x7fa56c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c523968f32580d0a server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.274+0000 7fa580bf5700 1 -- 172.21.15.154:0/2635394976 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c011830 con 0x7fa57c0c4f30 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa580bf5700 1 -- 172.21.15.154:0/2635394976 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c011990 con 0x7fa57c0c4f30 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa580bf5700 1 -- 172.21.15.154:0/2635394976 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c011d20 con 0x7fa57c0c4f30 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 -- 172.21.15.154:0/2635394976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 msgr2=0x7fa57c1547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2635394976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c1547f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ca350 tx=0x7fa56c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 -- 172.21.15.154:0/2635394976 shutdown_connections 2024-03-14T09:04:10.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2635394976 >> v2:172.21.15.191:3300/0 conn(0x7fa57c0c7f70 0x7fa57c0c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2635394976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c1547f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 -- 172.21.15.154:0/2635394976 >> 172.21.15.154:0/2635394976 conn(0x7fa57c072080 msgr2=0x7fa57c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:10.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 -- 172.21.15.154:0/2635394976 shutdown_connections 2024-03-14T09:04:10.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.275+0000 7fa583e5b700 1 -- 172.21.15.154:0/2635394976 wait complete. 2024-03-14T09:04:10.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.276+0000 7fa583e5b700 1 Processor -- start 2024-03-14T09:04:10.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.276+0000 7fa583e5b700 1 -- start start 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 0x7fa57c1ea600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa57c0c8350 0x7fa57c1eab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0c13b0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c1573c0 con 0x7fa57c0c8350 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa583e5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c157bd0 con 0x7fa57c0c7f70 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43482/0 (socket says 172.21.15.154:43482) 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 -- 172.21.15.154:0/2948875908 learned_addr learned my addr 172.21.15.154:0/2948875908 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 0x7fa57c1ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa5823f8700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa57c0c8350 0x7fa57c1eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 -- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 msgr2=0x7fa57c1ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 0x7fa57c1ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 -- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa57c0c8350 msgr2=0x7fa57c1eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa57c0c8350 0x7fa57c1eab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa581bf7700 1 -- 172.21.15.154:0/2948875908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009550 con 0x7fa57c0c4f30 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.277+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 0x7fa57c1ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa581bf7700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9e0 tx=0x7fa57800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578009710 con 0x7fa57c0c4f30 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800c850 con 0x7fa57c0c4f30 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0c1bf0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800cbc0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.278+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c157610 con 0x7fa57c0c4f30 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.279+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c0697d0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.279+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa57800cd20 con 0x7fa57c0c4f30 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.279+0000 7fa5727fc700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 0x7fa56006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.280+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fa5780891c0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.280+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 0x7fa56006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.280+0000 7fa5813f6700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 0x7fa56006ccf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009c10 tx=0x7fa56c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.283 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.284+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa5780542f0 con 0x7fa57c0c4f30 2024-03-14T09:04:10.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.481+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa57c1ea220 con 0x7fa57c0c4f30 2024-03-14T09:04:10.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.482+0000 7fa5727fc700 1 -- 172.21.15.154:0/2948875908 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+19857 (secure 0 0 0) 0x7fa578057910 con 0x7fa57c0c4f30 2024-03-14T09:04:10.482 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:10.483 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":73,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:09.768099+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 msgr2=0x7fa56006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 0x7fa56006ccf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009c10 tx=0x7fa56c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 msgr2=0x7fa57c153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e9e0 tx=0x7fa57800eda0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 shutdown_connections 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa57c0c8350 0x7fa57c1eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa56006a830 0x7fa56006ccf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa57c0c4f30 0x7fa57c153ec0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 --2- 172.21.15.154:0/2948875908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa57c0c7f70 0x7fa57c1ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 >> 172.21.15.154:0/2948875908 conn(0x7fa57c072080 msgr2=0x7fa57c0aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 shutdown_connections 2024-03-14T09:04:10.485 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.485+0000 7fa583e5b700 1 -- 172.21.15.154:0/2948875908 wait complete. 2024-03-14T09:04:10.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-14T09:04:10.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.622+0000 7ffb8fdf6700 1 Processor -- start 2024-03-14T09:04:10.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.622+0000 7ffb8fdf6700 1 -- start start 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8fdf6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb8806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8fdf6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb8806acb0 0x7ffb88155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8fdf6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880ac8e0 con 0x7ffb88156330 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8fdf6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8806b310 con 0x7ffb8806acb0 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8fdf6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8806de30 con 0x7ffb8814eaf0 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8d391700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb8806acb0 0x7ffb88155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8e393700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffb88156330 0x7ffb88157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43446/0 (socket says 172.21.15.154:43446) 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8e393700 1 -- 172.21.15.154:0/343179072 learned_addr learned my addr 172.21.15.154:0/343179072 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.623+0000 7ffb8db92700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb8806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8d391700 1 -- 172.21.15.154:0/343179072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 msgr2=0x7ffb8806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8d391700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb8806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8d391700 1 -- 172.21.15.154:0/343179072 >> v1:172.21.15.154:6789/0 conn(0x7ffb88156330 legacy=0x7ffb88157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8d391700 1 -- 172.21.15.154:0/343179072 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880acb30 con 0x7ffb8806acb0 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8cb90700 1 -- 172.21.15.154:0/343179072 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110568947 0 0) 0x7ffb880ac8e0 con 0x7ffb88156330 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8d391700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb88155df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009840 tx=0x7ffb84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20203677452f47a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8cb90700 1 -- 172.21.15.154:0/343179072 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84011830 con 0x7ffb8806acb0 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8cb90700 1 -- 172.21.15.154:0/343179072 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84011990 con 0x7ffb8806acb0 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8cb90700 1 -- 172.21.15.154:0/343179072 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84011d20 con 0x7ffb8806acb0 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/343179072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 msgr2=0x7ffb88155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.624+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb88155df0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009840 tx=0x7ffb84009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/343179072 shutdown_connections 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb8806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/343179072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb88155df0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/343179072 >> 172.21.15.154:0/343179072 conn(0x7ffb880c3880 msgr2=0x7ffb880c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:10.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/343179072 shutdown_connections 2024-03-14T09:04:10.627 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.625+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/343179072 wait complete. 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.626+0000 7ffb8fdf6700 1 Processor -- start 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.626+0000 7ffb8fdf6700 1 -- start start 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb88153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb88156330 0x7ffb881473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb881542b0 con 0x7ffb8806acb0 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb881522d0 con 0x7ffb88156330 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88151ff0 con 0x7ffb8814eaf0 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43504/0 (socket says 172.21.15.154:43504) 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 -- 172.21.15.154:0/3105869776 learned_addr learned my addr 172.21.15.154:0/3105869776 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8e393700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb88156330 0x7ffb881473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8d391700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb88153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 -- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 msgr2=0x7ffb88153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb88153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 -- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb88156330 msgr2=0x7ffb881473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb88156330 0x7ffb881473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 -- 172.21.15.154:0/3105869776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb840094a0 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8db92700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9b0 tx=0x7ffb7800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78009710 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7800ee50 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb881ee250 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.627+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800ca50 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.628+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb881edf50 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.629+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb880697d0 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.629+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ffb7800cbb0 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.629+0000 7ffb7e7fc700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 0x7ffb6c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.629+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ffb78089000 con 0x7ffb8806acb0 2024-03-14T09:04:10.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.629+0000 7ffb8d391700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 0x7ffb6c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:10.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.630+0000 7ffb8d391700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 0x7ffb6c06ccf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffb84000c00 tx=0x7ffb8401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:10.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.634+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ffb78054130 con 0x7ffb8806acb0 2024-03-14T09:04:10.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:10.874+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb8806b470 con 0x7ffb8806acb0 2024-03-14T09:04:11.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.784+0000 7ffb7e7fc700 1 -- 172.21.15.154:0/3105869776 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 119+0+0 (secure 0 0 0) 0x7ffb7800ce60 con 0x7ffb8806acb0 2024-03-14T09:04:11.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 msgr2=0x7ffb6c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:11.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 0x7ffb6c06ccf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffb84000c00 tx=0x7ffb8401c040 comp rx=0 tx=0).stop 2024-03-14T09:04:11.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 msgr2=0x7ffb881eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:11.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e9b0 tx=0x7ffb7800ecc0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 shutdown_connections 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb88156330 0x7ffb881473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ffb6c06a830 0x7ffb6c06ccf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb8806acb0 0x7ffb881eda10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 --2- 172.21.15.154:0/3105869776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb8814eaf0 0x7ffb88153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 >> 172.21.15.154:0/3105869776 conn(0x7ffb880c3880 msgr2=0x7ffb8806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 shutdown_connections 2024-03-14T09:04:11.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.787+0000 7ffb8fdf6700 1 -- 172.21.15.154:0/3105869776 wait complete. 2024-03-14T09:04:11.802 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-14T09:04:11.802 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-14T09:04:11.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 Processor -- start 2024-03-14T09:04:11.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 -- start start 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3640a8540 0x7fb36413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 0x7fb364154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36413ade0 con 0x7fb3641550b0 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36406fb30 con 0x7fb3640a8540 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.933+0000 7fb36a2b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36406ec30 con 0x7fb3640c3250 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3640a8540 0x7fb36413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3640a8540 0x7fb36413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43520/0 (socket says 172.21.15.154:43520) 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 -- 172.21.15.154:0/470843942 learned_addr learned my addr 172.21.15.154:0/470843942 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:11.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb3637fe700 1 --2- 172.21.15.154:0/470843942 >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 0x7fb364154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 -- 172.21.15.154:0/470843942 >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 msgr2=0x7fb364154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 --2- 172.21.15.154:0/470843942 >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 0x7fb364154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 -- 172.21.15.154:0/470843942 >> v1:172.21.15.191:6789/0 conn(0x7fb3641550b0 legacy=0x7fb3640c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 -- 172.21.15.154:0/470843942 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36406ef10 con 0x7fb3640a8540 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb3637fe700 1 --2- 172.21.15.154:0/470843942 >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 0x7fb364154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb363fff700 1 --2- 172.21.15.154:0/470843942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3640a8540 0x7fb36413a8a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ad180 tx=0x7fb354009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d3a34791bf5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb362ffd700 1 -- 172.21.15.154:0/470843942 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354004230 con 0x7fb3640a8540 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb362ffd700 1 -- 172.21.15.154:0/470843942 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354004390 con 0x7fb3640a8540 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.934+0000 7fb362ffd700 1 -- 172.21.15.154:0/470843942 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3540036a0 con 0x7fb3640a8540 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 -- 172.21.15.154:0/470843942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3640a8540 msgr2=0x7fb36413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/470843942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3640a8540 0x7fb36413a8a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb3640ad180 tx=0x7fb354009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 -- 172.21.15.154:0/470843942 shutdown_connections 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/470843942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3640a8540 0x7fb36413a8a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/470843942 >> v2:172.21.15.191:3300/0 conn(0x7fb3640c3250 0x7fb364154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 -- 172.21.15.154:0/470843942 >> 172.21.15.154:0/470843942 conn(0x7fb3640c3880 msgr2=0x7fb3640c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:11.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 -- 172.21.15.154:0/470843942 shutdown_connections 2024-03-14T09:04:11.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.935+0000 7fb36a2b1700 1 -- 172.21.15.154:0/470843942 wait complete. 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.936+0000 7fb36a2b1700 1 Processor -- start 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.936+0000 7fb36a2b1700 1 -- start start 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 0x7fb3641509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 0x7fb3640aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb364150f20 con 0x7fb3641550b0 2024-03-14T09:04:11.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3641ec660 con 0x7fb3640a8540 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3641501e0 con 0x7fb3640c3250 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43526/0 (socket says 172.21.15.154:43526) 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 -- 172.21.15.154:0/3598367306 learned_addr learned my addr 172.21.15.154:0/3598367306 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb3637fe700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 0x7fb3640aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb363fff700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 0x7fb3641509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 -- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 msgr2=0x7fb3640aacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 0x7fb3640aacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 -- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 msgr2=0x7fb3641509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 0x7fb3641509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 -- 172.21.15.154:0/3598367306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb354009550 con 0x7fb3641550b0 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb363fff700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 0x7fb3641509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb3637fe700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 0x7fb3640aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:11.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36884e700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00b970 tx=0x7fb35c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00c9b0 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c00cb10 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3641504c0 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.937+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c014440 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.938+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3640aa030 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.938+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb364150030 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.939+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb35c01d3b0 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.939+0000 7fb360ff9700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 0x7fb35006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.939+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fb35c088d80 con 0x7fb3641550b0 2024-03-14T09:04:11.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.940+0000 7fb363fff700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 0x7fb35006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:11.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.940+0000 7fb363fff700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 0x7fb35006ccf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb354009130 tx=0x7fb354008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:11.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:11.944+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb35c053e30 con 0x7fb3641550b0 2024-03-14T09:04:12.181 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.181+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3640c2a40 con 0x7fb3641550b0 2024-03-14T09:04:12.670 INFO:tasks.ceph.mds.c.smithi154.stderr:2024-03-14T09:04:12.671+0000 7f6783001700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:04:12.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.785+0000 7fb360ff9700 1 -- 172.21.15.154:0/3598367306 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb35c057450 con 0x7fb3641550b0 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 msgr2=0x7fb35006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 0x7fb35006ccf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb354009130 tx=0x7fb354008030 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 msgr2=0x7fb3641ebb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00b970 tx=0x7fb35c00bd30 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 shutdown_connections 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb35006a830 0x7fb35006ccf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3641550b0 0x7fb3641ebb50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3640a8540 0x7fb3641509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 --2- 172.21.15.154:0/3598367306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3640c3250 0x7fb3640aacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 >> 172.21.15.154:0/3598367306 conn(0x7fb3640c3880 msgr2=0x7fb3640c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 shutdown_connections 2024-03-14T09:04:12.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.788+0000 7fb36a2b1700 1 -- 172.21.15.154:0/3598367306 wait complete. 2024-03-14T09:04:12.801 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-14T09:04:12.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:12.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.926+0000 7f25c7552700 1 Processor -- start 2024-03-14T09:04:12.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.926+0000 7f25c7552700 1 -- start start 2024-03-14T09:04:12.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c7552700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f25c00a8540 0x7f25c013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c7552700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25c00c3250 0x7f25c0154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c7552700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c013ade0 con 0x7f25c01550b0 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c7552700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c006fb30 con 0x7f25c00c3250 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c7552700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c006ec30 con 0x7f25c00a8540 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25c00c3250 0x7f25c0154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25c00c3250 0x7f25c0154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43528/0 (socket says 172.21.15.154:43528) 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 -- 172.21.15.154:0/3038933498 learned_addr learned my addr 172.21.15.154:0/3038933498 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 -- 172.21.15.154:0/3038933498 >> v2:172.21.15.191:3300/0 conn(0x7f25c00a8540 msgr2=0x7f25c013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 --2- 172.21.15.154:0/3038933498 >> v2:172.21.15.191:3300/0 conn(0x7f25c00a8540 0x7f25c013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 -- 172.21.15.154:0/3038933498 >> v1:172.21.15.191:6789/0 conn(0x7f25c01550b0 legacy=0x7f25c00c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.927+0000 7f25c4aed700 1 -- 172.21.15.154:0/3038933498 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c006ef10 con 0x7f25c00c3250 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c4aed700 1 --2- 172.21.15.154:0/3038933498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c00c3250 0x7f25c0154b70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25c0070da0 tx=0x7f25bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d63e3e9fcbb254 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25b7fff700 1 -- 172.21.15.154:0/3038933498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc004230 con 0x7f25c00c3250 2024-03-14T09:04:12.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25b7fff700 1 -- 172.21.15.154:0/3038933498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc004390 con 0x7f25c00c3250 2024-03-14T09:04:12.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25b7fff700 1 -- 172.21.15.154:0/3038933498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0036a0 con 0x7f25c00c3250 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 -- 172.21.15.154:0/3038933498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c00c3250 msgr2=0x7f25c0154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 --2- 172.21.15.154:0/3038933498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c00c3250 0x7f25c0154b70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25c0070da0 tx=0x7f25bc009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 -- 172.21.15.154:0/3038933498 shutdown_connections 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 --2- 172.21.15.154:0/3038933498 >> v2:172.21.15.191:3300/0 conn(0x7f25c00a8540 0x7f25c013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 --2- 172.21.15.154:0/3038933498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c00c3250 0x7f25c0154b70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 -- 172.21.15.154:0/3038933498 >> 172.21.15.154:0/3038933498 conn(0x7f25c00c3880 msgr2=0x7f25c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 -- 172.21.15.154:0/3038933498 shutdown_connections 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.928+0000 7f25c7552700 1 -- 172.21.15.154:0/3038933498 wait complete. 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.929+0000 7f25c7552700 1 Processor -- start 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.929+0000 7f25c7552700 1 -- start start 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c00a8540 0x7f25c00a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 0x7f25c00c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01523b0 con 0x7f25c01550b0 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01ee0e0 con 0x7f25c00c3250 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c7552700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01ee380 con 0x7f25c00a8540 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c5aef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c5aef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43540/0 (socket says 172.21.15.154:43540) 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c5aef700 1 -- 172.21.15.154:0/4001080030 learned_addr learned my addr 172.21.15.154:0/4001080030 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.930+0000 7f25c4aed700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 0x7f25c00c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c52ee700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c00a8540 0x7f25c00a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 -- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c00a8540 msgr2=0x7f25c00a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c00a8540 0x7f25c00a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 -- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 msgr2=0x7f25c00c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 0x7f25c00c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:12.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 -- 172.21.15.154:0/4001080030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009550 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c4aed700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 0x7f25c00c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c5aef700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25b800b990 tx=0x7f25b800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800c9d0 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800cb30 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c014f520 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8014440 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.931+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00c0a60 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.932+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c014e550 con 0x7f25c01550b0 2024-03-14T09:04:12.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.932+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f25b8006d50 con 0x7f25c01550b0 2024-03-14T09:04:12.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.933+0000 7f25b5ffb700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 0x7f25a406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:12.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.933+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f25b8088b30 con 0x7f25c01550b0 2024-03-14T09:04:12.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.933+0000 7f25c52ee700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 0x7f25a406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:12.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.934+0000 7f25c52ee700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 0x7f25a406ccf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f25b0006fd0 tx=0x7f25b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:12.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:12.937+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f25c014e550 con 0x7f25c01550b0 2024-03-14T09:04:13.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.172+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c014e550 con 0x7f25c01550b0 2024-03-14T09:04:13.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.172+0000 7f25b5ffb700 1 -- 172.21.15.154:0/4001080030 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5958 (secure 0 0 0) 0x7f25c014e550 con 0x7f25c01550b0 2024-03-14T09:04:13.172 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:13.172 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.174+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 msgr2=0x7f25a406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.174+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 0x7f25a406ccf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f25b0006fd0 tx=0x7f25b0008040 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.174+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 msgr2=0x7f25c0133660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.174+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25b800b990 tx=0x7f25b800bd50 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 shutdown_connections 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25a406a830 0x7f25a406ccf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25c01550b0 0x7f25c0133660 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c00a8540 0x7f25c00a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 --2- 172.21.15.154:0/4001080030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c00c3250 0x7f25c00c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 >> 172.21.15.154:0/4001080030 conn(0x7f25c00c3880 msgr2=0x7f25c01397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 shutdown_connections 2024-03-14T09:04:13.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.175+0000 7f25c7552700 1 -- 172.21.15.154:0/4001080030 wait complete. 2024-03-14T09:04:13.176 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 50 2024-03-14T09:04:13.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:13.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.309+0000 7f9c0a469700 1 Processor -- start 2024-03-14T09:04:13.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.309+0000 7f9c0a469700 1 -- start start 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c0a469700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9c04156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c0a469700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040ac8e0 con 0x7f9c0415d270 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c0a469700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0406de30 con 0x7f9c0414eaf0 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c0a469700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0406b310 con 0x7f9c04155df0 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c08a06700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9c0415d270 0x7f9c0415f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43458/0 (socket says 172.21.15.154:43458) 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c08a06700 1 -- 172.21.15.154:0/315681153 learned_addr learned my addr 172.21.15.154:0/315681153 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c037fe700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9c04156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085063723 0 0) 0x7f9c040ac8e0 con 0x7f9c0415d270 2024-03-14T09:04:13.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.310+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bec0035b0 con 0x7f9c0415d270 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477627987 0 0) 0x7f9bec0035b0 con 0x7f9c0415d270 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 msgr2=0x7f9c04156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9c04156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 >> v1:172.21.15.191:6789/0 conn(0x7f9c0414eaf0 legacy=0x7f9c0406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040acb30 con 0x7f9c0415d270 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9bfc0019f0 con 0x7f9c0415d270 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 >> v1:172.21.15.154:6789/0 conn(0x7f9c0415d270 legacy=0x7f9c0415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9bec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bec004850 0x7f9bec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9bec005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0406b310 con 0x7f9bec003a30 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec00bb40 con 0x7f9bec004850 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec00bd00 con 0x7f9c04155df0 2024-03-14T09:04:13.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9bec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c037fe700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bec004850 0x7f9bec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c08a06700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9bec005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 -- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 msgr2=0x7f9bec005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c04155df0 0x7f9bec005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 -- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bec004850 msgr2=0x7f9bec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bec004850 0x7f9bec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 -- 172.21.15.154:0/315681153 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0035b0 con 0x7f9bec003a30 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c03fff700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9bec003e10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80018d0 tx=0x7f9bec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123de7d2f386505b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800c690 con 0x7f9bec003a30 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800c810 con 0x7f9bec003a30 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.311+0000 7f9c02ffd700 1 -- 172.21.15.154:0/315681153 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800cbc0 con 0x7f9bec003a30 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 -- 172.21.15.154:0/315681153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 msgr2=0x7f9bec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 --2- 172.21.15.154:0/315681153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9bec003e10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80018d0 tx=0x7f9bec005880 comp rx=0 tx=0).stop 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c08a06700 1 -- 172.21.15.154:0/315681153 reap_dead start 2024-03-14T09:04:13.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 -- 172.21.15.154:0/315681153 shutdown_connections 2024-03-14T09:04:13.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 -- 172.21.15.154:0/315681153 >> 172.21.15.154:0/315681153 conn(0x7f9c040c3880 msgr2=0x7f9c040c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:13.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 -- 172.21.15.154:0/315681153 shutdown_connections 2024-03-14T09:04:13.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.312+0000 7f9c0a469700 1 -- 172.21.15.154:0/315681153 wait complete. 2024-03-14T09:04:13.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.313+0000 7f9c0a469700 1 Processor -- start 2024-03-14T09:04:13.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.313+0000 7f9c0a469700 1 -- start start 2024-03-14T09:04:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c041ee170 0x7f9c041413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0413c870 0x7f9c04141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0413c190 con 0x7f9bec003a30 2024-03-14T09:04:13.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041ee900 con 0x7f9c041ee170 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c0a469700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040c1630 con 0x7f9c0413c870 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43562/0 (socket says 172.21.15.154:43562) 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 -- 172.21.15.154:0/1744302047 learned_addr learned my addr 172.21.15.154:0/1744302047 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c037fe700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c041ee170 0x7f9c041413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c08a06700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0413c870 0x7f9c04141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 -- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0413c870 msgr2=0x7f9c04141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0413c870 0x7f9c04141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 -- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c041ee170 msgr2=0x7f9c041413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c041ee170 0x7f9c041413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 -- 172.21.15.154:0/1744302047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040c3530 con 0x7f9bec003a30 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.314+0000 7f9c03fff700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c0e0 tx=0x7f9bf8001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.315+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800cb80 con 0x7f9bec003a30 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.315+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8021070 con 0x7f9bec003a30 2024-03-14T09:04:13.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.315+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c041ee620 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.315+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800cee0 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.315+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c040c1ac0 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.316+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9bf8023030 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.316+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040697d0 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.316+0000 7f9c00ff9700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 0x7f9bf006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.316+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9bf808a3e0 con 0x7f9bec003a30 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.317+0000 7f9c037fe700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 0x7f9bf006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.317+0000 7f9c037fe700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 0x7f9bf006ccf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c041ee550 tx=0x7f9bf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.321+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9bf8055510 con 0x7f9bec003a30 2024-03-14T09:04:13.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.519+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c0406b470 con 0x7f9bec003a30 2024-03-14T09:04:13.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.521+0000 7f9c00ff9700 1 -- 172.21.15.154:0/1744302047 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f9bf8058b30 con 0x7f9bec003a30 2024-03-14T09:04:13.521 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:13.521 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:13.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 msgr2=0x7f9bf006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 0x7f9bf006ccf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c041ee550 tx=0x7f9bf4008040 comp rx=0 tx=0).stop 2024-03-14T09:04:13.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 msgr2=0x7f9c041edc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c0e0 tx=0x7f9bf8001230 comp rx=0 tx=0).stop 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 shutdown_connections 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0413c870 0x7f9c04141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bf006a830 0x7f9bf006ccf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bec003a30 0x7f9c041edc30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 --2- 172.21.15.154:0/1744302047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c041ee170 0x7f9c041413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 >> 172.21.15.154:0/1744302047 conn(0x7f9c040c3880 msgr2=0x7f9c0406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 shutdown_connections 2024-03-14T09:04:13.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.523+0000 7f9c0a469700 1 -- 172.21.15.154:0/1744302047 wait complete. 2024-03-14T09:04:13.536 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-14T09:04:13.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:13.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.657+0000 7ff8efc46700 1 Processor -- start 2024-03-14T09:04:13.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.657+0000 7ff8efc46700 1 -- start start 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8efc46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8e806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8efc46700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8e806acb0 0x7ff8e8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8efc46700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e80ac8e0 con 0x7ff8e8156330 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8efc46700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e806b310 con 0x7ff8e806acb0 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8efc46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e806de30 con 0x7ff8e814eaf0 2024-03-14T09:04:13.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ee1e3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff8e8156330 0x7ff8e8157ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43474/0 (socket says 172.21.15.154:43474) 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ee1e3700 1 -- 172.21.15.154:0/4263111409 learned_addr learned my addr 172.21.15.154:0/4263111409 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> v2:172.21.15.191:3300/0 conn(0x7ff8e806acb0 0x7ff8e8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701245036 0 0) 0x7ff8e80ac8e0 con 0x7ff8e8156330 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8e806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.658+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d40035b0 con 0x7ff8e8156330 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010187195 0 0) 0x7ff8d40035b0 con 0x7ff8e8156330 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 msgr2=0x7ff8e806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8e806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 >> v2:172.21.15.191:3300/0 conn(0x7ff8e806acb0 msgr2=0x7ff8e8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 --2- 172.21.15.154:0/4263111409 >> v2:172.21.15.191:3300/0 conn(0x7ff8e806acb0 0x7ff8e8155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e80acb30 con 0x7ff8e8156330 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7ff8e00019f0 con 0x7ff8e8156330 2024-03-14T09:04:13.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> v2:172.21.15.191:3300/0 conn(0x7ff8e806acb0 0x7ff8e8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 >> v1:172.21.15.154:6789/0 conn(0x7ff8e8156330 legacy=0x7ff8e8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d4003f80 0x7ff8d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4004d90 0x7ff8d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e806b310 con 0x7ff8d4003f80 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e806de30 con 0x7ff8d4004d90 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d400baf0 con 0x7ff8e814eaf0 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d4003f80 0x7ff8d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ee1e3700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4004d90 0x7ff8d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 msgr2=0x7ff8d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e814eaf0 0x7ff8d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4004d90 msgr2=0x7ff8d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:13.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4004d90 0x7ff8d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/4263111409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d40035b0 con 0x7ff8d4003f80 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.659+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d4003f80 0x7ff8d4004850 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009d30 tx=0x7ff8d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ddc9e3266e3aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4009e80 con 0x7ff8d4003f80 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400e9a0 con 0x7ff8d4003f80 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8ec9e0700 1 -- 172.21.15.154:0/4263111409 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400ed30 con 0x7ff8d4003f80 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 -- 172.21.15.154:0/4263111409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d4003f80 msgr2=0x7ff8d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 --2- 172.21.15.154:0/4263111409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8d4003f80 0x7ff8d4004850 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009d30 tx=0x7ff8d4005880 comp rx=0 tx=0).stop 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8ee1e3700 1 -- 172.21.15.154:0/4263111409 reap_dead start 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 -- 172.21.15.154:0/4263111409 shutdown_connections 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 -- 172.21.15.154:0/4263111409 >> 172.21.15.154:0/4263111409 conn(0x7ff8e80c3880 msgr2=0x7ff8e80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 -- 172.21.15.154:0/4263111409 shutdown_connections 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.660+0000 7ff8efc46700 1 -- 172.21.15.154:0/4263111409 wait complete. 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.661+0000 7ff8efc46700 1 Processor -- start 2024-03-14T09:04:13.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.661+0000 7ff8efc46700 1 -- start start 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 0x7ff8e8144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e80c22e0 0x7ff8e8142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e814abe0 con 0x7ff8e8144b60 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81434b0 con 0x7ff8d4003f80 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8efc46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8142fc0 con 0x7ff8e80c22e0 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8ed1e1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8ed1e1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43578/0 (socket says 172.21.15.154:43578) 2024-03-14T09:04:13.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/942978397 learned_addr learned my addr 172.21.15.154:0/942978397 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 0x7ff8e8144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.662+0000 7ff8ee1e3700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e80c22e0 0x7ff8e8142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e80c22e0 msgr2=0x7ff8e8142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e80c22e0 0x7ff8e8142160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 msgr2=0x7ff8e8144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 0x7ff8e8144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 -- 172.21.15.154:0/942978397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8146000 con 0x7ff8e8144b60 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 0x7ff8e8144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8ed1e1700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400ba40 tx=0x7ff8e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4006e60 con 0x7ff8e8144b60 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4003b10 con 0x7ff8e8144b60 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4009e80 con 0x7ff8e8144b60 2024-03-14T09:04:13.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8143690 con 0x7ff8e8144b60 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.663+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e81431d0 con 0x7ff8e8144b60 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.664+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e81556e0 con 0x7ff8e8144b60 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.664+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff8e4013470 con 0x7ff8e8144b60 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.665+0000 7ff8de7fc700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 0x7ff8cc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.665+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff8e408aca0 con 0x7ff8e8144b60 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.665+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 0x7ff8cc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.665+0000 7ff8ed9e2700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 0x7ff8cc06cd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8004150 tx=0x7ff8d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:13.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.669+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff8e81556e0 con 0x7ff8e8144b60 2024-03-14T09:04:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.860+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8e806e080 con 0x7ff8cc06a860 2024-03-14T09:04:13.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.874+0000 7ff8de7fc700 1 -- 172.21.15.154:0/942978397 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528102 (secure 0 0 0) 0x7ff8d8019040 con 0x7ff8cc06a860 2024-03-14T09:04:13.881 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 msgr2=0x7ff8cc06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 0x7ff8cc06cd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8004150 tx=0x7ff8d8008040 comp rx=0 tx=0).stop 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 msgr2=0x7ff8e80c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400ba40 tx=0x7ff8e40047c0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 shutdown_connections 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8e80c22e0 0x7ff8e8142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4003f80 0x7ff8e8144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff8cc06a860 0x7ff8cc06cd20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.887 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 --2- 172.21.15.154:0/942978397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e8144b60 0x7ff8e80c1da0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:13.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 >> 172.21.15.154:0/942978397 conn(0x7ff8e80c3880 msgr2=0x7ff8e80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:13.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 shutdown_connections 2024-03-14T09:04:13.888 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:13.879+0000 7ff8efc46700 1 -- 172.21.15.154:0/942978397 wait complete. 2024-03-14T09:04:13.888 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:13.891 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-14T09:04:13.558917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":336,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224552,"kb_used_data":8392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750168,"statfs":{"total":773094113280,"available":772864172032,"internally_reserved":0,"allocated":8593408,"data_stored":4426286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.199295"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400182+0000","last_change":"2024-03-14T09:04:09.795463+0000","last_active":"2024-03-14T09:04:11.400182+0000","last_peered":"2024-03-14T09:04:11.400182+0000","last_clean":"2024-03-14T09:04:11.400182+0000","last_became_active":"2024-03-14T09:04:09.795259+0000","last_became_peered":"2024-03-14T09:04:09.795259+0000","last_unstale":"2024-03-14T09:04:11.400182+0000","last_undegraded":"2024-03-14T09:04:11.400182+0000","last_fullsized":"2024-03-14T09:04:11.400182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:37.967825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779210+0000","last_change":"2024-03-14T09:03:57.183867+0000","last_active":"2024-03-14T09:04:09.779210+0000","last_peered":"2024-03-14T09:04:09.779210+0000","last_clean":"2024-03-14T09:04:09.779210+0000","last_became_active":"2024-03-14T09:03:57.183371+0000","last_became_peered":"2024-03-14T09:03:57.183371+0000","last_unstale":"2024-03-14T09:04:09.779210+0000","last_undegraded":"2024-03-14T09:04:09.779210+0000","last_fullsized":"2024-03-14T09:04:09.779210+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783772+0000","last_change":"2024-03-14T09:04:09.791872+0000","last_active":"2024-03-14T09:04:10.783772+0000","last_peered":"2024-03-14T09:04:10.783772+0000","last_clean":"2024-03-14T09:04:10.783772+0000","last_became_active":"2024-03-14T09:04:09.791341+0000","last_became_peered":"2024-03-14T09:04:09.791341+0000","last_unstale":"2024-03-14T09:04:10.783772+0000","last_undegraded":"2024-03-14T09:04:10.783772+0000","last_fullsized":"2024-03-14T09:04:10.783772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:07.293563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781438+0000","last_change":"2024-03-14T09:03:57.187973+0000","last_active":"2024-03-14T09:04:09.781438+0000","last_peered":"2024-03-14T09:04:09.781438+0000","last_clean":"2024-03-14T09:04:09.781438+0000","last_became_active":"2024-03-14T09:03:57.187737+0000","last_became_peered":"2024-03-14T09:03:57.187737+0000","last_unstale":"2024-03-14T09:04:09.781438+0000","last_undegraded":"2024-03-14T09:04:09.781438+0000","last_fullsized":"2024-03-14T09:04:09.781438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:39.248247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400288+0000","last_change":"2024-03-14T09:04:09.789743+0000","last_active":"2024-03-14T09:04:11.400288+0000","last_peered":"2024-03-14T09:04:11.400288+0000","last_clean":"2024-03-14T09:04:11.400288+0000","last_became_active":"2024-03-14T09:04:09.789396+0000","last_became_peered":"2024-03-14T09:04:09.789396+0000","last_unstale":"2024-03-14T09:04:11.400288+0000","last_undegraded":"2024-03-14T09:04:11.400288+0000","last_fullsized":"2024-03-14T09:04:11.400288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:31.883445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780964+0000","last_change":"2024-03-14T09:03:57.180121+0000","last_active":"2024-03-14T09:04:09.780964+0000","last_peered":"2024-03-14T09:04:09.780964+0000","last_clean":"2024-03-14T09:04:09.780964+0000","last_became_active":"2024-03-14T09:03:57.179465+0000","last_became_peered":"2024-03-14T09:03:57.179465+0000","last_unstale":"2024-03-14T09:04:09.780964+0000","last_undegraded":"2024-03-14T09:04:09.780964+0000","last_fullsized":"2024-03-14T09:04:09.780964+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401365+0000","last_change":"2024-03-14T09:04:09.788938+0000","last_active":"2024-03-14T09:04:11.401365+0000","last_peered":"2024-03-14T09:04:11.401365+0000","last_clean":"2024-03-14T09:04:11.401365+0000","last_became_active":"2024-03-14T09:04:09.788510+0000","last_became_peered":"2024-03-14T09:04:09.788510+0000","last_unstale":"2024-03-14T09:04:11.401365+0000","last_undegraded":"2024-03-14T09:04:11.401365+0000","last_fullsized":"2024-03-14T09:04:11.401365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:21.360800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783372+0000","last_change":"2024-03-14T09:03:57.191808+0000","last_active":"2024-03-14T09:04:09.783372+0000","last_peered":"2024-03-14T09:04:09.783372+0000","last_clean":"2024-03-14T09:04:09.783372+0000","last_became_active":"2024-03-14T09:03:57.187786+0000","last_became_peered":"2024-03-14T09:03:57.187786+0000","last_unstale":"2024-03-14T09:04:09.783372+0000","last_undegraded":"2024-03-14T09:04:09.783372+0000","last_fullsized":"2024-03-14T09:04:09.783372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:23.462424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400340+0000","last_change":"2024-03-14T09:04:09.789579+0000","last_active":"2024-03-14T09:04:11.400340+0000","last_peered":"2024-03-14T09:04:11.400340+0000","last_clean":"2024-03-14T09:04:11.400340+0000","last_became_active":"2024-03-14T09:04:09.789295+0000","last_became_peered":"2024-03-14T09:04:09.789295+0000","last_unstale":"2024-03-14T09:04:11.400340+0000","last_undegraded":"2024-03-14T09:04:11.400340+0000","last_fullsized":"2024-03-14T09:04:11.400340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:07.483738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793680+0000","last_change":"2024-03-14T09:03:57.186814+0000","last_active":"2024-03-14T09:04:09.793680+0000","last_peered":"2024-03-14T09:04:09.793680+0000","last_clean":"2024-03-14T09:04:09.793680+0000","last_became_active":"2024-03-14T09:03:57.186480+0000","last_became_peered":"2024-03-14T09:03:57.186480+0000","last_unstale":"2024-03-14T09:04:09.793680+0000","last_undegraded":"2024-03-14T09:04:09.793680+0000","last_fullsized":"2024-03-14T09:04:09.793680+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400422+0000","last_change":"2024-03-14T09:04:09.788892+0000","last_active":"2024-03-14T09:04:11.400422+0000","last_peered":"2024-03-14T09:04:11.400422+0000","last_clean":"2024-03-14T09:04:11.400422+0000","last_became_active":"2024-03-14T09:04:09.788657+0000","last_became_peered":"2024-03-14T09:04:09.788657+0000","last_unstale":"2024-03-14T09:04:11.400422+0000","last_undegraded":"2024-03-14T09:04:11.400422+0000","last_fullsized":"2024-03-14T09:04:11.400422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:47:13.189988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784282+0000","last_change":"2024-03-14T09:03:57.179120+0000","last_active":"2024-03-14T09:04:09.784282+0000","last_peered":"2024-03-14T09:04:09.784282+0000","last_clean":"2024-03-14T09:04:09.784282+0000","last_became_active":"2024-03-14T09:03:57.178824+0000","last_became_peered":"2024-03-14T09:03:57.178824+0000","last_unstale":"2024-03-14T09:04:09.784282+0000","last_undegraded":"2024-03-14T09:04:09.784282+0000","last_fullsized":"2024-03-14T09:04:09.784282+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400767+0000","last_change":"2024-03-14T09:04:09.786125+0000","last_active":"2024-03-14T09:04:11.400767+0000","last_peered":"2024-03-14T09:04:11.400767+0000","last_clean":"2024-03-14T09:04:11.400767+0000","last_became_active":"2024-03-14T09:04:09.785805+0000","last_became_peered":"2024-03-14T09:04:09.785805+0000","last_unstale":"2024-03-14T09:04:11.400767+0000","last_undegraded":"2024-03-14T09:04:11.400767+0000","last_fullsized":"2024-03-14T09:04:11.400767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:07.367499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780045+0000","last_change":"2024-03-14T09:03:57.176658+0000","last_active":"2024-03-14T09:04:09.780045+0000","last_peered":"2024-03-14T09:04:09.780045+0000","last_clean":"2024-03-14T09:04:09.780045+0000","last_became_active":"2024-03-14T09:03:57.176352+0000","last_became_peered":"2024-03-14T09:03:57.176352+0000","last_unstale":"2024-03-14T09:04:09.780045+0000","last_undegraded":"2024-03-14T09:04:09.780045+0000","last_fullsized":"2024-03-14T09:04:09.780045+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:51:09.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401267+0000","last_change":"2024-03-14T09:04:09.790488+0000","last_active":"2024-03-14T09:04:11.401267+0000","last_peered":"2024-03-14T09:04:11.401267+0000","last_clean":"2024-03-14T09:04:11.401267+0000","last_became_active":"2024-03-14T09:04:09.790221+0000","last_became_peered":"2024-03-14T09:04:09.790221+0000","last_unstale":"2024-03-14T09:04:11.401267+0000","last_undegraded":"2024-03-14T09:04:11.401267+0000","last_fullsized":"2024-03-14T09:04:11.401267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:19.679079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783314+0000","last_change":"2024-03-14T09:03:57.186734+0000","last_active":"2024-03-14T09:04:09.783314+0000","last_peered":"2024-03-14T09:04:09.783314+0000","last_clean":"2024-03-14T09:04:09.783314+0000","last_became_active":"2024-03-14T09:03:57.186455+0000","last_became_peered":"2024-03-14T09:03:57.186455+0000","last_unstale":"2024-03-14T09:04:09.783314+0000","last_undegraded":"2024-03-14T09:04:09.783314+0000","last_fullsized":"2024-03-14T09:04:09.783314+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400975+0000","last_change":"2024-03-14T09:04:09.801347+0000","last_active":"2024-03-14T09:04:11.400975+0000","last_peered":"2024-03-14T09:04:11.400975+0000","last_clean":"2024-03-14T09:04:11.400975+0000","last_became_active":"2024-03-14T09:04:09.801060+0000","last_became_peered":"2024-03-14T09:04:09.801060+0000","last_unstale":"2024-03-14T09:04:11.400975+0000","last_undegraded":"2024-03-14T09:04:11.400975+0000","last_fullsized":"2024-03-14T09:04:11.400975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:40.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784810+0000","last_change":"2024-03-14T09:03:57.186757+0000","last_active":"2024-03-14T09:04:09.784810+0000","last_peered":"2024-03-14T09:04:09.784810+0000","last_clean":"2024-03-14T09:04:09.784810+0000","last_became_active":"2024-03-14T09:03:57.186486+0000","last_became_peered":"2024-03-14T09:03:57.186486+0000","last_unstale":"2024-03-14T09:04:09.784810+0000","last_undegraded":"2024-03-14T09:04:09.784810+0000","last_fullsized":"2024-03-14T09:04:09.784810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782884+0000","last_change":"2024-03-14T09:04:09.799551+0000","last_active":"2024-03-14T09:04:10.782884+0000","last_peered":"2024-03-14T09:04:10.782884+0000","last_clean":"2024-03-14T09:04:10.782884+0000","last_became_active":"2024-03-14T09:04:09.799287+0000","last_became_peered":"2024-03-14T09:04:09.799287+0000","last_unstale":"2024-03-14T09:04:10.782884+0000","last_undegraded":"2024-03-14T09:04:10.782884+0000","last_fullsized":"2024-03-14T09:04:10.782884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778187+0000","last_change":"2024-03-14T09:03:57.180488+0000","last_active":"2024-03-14T09:04:09.778187+0000","last_peered":"2024-03-14T09:04:09.778187+0000","last_clean":"2024-03-14T09:04:09.778187+0000","last_became_active":"2024-03-14T09:03:57.180202+0000","last_became_peered":"2024-03-14T09:03:57.180202+0000","last_unstale":"2024-03-14T09:04:09.778187+0000","last_undegraded":"2024-03-14T09:04:09.778187+0000","last_fullsized":"2024-03-14T09:04:09.778187+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783557+0000","last_change":"2024-03-14T09:04:09.800401+0000","last_active":"2024-03-14T09:04:10.783557+0000","last_peered":"2024-03-14T09:04:10.783557+0000","last_clean":"2024-03-14T09:04:10.783557+0000","last_became_active":"2024-03-14T09:04:09.800120+0000","last_became_peered":"2024-03-14T09:04:09.800120+0000","last_unstale":"2024-03-14T09:04:10.783557+0000","last_undegraded":"2024-03-14T09:04:10.783557+0000","last_fullsized":"2024-03-14T09:04:10.783557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:00.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783772+0000","last_change":"2024-03-14T09:04:09.374689+0000","last_active":"2024-03-14T09:04:10.783772+0000","last_peered":"2024-03-14T09:04:10.783772+0000","last_clean":"2024-03-14T09:04:10.783772+0000","last_became_active":"2024-03-14T09:04:09.374377+0000","last_became_peered":"2024-03-14T09:04:09.374377+0000","last_unstale":"2024-03-14T09:04:10.783772+0000","last_undegraded":"2024-03-14T09:04:10.783772+0000","last_fullsized":"2024-03-14T09:04:10.783772+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:37.206868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400509+0000","last_change":"2024-03-14T09:04:09.793291+0000","last_active":"2024-03-14T09:04:11.400509+0000","last_peered":"2024-03-14T09:04:11.400509+0000","last_clean":"2024-03-14T09:04:11.400509+0000","last_became_active":"2024-03-14T09:04:09.793025+0000","last_became_peered":"2024-03-14T09:04:09.793025+0000","last_unstale":"2024-03-14T09:04:11.400509+0000","last_undegraded":"2024-03-14T09:04:11.400509+0000","last_fullsized":"2024-03-14T09:04:11.400509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:45:57.150041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776759+0000","last_change":"2024-03-14T09:03:57.182863+0000","last_active":"2024-03-14T09:04:09.776759+0000","last_peered":"2024-03-14T09:04:09.776759+0000","last_clean":"2024-03-14T09:04:09.776759+0000","last_became_active":"2024-03-14T09:03:57.181034+0000","last_became_peered":"2024-03-14T09:03:57.181034+0000","last_unstale":"2024-03-14T09:04:09.776759+0000","last_undegraded":"2024-03-14T09:04:09.776759+0000","last_fullsized":"2024-03-14T09:04:09.776759+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783014+0000","last_change":"2024-03-14T09:04:09.792625+0000","last_active":"2024-03-14T09:04:10.783014+0000","last_peered":"2024-03-14T09:04:10.783014+0000","last_clean":"2024-03-14T09:04:10.783014+0000","last_became_active":"2024-03-14T09:04:09.792370+0000","last_became_peered":"2024-03-14T09:04:09.792370+0000","last_unstale":"2024-03-14T09:04:10.783014+0000","last_undegraded":"2024-03-14T09:04:10.783014+0000","last_fullsized":"2024-03-14T09:04:10.783014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:20:45.664696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779881+0000","last_change":"2024-03-14T09:03:57.177255+0000","last_active":"2024-03-14T09:04:09.779881+0000","last_peered":"2024-03-14T09:04:09.779881+0000","last_clean":"2024-03-14T09:04:09.779881+0000","last_became_active":"2024-03-14T09:03:57.176972+0000","last_became_peered":"2024-03-14T09:03:57.176972+0000","last_unstale":"2024-03-14T09:04:09.779881+0000","last_undegraded":"2024-03-14T09:04:09.779881+0000","last_fullsized":"2024-03-14T09:04:09.779881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:41.881683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400979+0000","last_change":"2024-03-14T09:04:09.799979+0000","last_active":"2024-03-14T09:04:11.400979+0000","last_peered":"2024-03-14T09:04:11.400979+0000","last_clean":"2024-03-14T09:04:11.400979+0000","last_became_active":"2024-03-14T09:04:09.799709+0000","last_became_peered":"2024-03-14T09:04:09.799709+0000","last_unstale":"2024-03-14T09:04:11.400979+0000","last_undegraded":"2024-03-14T09:04:11.400979+0000","last_fullsized":"2024-03-14T09:04:11.400979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785807+0000","last_change":"2024-03-14T09:03:57.191926+0000","last_active":"2024-03-14T09:04:09.785807+0000","last_peered":"2024-03-14T09:04:09.785807+0000","last_clean":"2024-03-14T09:04:09.785807+0000","last_became_active":"2024-03-14T09:03:57.191416+0000","last_became_peered":"2024-03-14T09:03:57.191416+0000","last_unstale":"2024-03-14T09:04:09.785807+0000","last_undegraded":"2024-03-14T09:04:09.785807+0000","last_fullsized":"2024-03-14T09:04:09.785807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:25.082584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783412+0000","last_change":"2024-03-14T09:04:09.801660+0000","last_active":"2024-03-14T09:04:10.783412+0000","last_peered":"2024-03-14T09:04:10.783412+0000","last_clean":"2024-03-14T09:04:10.783412+0000","last_became_active":"2024-03-14T09:04:09.801416+0000","last_became_peered":"2024-03-14T09:04:09.801416+0000","last_unstale":"2024-03-14T09:04:10.783412+0000","last_undegraded":"2024-03-14T09:04:10.783412+0000","last_fullsized":"2024-03-14T09:04:10.783412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:29:18.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782475+0000","last_change":"2024-03-14T09:03:57.175406+0000","last_active":"2024-03-14T09:04:09.782475+0000","last_peered":"2024-03-14T09:04:09.782475+0000","last_clean":"2024-03-14T09:04:09.782475+0000","last_became_active":"2024-03-14T09:03:57.175155+0000","last_became_peered":"2024-03-14T09:03:57.175155+0000","last_unstale":"2024-03-14T09:04:09.782475+0000","last_undegraded":"2024-03-14T09:04:09.782475+0000","last_fullsized":"2024-03-14T09:04:09.782475+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:19.270920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784302+0000","last_change":"2024-03-14T09:04:09.793701+0000","last_active":"2024-03-14T09:04:10.784302+0000","last_peered":"2024-03-14T09:04:10.784302+0000","last_clean":"2024-03-14T09:04:10.784302+0000","last_became_active":"2024-03-14T09:04:09.792889+0000","last_became_peered":"2024-03-14T09:04:09.792889+0000","last_unstale":"2024-03-14T09:04:10.784302+0000","last_undegraded":"2024-03-14T09:04:10.784302+0000","last_fullsized":"2024-03-14T09:04:10.784302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:17.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793669+0000","last_change":"2024-03-14T09:03:57.181942+0000","last_active":"2024-03-14T09:04:09.793669+0000","last_peered":"2024-03-14T09:04:09.793669+0000","last_clean":"2024-03-14T09:04:09.793669+0000","last_became_active":"2024-03-14T09:03:57.181669+0000","last_became_peered":"2024-03-14T09:03:57.181669+0000","last_unstale":"2024-03-14T09:04:09.793669+0000","last_undegraded":"2024-03-14T09:04:09.793669+0000","last_fullsized":"2024-03-14T09:04:09.793669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784332+0000","last_change":"2024-03-14T09:04:09.797382+0000","last_active":"2024-03-14T09:04:10.784332+0000","last_peered":"2024-03-14T09:04:10.784332+0000","last_clean":"2024-03-14T09:04:10.784332+0000","last_became_active":"2024-03-14T09:04:09.797137+0000","last_became_peered":"2024-03-14T09:04:09.797137+0000","last_unstale":"2024-03-14T09:04:10.784332+0000","last_undegraded":"2024-03-14T09:04:10.784332+0000","last_fullsized":"2024-03-14T09:04:10.784332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:38:59.136969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796582+0000","last_change":"2024-03-14T09:03:57.177217+0000","last_active":"2024-03-14T09:04:09.796582+0000","last_peered":"2024-03-14T09:04:09.796582+0000","last_clean":"2024-03-14T09:04:09.796582+0000","last_became_active":"2024-03-14T09:03:57.176941+0000","last_became_peered":"2024-03-14T09:03:57.176941+0000","last_unstale":"2024-03-14T09:04:09.796582+0000","last_undegraded":"2024-03-14T09:04:09.796582+0000","last_fullsized":"2024-03-14T09:04:09.796582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783012+0000","last_change":"2024-03-14T09:04:09.796023+0000","last_active":"2024-03-14T09:04:10.783012+0000","last_peered":"2024-03-14T09:04:10.783012+0000","last_clean":"2024-03-14T09:04:10.783012+0000","last_became_active":"2024-03-14T09:04:09.794309+0000","last_became_peered":"2024-03-14T09:04:09.794309+0000","last_unstale":"2024-03-14T09:04:10.783012+0000","last_undegraded":"2024-03-14T09:04:10.783012+0000","last_fullsized":"2024-03-14T09:04:10.783012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:49.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784613+0000","last_change":"2024-03-14T09:03:57.176770+0000","last_active":"2024-03-14T09:04:09.784613+0000","last_peered":"2024-03-14T09:04:09.784613+0000","last_clean":"2024-03-14T09:04:09.784613+0000","last_became_active":"2024-03-14T09:03:57.176496+0000","last_became_peered":"2024-03-14T09:03:57.176496+0000","last_unstale":"2024-03-14T09:04:09.784613+0000","last_undegraded":"2024-03-14T09:04:09.784613+0000","last_fullsized":"2024-03-14T09:04:09.784613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:32.959955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401480+0000","last_change":"2024-03-14T09:04:09.791778+0000","last_active":"2024-03-14T09:04:11.401480+0000","last_peered":"2024-03-14T09:04:11.401480+0000","last_clean":"2024-03-14T09:04:11.401480+0000","last_became_active":"2024-03-14T09:04:09.791462+0000","last_became_peered":"2024-03-14T09:04:09.791462+0000","last_unstale":"2024-03-14T09:04:11.401480+0000","last_undegraded":"2024-03-14T09:04:11.401480+0000","last_fullsized":"2024-03-14T09:04:11.401480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:34:27.112960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773773+0000","last_change":"2024-03-14T09:03:57.188745+0000","last_active":"2024-03-14T09:04:09.773773+0000","last_peered":"2024-03-14T09:04:09.773773+0000","last_clean":"2024-03-14T09:04:09.773773+0000","last_became_active":"2024-03-14T09:03:57.187830+0000","last_became_peered":"2024-03-14T09:03:57.187830+0000","last_unstale":"2024-03-14T09:04:09.773773+0000","last_undegraded":"2024-03-14T09:04:09.773773+0000","last_fullsized":"2024-03-14T09:04:09.773773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783976+0000","last_change":"2024-03-14T09:04:09.790848+0000","last_active":"2024-03-14T09:04:10.783976+0000","last_peered":"2024-03-14T09:04:10.783976+0000","last_clean":"2024-03-14T09:04:10.783976+0000","last_became_active":"2024-03-14T09:04:09.790636+0000","last_became_peered":"2024-03-14T09:04:09.790636+0000","last_unstale":"2024-03-14T09:04:10.783976+0000","last_undegraded":"2024-03-14T09:04:10.783976+0000","last_fullsized":"2024-03-14T09:04:10.783976+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:53:48.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783521+0000","last_change":"2024-03-14T09:03:57.186877+0000","last_active":"2024-03-14T09:04:09.783521+0000","last_peered":"2024-03-14T09:04:09.783521+0000","last_clean":"2024-03-14T09:04:09.783521+0000","last_became_active":"2024-03-14T09:03:57.186581+0000","last_became_peered":"2024-03-14T09:03:57.186581+0000","last_unstale":"2024-03-14T09:04:09.783521+0000","last_undegraded":"2024-03-14T09:04:09.783521+0000","last_fullsized":"2024-03-14T09:04:09.783521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:51.324739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400204+0000","last_change":"2024-03-14T09:04:09.789552+0000","last_active":"2024-03-14T09:04:11.400204+0000","last_peered":"2024-03-14T09:04:11.400204+0000","last_clean":"2024-03-14T09:04:11.400204+0000","last_became_active":"2024-03-14T09:04:09.789296+0000","last_became_peered":"2024-03-14T09:04:09.789296+0000","last_unstale":"2024-03-14T09:04:11.400204+0000","last_undegraded":"2024-03-14T09:04:11.400204+0000","last_fullsized":"2024-03-14T09:04:11.400204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:20:45.664696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784704+0000","last_change":"2024-03-14T09:03:57.187026+0000","last_active":"2024-03-14T09:04:09.784704+0000","last_peered":"2024-03-14T09:04:09.784704+0000","last_clean":"2024-03-14T09:04:09.784704+0000","last_became_active":"2024-03-14T09:03:57.186774+0000","last_became_peered":"2024-03-14T09:03:57.186774+0000","last_unstale":"2024-03-14T09:04:09.784704+0000","last_undegraded":"2024-03-14T09:04:09.784704+0000","last_fullsized":"2024-03-14T09:04:09.784704+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:55:25.363869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400786+0000","last_change":"2024-03-14T09:04:09.792469+0000","last_active":"2024-03-14T09:04:11.400786+0000","last_peered":"2024-03-14T09:04:11.400786+0000","last_clean":"2024-03-14T09:04:11.400786+0000","last_became_active":"2024-03-14T09:04:09.791828+0000","last_became_peered":"2024-03-14T09:04:09.791828+0000","last_unstale":"2024-03-14T09:04:11.400786+0000","last_undegraded":"2024-03-14T09:04:11.400786+0000","last_fullsized":"2024-03-14T09:04:11.400786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777838+0000","last_change":"2024-03-14T09:03:57.189350+0000","last_active":"2024-03-14T09:04:09.777838+0000","last_peered":"2024-03-14T09:04:09.777838+0000","last_clean":"2024-03-14T09:04:09.777838+0000","last_became_active":"2024-03-14T09:03:57.189109+0000","last_became_peered":"2024-03-14T09:03:57.189109+0000","last_unstale":"2024-03-14T09:04:09.777838+0000","last_undegraded":"2024-03-14T09:04:09.777838+0000","last_fullsized":"2024-03-14T09:04:09.777838+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:18.878301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784555+0000","last_change":"2024-03-14T09:04:09.796033+0000","last_active":"2024-03-14T09:04:10.784555+0000","last_peered":"2024-03-14T09:04:10.784555+0000","last_clean":"2024-03-14T09:04:10.784555+0000","last_became_active":"2024-03-14T09:04:09.795784+0000","last_became_peered":"2024-03-14T09:04:09.795784+0000","last_unstale":"2024-03-14T09:04:10.784555+0000","last_undegraded":"2024-03-14T09:04:10.784555+0000","last_fullsized":"2024-03-14T09:04:10.784555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:54.494207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777203+0000","last_change":"2024-03-14T09:03:57.186278+0000","last_active":"2024-03-14T09:04:09.777203+0000","last_peered":"2024-03-14T09:04:09.777203+0000","last_clean":"2024-03-14T09:04:09.777203+0000","last_became_active":"2024-03-14T09:03:57.186014+0000","last_became_peered":"2024-03-14T09:03:57.186014+0000","last_unstale":"2024-03-14T09:04:09.777203+0000","last_undegraded":"2024-03-14T09:04:09.777203+0000","last_fullsized":"2024-03-14T09:04:09.777203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400096+0000","last_change":"2024-03-14T09:04:09.790562+0000","last_active":"2024-03-14T09:04:11.400096+0000","last_peered":"2024-03-14T09:04:11.400096+0000","last_clean":"2024-03-14T09:04:11.400096+0000","last_became_active":"2024-03-14T09:04:09.790288+0000","last_became_peered":"2024-03-14T09:04:09.790288+0000","last_unstale":"2024-03-14T09:04:11.400096+0000","last_undegraded":"2024-03-14T09:04:11.400096+0000","last_fullsized":"2024-03-14T09:04:11.400096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:33:45.300921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780089+0000","last_change":"2024-03-14T09:03:57.186749+0000","last_active":"2024-03-14T09:04:09.780089+0000","last_peered":"2024-03-14T09:04:09.780089+0000","last_clean":"2024-03-14T09:04:09.780089+0000","last_became_active":"2024-03-14T09:03:57.186502+0000","last_became_peered":"2024-03-14T09:03:57.186502+0000","last_unstale":"2024-03-14T09:04:09.780089+0000","last_undegraded":"2024-03-14T09:04:09.780089+0000","last_fullsized":"2024-03-14T09:04:09.780089+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:10.996899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401082+0000","last_change":"2024-03-14T09:04:09.797676+0000","last_active":"2024-03-14T09:04:11.401082+0000","last_peered":"2024-03-14T09:04:11.401082+0000","last_clean":"2024-03-14T09:04:11.401082+0000","last_became_active":"2024-03-14T09:04:09.797396+0000","last_became_peered":"2024-03-14T09:04:09.797396+0000","last_unstale":"2024-03-14T09:04:11.401082+0000","last_undegraded":"2024-03-14T09:04:11.401082+0000","last_fullsized":"2024-03-14T09:04:11.401082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:07.293563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782345+0000","last_change":"2024-03-14T09:03:57.185209+0000","last_active":"2024-03-14T09:04:09.782345+0000","last_peered":"2024-03-14T09:04:09.782345+0000","last_clean":"2024-03-14T09:04:09.782345+0000","last_became_active":"2024-03-14T09:03:57.184942+0000","last_became_peered":"2024-03-14T09:03:57.184942+0000","last_unstale":"2024-03-14T09:04:09.782345+0000","last_undegraded":"2024-03-14T09:04:09.782345+0000","last_fullsized":"2024-03-14T09:04:09.782345+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:37.052225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401410+0000","last_change":"2024-03-14T09:04:09.785193+0000","last_active":"2024-03-14T09:04:11.401410+0000","last_peered":"2024-03-14T09:04:11.401410+0000","last_clean":"2024-03-14T09:04:11.401410+0000","last_became_active":"2024-03-14T09:04:09.784910+0000","last_became_peered":"2024-03-14T09:04:09.784910+0000","last_unstale":"2024-03-14T09:04:11.401410+0000","last_undegraded":"2024-03-14T09:04:11.401410+0000","last_fullsized":"2024-03-14T09:04:11.401410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780425+0000","last_change":"2024-03-14T09:03:57.189881+0000","last_active":"2024-03-14T09:04:09.780425+0000","last_peered":"2024-03-14T09:04:09.780425+0000","last_clean":"2024-03-14T09:04:09.780425+0000","last_became_active":"2024-03-14T09:03:57.189632+0000","last_became_peered":"2024-03-14T09:03:57.189632+0000","last_unstale":"2024-03-14T09:04:09.780425+0000","last_undegraded":"2024-03-14T09:04:09.780425+0000","last_fullsized":"2024-03-14T09:04:09.780425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402505+0000","last_change":"2024-03-14T09:04:09.795512+0000","last_active":"2024-03-14T09:04:11.402505+0000","last_peered":"2024-03-14T09:04:11.402505+0000","last_clean":"2024-03-14T09:04:11.402505+0000","last_became_active":"2024-03-14T09:04:09.795282+0000","last_became_peered":"2024-03-14T09:04:09.795282+0000","last_unstale":"2024-03-14T09:04:11.402505+0000","last_undegraded":"2024-03-14T09:04:11.402505+0000","last_fullsized":"2024-03-14T09:04:11.402505+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:43.806093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796548+0000","last_change":"2024-03-14T09:03:57.168978+0000","last_active":"2024-03-14T09:04:09.796548+0000","last_peered":"2024-03-14T09:04:09.796548+0000","last_clean":"2024-03-14T09:04:09.796548+0000","last_became_active":"2024-03-14T09:03:57.167802+0000","last_became_peered":"2024-03-14T09:03:57.167802+0000","last_unstale":"2024-03-14T09:04:09.796548+0000","last_undegraded":"2024-03-14T09:04:09.796548+0000","last_fullsized":"2024-03-14T09:04:09.796548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401609+0000","last_change":"2024-03-14T09:04:09.793855+0000","last_active":"2024-03-14T09:04:11.401609+0000","last_peered":"2024-03-14T09:04:11.401609+0000","last_clean":"2024-03-14T09:04:11.401609+0000","last_became_active":"2024-03-14T09:04:09.793402+0000","last_became_peered":"2024-03-14T09:04:09.793402+0000","last_unstale":"2024-03-14T09:04:11.401609+0000","last_undegraded":"2024-03-14T09:04:11.401609+0000","last_fullsized":"2024-03-14T09:04:11.401609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:53:48.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777708+0000","last_change":"2024-03-14T09:03:57.183439+0000","last_active":"2024-03-14T09:04:09.777708+0000","last_peered":"2024-03-14T09:04:09.777708+0000","last_clean":"2024-03-14T09:04:09.777708+0000","last_became_active":"2024-03-14T09:03:57.183150+0000","last_became_peered":"2024-03-14T09:03:57.183150+0000","last_unstale":"2024-03-14T09:04:09.777708+0000","last_undegraded":"2024-03-14T09:04:09.777708+0000","last_fullsized":"2024-03-14T09:04:09.777708+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783103+0000","last_change":"2024-03-14T09:04:09.784256+0000","last_active":"2024-03-14T09:04:10.783103+0000","last_peered":"2024-03-14T09:04:10.783103+0000","last_clean":"2024-03-14T09:04:10.783103+0000","last_became_active":"2024-03-14T09:04:09.784014+0000","last_became_peered":"2024-03-14T09:04:09.784014+0000","last_unstale":"2024-03-14T09:04:10.783103+0000","last_undegraded":"2024-03-14T09:04:10.783103+0000","last_fullsized":"2024-03-14T09:04:10.783103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:34:27.112960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783362+0000","last_change":"2024-03-14T09:03:57.184782+0000","last_active":"2024-03-14T09:04:09.783362+0000","last_peered":"2024-03-14T09:04:09.783362+0000","last_clean":"2024-03-14T09:04:09.783362+0000","last_became_active":"2024-03-14T09:03:57.184507+0000","last_became_peered":"2024-03-14T09:03:57.184507+0000","last_unstale":"2024-03-14T09:04:09.783362+0000","last_undegraded":"2024-03-14T09:04:09.783362+0000","last_fullsized":"2024-03-14T09:04:09.783362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:48.425158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400912+0000","last_change":"2024-03-14T09:04:09.800044+0000","last_active":"2024-03-14T09:04:11.400912+0000","last_peered":"2024-03-14T09:04:11.400912+0000","last_clean":"2024-03-14T09:04:11.400912+0000","last_became_active":"2024-03-14T09:04:09.799798+0000","last_became_peered":"2024-03-14T09:04:09.799798+0000","last_unstale":"2024-03-14T09:04:11.400912+0000","last_undegraded":"2024-03-14T09:04:11.400912+0000","last_fullsized":"2024-03-14T09:04:11.400912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:30.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777727+0000","last_change":"2024-03-14T09:03:57.185432+0000","last_active":"2024-03-14T09:04:09.777727+0000","last_peered":"2024-03-14T09:04:09.777727+0000","last_clean":"2024-03-14T09:04:09.777727+0000","last_became_active":"2024-03-14T09:03:57.184879+0000","last_became_peered":"2024-03-14T09:03:57.184879+0000","last_unstale":"2024-03-14T09:04:09.777727+0000","last_undegraded":"2024-03-14T09:04:09.777727+0000","last_fullsized":"2024-03-14T09:04:09.777727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:22.429275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400959+0000","last_change":"2024-03-14T09:04:09.791951+0000","last_active":"2024-03-14T09:04:11.400959+0000","last_peered":"2024-03-14T09:04:11.400959+0000","last_clean":"2024-03-14T09:04:11.400959+0000","last_became_active":"2024-03-14T09:04:09.791710+0000","last_became_peered":"2024-03-14T09:04:09.791710+0000","last_unstale":"2024-03-14T09:04:11.400959+0000","last_undegraded":"2024-03-14T09:04:11.400959+0000","last_fullsized":"2024-03-14T09:04:11.400959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:53:59.403833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793523+0000","last_change":"2024-03-14T09:03:57.186461+0000","last_active":"2024-03-14T09:04:09.793523+0000","last_peered":"2024-03-14T09:04:09.793523+0000","last_clean":"2024-03-14T09:04:09.793523+0000","last_became_active":"2024-03-14T09:03:57.186176+0000","last_became_peered":"2024-03-14T09:03:57.186176+0000","last_unstale":"2024-03-14T09:04:09.793523+0000","last_undegraded":"2024-03-14T09:04:09.793523+0000","last_fullsized":"2024-03-14T09:04:09.793523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784476+0000","last_change":"2024-03-14T09:04:09.792952+0000","last_active":"2024-03-14T09:04:10.784476+0000","last_peered":"2024-03-14T09:04:10.784476+0000","last_clean":"2024-03-14T09:04:10.784476+0000","last_became_active":"2024-03-14T09:04:09.792695+0000","last_became_peered":"2024-03-14T09:04:09.792695+0000","last_unstale":"2024-03-14T09:04:10.784476+0000","last_undegraded":"2024-03-14T09:04:10.784476+0000","last_fullsized":"2024-03-14T09:04:10.784476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:30.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776622+0000","last_change":"2024-03-14T09:03:57.185583+0000","last_active":"2024-03-14T09:04:09.776622+0000","last_peered":"2024-03-14T09:04:09.776622+0000","last_clean":"2024-03-14T09:04:09.776622+0000","last_became_active":"2024-03-14T09:03:57.185292+0000","last_became_peered":"2024-03-14T09:03:57.185292+0000","last_unstale":"2024-03-14T09:04:09.776622+0000","last_undegraded":"2024-03-14T09:04:09.776622+0000","last_fullsized":"2024-03-14T09:04:09.776622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400517+0000","last_change":"2024-03-14T09:04:09.800474+0000","last_active":"2024-03-14T09:04:11.400517+0000","last_peered":"2024-03-14T09:04:11.400517+0000","last_clean":"2024-03-14T09:04:11.400517+0000","last_became_active":"2024-03-14T09:04:09.800187+0000","last_became_peered":"2024-03-14T09:04:09.800187+0000","last_unstale":"2024-03-14T09:04:11.400517+0000","last_undegraded":"2024-03-14T09:04:11.400517+0000","last_fullsized":"2024-03-14T09:04:11.400517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:34.901831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401535+0000","last_change":"2024-03-14T09:04:09.375153+0000","last_active":"2024-03-14T09:04:11.401535+0000","last_peered":"2024-03-14T09:04:11.401535+0000","last_clean":"2024-03-14T09:04:11.401535+0000","last_became_active":"2024-03-14T09:04:09.374755+0000","last_became_peered":"2024-03-14T09:04:09.374755+0000","last_unstale":"2024-03-14T09:04:11.401535+0000","last_undegraded":"2024-03-14T09:04:11.401535+0000","last_fullsized":"2024-03-14T09:04:11.401535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:48:49.117112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796295+0000","last_change":"2024-03-14T09:03:57.182704+0000","last_active":"2024-03-14T09:04:09.796295+0000","last_peered":"2024-03-14T09:04:09.796295+0000","last_clean":"2024-03-14T09:04:09.796295+0000","last_became_active":"2024-03-14T09:03:57.182438+0000","last_became_peered":"2024-03-14T09:03:57.182438+0000","last_unstale":"2024-03-14T09:04:09.796295+0000","last_undegraded":"2024-03-14T09:04:09.796295+0000","last_fullsized":"2024-03-14T09:04:09.796295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"69'4","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779504+0000","last_change":"2024-03-14T09:03:56.771973+0000","last_active":"2024-03-14T09:04:09.779504+0000","last_peered":"2024-03-14T09:04:09.779504+0000","last_clean":"2024-03-14T09:04:09.779504+0000","last_became_active":"2024-03-14T09:03:56.771714+0000","last_became_peered":"2024-03-14T09:03:56.771714+0000","last_unstale":"2024-03-14T09:04:09.779504+0000","last_undegraded":"2024-03-14T09:04:09.779504+0000","last_fullsized":"2024-03-14T09:04:09.779504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:17.873268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783195+0000","last_change":"2024-03-14T09:04:09.803323+0000","last_active":"2024-03-14T09:04:10.783195+0000","last_peered":"2024-03-14T09:04:10.783195+0000","last_clean":"2024-03-14T09:04:10.783195+0000","last_became_active":"2024-03-14T09:04:09.801887+0000","last_became_peered":"2024-03-14T09:04:09.801887+0000","last_unstale":"2024-03-14T09:04:10.783195+0000","last_undegraded":"2024-03-14T09:04:10.783195+0000","last_fullsized":"2024-03-14T09:04:10.783195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:02.116523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401359+0000","last_change":"2024-03-14T09:04:09.366923+0000","last_active":"2024-03-14T09:04:11.401359+0000","last_peered":"2024-03-14T09:04:11.401359+0000","last_clean":"2024-03-14T09:04:11.401359+0000","last_became_active":"2024-03-14T09:04:09.366662+0000","last_became_peered":"2024-03-14T09:04:09.366662+0000","last_unstale":"2024-03-14T09:04:11.401359+0000","last_undegraded":"2024-03-14T09:04:11.401359+0000","last_fullsized":"2024-03-14T09:04:11.401359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:13.944461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779306+0000","last_change":"2024-03-14T09:03:57.185961+0000","last_active":"2024-03-14T09:04:09.779306+0000","last_peered":"2024-03-14T09:04:09.779306+0000","last_clean":"2024-03-14T09:04:09.779306+0000","last_became_active":"2024-03-14T09:03:57.185658+0000","last_became_peered":"2024-03-14T09:03:57.185658+0000","last_unstale":"2024-03-14T09:04:09.779306+0000","last_undegraded":"2024-03-14T09:04:09.779306+0000","last_fullsized":"2024-03-14T09:04:09.779306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796199+0000","last_change":"2024-03-14T09:03:56.769740+0000","last_active":"2024-03-14T09:04:09.796199+0000","last_peered":"2024-03-14T09:04:09.796199+0000","last_clean":"2024-03-14T09:04:09.796199+0000","last_became_active":"2024-03-14T09:03:56.769429+0000","last_became_peered":"2024-03-14T09:03:56.769429+0000","last_unstale":"2024-03-14T09:04:09.796199+0000","last_undegraded":"2024-03-14T09:04:09.796199+0000","last_fullsized":"2024-03-14T09:04:09.796199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.785834+0000","last_change":"2024-03-14T09:04:09.792199+0000","last_active":"2024-03-14T09:04:10.785834+0000","last_peered":"2024-03-14T09:04:10.785834+0000","last_clean":"2024-03-14T09:04:10.785834+0000","last_became_active":"2024-03-14T09:04:09.791888+0000","last_became_peered":"2024-03-14T09:04:09.791888+0000","last_unstale":"2024-03-14T09:04:10.785834+0000","last_undegraded":"2024-03-14T09:04:10.785834+0000","last_fullsized":"2024-03-14T09:04:10.785834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:16:57.041712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"73'4","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401441+0000","last_change":"2024-03-14T09:04:09.375907+0000","last_active":"2024-03-14T09:04:11.401441+0000","last_peered":"2024-03-14T09:04:11.401441+0000","last_clean":"2024-03-14T09:04:11.401441+0000","last_became_active":"2024-03-14T09:04:09.375650+0000","last_became_peered":"2024-03-14T09:04:09.375650+0000","last_unstale":"2024-03-14T09:04:11.401441+0000","last_undegraded":"2024-03-14T09:04:11.401441+0000","last_fullsized":"2024-03-14T09:04:11.401441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:33:47.940938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777616+0000","last_change":"2024-03-14T09:03:57.188083+0000","last_active":"2024-03-14T09:04:09.777616+0000","last_peered":"2024-03-14T09:04:09.777616+0000","last_clean":"2024-03-14T09:04:09.777616+0000","last_became_active":"2024-03-14T09:03:57.187832+0000","last_became_peered":"2024-03-14T09:03:57.187832+0000","last_unstale":"2024-03-14T09:04:09.777616+0000","last_undegraded":"2024-03-14T09:04:09.777616+0000","last_fullsized":"2024-03-14T09:04:09.777616+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773860+0000","last_change":"2024-03-14T09:03:56.771927+0000","last_active":"2024-03-14T09:04:09.773860+0000","last_peered":"2024-03-14T09:04:09.773860+0000","last_clean":"2024-03-14T09:04:09.773860+0000","last_became_active":"2024-03-14T09:03:56.771684+0000","last_became_peered":"2024-03-14T09:03:56.771684+0000","last_unstale":"2024-03-14T09:04:09.773860+0000","last_undegraded":"2024-03-14T09:04:09.773860+0000","last_fullsized":"2024-03-14T09:04:09.773860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784332+0000","last_change":"2024-03-14T09:04:09.790695+0000","last_active":"2024-03-14T09:04:10.784332+0000","last_peered":"2024-03-14T09:04:10.784332+0000","last_clean":"2024-03-14T09:04:10.784332+0000","last_became_active":"2024-03-14T09:04:09.790184+0000","last_became_peered":"2024-03-14T09:04:09.790184+0000","last_unstale":"2024-03-14T09:04:10.784332+0000","last_undegraded":"2024-03-14T09:04:10.784332+0000","last_fullsized":"2024-03-14T09:04:10.784332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:53:59.403833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400936+0000","last_change":"2024-03-14T09:04:09.379525+0000","last_active":"2024-03-14T09:04:11.400936+0000","last_peered":"2024-03-14T09:04:11.400936+0000","last_clean":"2024-03-14T09:04:11.400936+0000","last_became_active":"2024-03-14T09:04:09.379216+0000","last_became_peered":"2024-03-14T09:04:09.379216+0000","last_unstale":"2024-03-14T09:04:11.400936+0000","last_undegraded":"2024-03-14T09:04:11.400936+0000","last_fullsized":"2024-03-14T09:04:11.400936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:13.620690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781015+0000","last_change":"2024-03-14T09:03:57.180285+0000","last_active":"2024-03-14T09:04:09.781015+0000","last_peered":"2024-03-14T09:04:09.781015+0000","last_clean":"2024-03-14T09:04:09.781015+0000","last_became_active":"2024-03-14T09:03:57.179838+0000","last_became_peered":"2024-03-14T09:03:57.179838+0000","last_unstale":"2024-03-14T09:04:09.781015+0000","last_undegraded":"2024-03-14T09:04:09.781015+0000","last_fullsized":"2024-03-14T09:04:09.781015+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.775275+0000","last_change":"2024-03-14T09:03:56.772021+0000","last_active":"2024-03-14T09:04:09.775275+0000","last_peered":"2024-03-14T09:04:09.775275+0000","last_clean":"2024-03-14T09:04:09.775275+0000","last_became_active":"2024-03-14T09:03:56.771764+0000","last_became_peered":"2024-03-14T09:03:56.771764+0000","last_unstale":"2024-03-14T09:04:09.775275+0000","last_undegraded":"2024-03-14T09:04:09.775275+0000","last_fullsized":"2024-03-14T09:04:09.775275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:30.727586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782933+0000","last_change":"2024-03-14T09:04:09.791770+0000","last_active":"2024-03-14T09:04:10.782933+0000","last_peered":"2024-03-14T09:04:10.782933+0000","last_clean":"2024-03-14T09:04:10.782933+0000","last_became_active":"2024-03-14T09:04:09.791566+0000","last_became_peered":"2024-03-14T09:04:09.791566+0000","last_unstale":"2024-03-14T09:04:10.782933+0000","last_undegraded":"2024-03-14T09:04:10.782933+0000","last_fullsized":"2024-03-14T09:04:10.782933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:33:45.300921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400327+0000","last_change":"2024-03-14T09:04:09.381693+0000","last_active":"2024-03-14T09:04:11.400327+0000","last_peered":"2024-03-14T09:04:11.400327+0000","last_clean":"2024-03-14T09:04:11.400327+0000","last_became_active":"2024-03-14T09:04:09.381368+0000","last_became_peered":"2024-03-14T09:04:09.381368+0000","last_unstale":"2024-03-14T09:04:11.400327+0000","last_undegraded":"2024-03-14T09:04:11.400327+0000","last_fullsized":"2024-03-14T09:04:11.400327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:54.608231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777577+0000","last_change":"2024-03-14T09:03:57.185267+0000","last_active":"2024-03-14T09:04:09.777577+0000","last_peered":"2024-03-14T09:04:09.777577+0000","last_clean":"2024-03-14T09:04:09.777577+0000","last_became_active":"2024-03-14T09:03:57.184614+0000","last_became_peered":"2024-03-14T09:03:57.184614+0000","last_unstale":"2024-03-14T09:04:09.777577+0000","last_undegraded":"2024-03-14T09:04:09.777577+0000","last_fullsized":"2024-03-14T09:04:09.777577+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:25.314389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793428+0000","last_change":"2024-03-14T09:03:56.770949+0000","last_active":"2024-03-14T09:04:09.793428+0000","last_peered":"2024-03-14T09:04:09.793428+0000","last_clean":"2024-03-14T09:04:09.793428+0000","last_became_active":"2024-03-14T09:03:56.770678+0000","last_became_peered":"2024-03-14T09:03:56.770678+0000","last_unstale":"2024-03-14T09:04:09.793428+0000","last_undegraded":"2024-03-14T09:04:09.793428+0000","last_fullsized":"2024-03-14T09:04:09.793428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783187+0000","last_change":"2024-03-14T09:04:09.800588+0000","last_active":"2024-03-14T09:04:10.783187+0000","last_peered":"2024-03-14T09:04:10.783187+0000","last_clean":"2024-03-14T09:04:10.783187+0000","last_became_active":"2024-03-14T09:04:09.800315+0000","last_became_peered":"2024-03-14T09:04:09.800315+0000","last_unstale":"2024-03-14T09:04:10.783187+0000","last_undegraded":"2024-03-14T09:04:10.783187+0000","last_fullsized":"2024-03-14T09:04:10.783187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:41.207805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.399939+0000","last_change":"2024-03-14T09:04:09.377623+0000","last_active":"2024-03-14T09:04:11.399939+0000","last_peered":"2024-03-14T09:04:11.399939+0000","last_clean":"2024-03-14T09:04:11.399939+0000","last_became_active":"2024-03-14T09:04:09.377333+0000","last_became_peered":"2024-03-14T09:04:09.377333+0000","last_unstale":"2024-03-14T09:04:11.399939+0000","last_undegraded":"2024-03-14T09:04:11.399939+0000","last_fullsized":"2024-03-14T09:04:11.399939+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:53.701753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793884+0000","last_change":"2024-03-14T09:03:57.178070+0000","last_active":"2024-03-14T09:04:09.793884+0000","last_peered":"2024-03-14T09:04:09.793884+0000","last_clean":"2024-03-14T09:04:09.793884+0000","last_became_active":"2024-03-14T09:03:57.177621+0000","last_became_peered":"2024-03-14T09:03:57.177621+0000","last_unstale":"2024-03-14T09:04:09.793884+0000","last_undegraded":"2024-03-14T09:04:09.793884+0000","last_fullsized":"2024-03-14T09:04:09.793884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:55.259556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793461+0000","last_change":"2024-03-14T09:03:56.768242+0000","last_active":"2024-03-14T09:04:09.793461+0000","last_peered":"2024-03-14T09:04:09.793461+0000","last_clean":"2024-03-14T09:04:09.793461+0000","last_became_active":"2024-03-14T09:03:56.767960+0000","last_became_peered":"2024-03-14T09:03:56.767960+0000","last_unstale":"2024-03-14T09:04:09.793461+0000","last_undegraded":"2024-03-14T09:04:09.793461+0000","last_fullsized":"2024-03-14T09:04:09.793461+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400802+0000","last_change":"2024-03-14T09:04:09.797741+0000","last_active":"2024-03-14T09:04:11.400802+0000","last_peered":"2024-03-14T09:04:11.400802+0000","last_clean":"2024-03-14T09:04:11.400802+0000","last_became_active":"2024-03-14T09:04:09.797472+0000","last_became_peered":"2024-03-14T09:04:09.797472+0000","last_unstale":"2024-03-14T09:04:11.400802+0000","last_undegraded":"2024-03-14T09:04:11.400802+0000","last_fullsized":"2024-03-14T09:04:11.400802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:50.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400473+0000","last_change":"2024-03-14T09:04:09.379004+0000","last_active":"2024-03-14T09:04:11.400473+0000","last_peered":"2024-03-14T09:04:11.400473+0000","last_clean":"2024-03-14T09:04:11.400473+0000","last_became_active":"2024-03-14T09:04:09.378729+0000","last_became_peered":"2024-03-14T09:04:09.378729+0000","last_unstale":"2024-03-14T09:04:11.400473+0000","last_undegraded":"2024-03-14T09:04:11.400473+0000","last_fullsized":"2024-03-14T09:04:11.400473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:20.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781239+0000","last_change":"2024-03-14T09:03:57.187305+0000","last_active":"2024-03-14T09:04:09.781239+0000","last_peered":"2024-03-14T09:04:09.781239+0000","last_clean":"2024-03-14T09:04:09.781239+0000","last_became_active":"2024-03-14T09:03:57.186346+0000","last_became_peered":"2024-03-14T09:03:57.186346+0000","last_unstale":"2024-03-14T09:04:09.781239+0000","last_undegraded":"2024-03-14T09:04:09.781239+0000","last_fullsized":"2024-03-14T09:04:09.781239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:04.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773576+0000","last_change":"2024-03-14T09:03:56.773036+0000","last_active":"2024-03-14T09:04:09.773576+0000","last_peered":"2024-03-14T09:04:09.773576+0000","last_clean":"2024-03-14T09:04:09.773576+0000","last_became_active":"2024-03-14T09:03:56.772740+0000","last_became_peered":"2024-03-14T09:03:56.772740+0000","last_unstale":"2024-03-14T09:04:09.773576+0000","last_undegraded":"2024-03-14T09:04:09.773576+0000","last_fullsized":"2024-03-14T09:04:09.773576+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:44.982623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400200+0000","last_change":"2024-03-14T09:04:09.787975+0000","last_active":"2024-03-14T09:04:11.400200+0000","last_peered":"2024-03-14T09:04:11.400200+0000","last_clean":"2024-03-14T09:04:11.400200+0000","last_became_active":"2024-03-14T09:04:09.787646+0000","last_became_peered":"2024-03-14T09:04:09.787646+0000","last_unstale":"2024-03-14T09:04:11.400200+0000","last_undegraded":"2024-03-14T09:04:11.400200+0000","last_fullsized":"2024-03-14T09:04:11.400200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:45:57.150041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400982+0000","last_change":"2024-03-14T09:04:09.381315+0000","last_active":"2024-03-14T09:04:11.400982+0000","last_peered":"2024-03-14T09:04:11.400982+0000","last_clean":"2024-03-14T09:04:11.400982+0000","last_became_active":"2024-03-14T09:04:09.381017+0000","last_became_peered":"2024-03-14T09:04:09.381017+0000","last_unstale":"2024-03-14T09:04:11.400982+0000","last_undegraded":"2024-03-14T09:04:11.400982+0000","last_fullsized":"2024-03-14T09:04:11.400982+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:04:59.084242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782553+0000","last_change":"2024-03-14T09:03:57.172128+0000","last_active":"2024-03-14T09:04:09.782553+0000","last_peered":"2024-03-14T09:04:09.782553+0000","last_clean":"2024-03-14T09:04:09.782553+0000","last_became_active":"2024-03-14T09:03:57.171877+0000","last_became_peered":"2024-03-14T09:03:57.171877+0000","last_unstale":"2024-03-14T09:04:09.782553+0000","last_undegraded":"2024-03-14T09:04:09.782553+0000","last_fullsized":"2024-03-14T09:04:09.782553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:25.355300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780677+0000","last_change":"2024-03-14T09:03:56.769464+0000","last_active":"2024-03-14T09:04:09.780677+0000","last_peered":"2024-03-14T09:04:09.780677+0000","last_clean":"2024-03-14T09:04:09.780677+0000","last_became_active":"2024-03-14T09:03:56.769231+0000","last_became_peered":"2024-03-14T09:03:56.769231+0000","last_unstale":"2024-03-14T09:04:09.780677+0000","last_undegraded":"2024-03-14T09:04:09.780677+0000","last_fullsized":"2024-03-14T09:04:09.780677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:50.319706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401132+0000","last_change":"2024-03-14T09:04:09.779677+0000","last_active":"2024-03-14T09:04:11.401132+0000","last_peered":"2024-03-14T09:04:11.401132+0000","last_clean":"2024-03-14T09:04:11.401132+0000","last_became_active":"2024-03-14T09:04:09.779363+0000","last_became_peered":"2024-03-14T09:04:09.779363+0000","last_unstale":"2024-03-14T09:04:11.401132+0000","last_undegraded":"2024-03-14T09:04:11.401132+0000","last_fullsized":"2024-03-14T09:04:11.401132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"73'3","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401357+0000","last_change":"2024-03-14T09:04:09.377329+0000","last_active":"2024-03-14T09:04:11.401357+0000","last_peered":"2024-03-14T09:04:11.401357+0000","last_clean":"2024-03-14T09:04:11.401357+0000","last_became_active":"2024-03-14T09:04:09.377014+0000","last_became_peered":"2024-03-14T09:04:09.377014+0000","last_unstale":"2024-03-14T09:04:11.401357+0000","last_undegraded":"2024-03-14T09:04:11.401357+0000","last_fullsized":"2024-03-14T09:04:11.401357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:48:49.117112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779309+0000","last_change":"2024-03-14T09:03:57.188282+0000","last_active":"2024-03-14T09:04:09.779309+0000","last_peered":"2024-03-14T09:04:09.779309+0000","last_clean":"2024-03-14T09:04:09.779309+0000","last_became_active":"2024-03-14T09:03:57.188018+0000","last_became_peered":"2024-03-14T09:03:57.188018+0000","last_unstale":"2024-03-14T09:04:09.779309+0000","last_undegraded":"2024-03-14T09:04:09.779309+0000","last_fullsized":"2024-03-14T09:04:09.779309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:22.289306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779742+0000","last_change":"2024-03-14T09:03:56.768124+0000","last_active":"2024-03-14T09:04:09.779742+0000","last_peered":"2024-03-14T09:04:09.779742+0000","last_clean":"2024-03-14T09:04:09.779742+0000","last_became_active":"2024-03-14T09:03:56.766669+0000","last_became_peered":"2024-03-14T09:03:56.766669+0000","last_unstale":"2024-03-14T09:04:09.779742+0000","last_undegraded":"2024-03-14T09:04:09.779742+0000","last_fullsized":"2024-03-14T09:04:09.779742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:11.739150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401238+0000","last_change":"2024-03-14T09:04:09.799468+0000","last_active":"2024-03-14T09:04:11.401238+0000","last_peered":"2024-03-14T09:04:11.401238+0000","last_clean":"2024-03-14T09:04:11.401238+0000","last_became_active":"2024-03-14T09:04:09.799145+0000","last_became_peered":"2024-03-14T09:04:09.799145+0000","last_unstale":"2024-03-14T09:04:11.401238+0000","last_undegraded":"2024-03-14T09:04:11.401238+0000","last_fullsized":"2024-03-14T09:04:11.401238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:04:01.338280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"73'6","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783315+0000","last_change":"2024-03-14T09:04:09.376557+0000","last_active":"2024-03-14T09:04:10.783315+0000","last_peered":"2024-03-14T09:04:10.783315+0000","last_clean":"2024-03-14T09:04:10.783315+0000","last_became_active":"2024-03-14T09:04:09.376141+0000","last_became_peered":"2024-03-14T09:04:09.376141+0000","last_unstale":"2024-03-14T09:04:10.783315+0000","last_undegraded":"2024-03-14T09:04:10.783315+0000","last_fullsized":"2024-03-14T09:04:10.783315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:30:20.802531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785007+0000","last_change":"2024-03-14T09:03:57.179078+0000","last_active":"2024-03-14T09:04:09.785007+0000","last_peered":"2024-03-14T09:04:09.785007+0000","last_clean":"2024-03-14T09:04:09.785007+0000","last_became_active":"2024-03-14T09:03:57.178782+0000","last_became_peered":"2024-03-14T09:03:57.178782+0000","last_unstale":"2024-03-14T09:04:09.785007+0000","last_undegraded":"2024-03-14T09:04:09.785007+0000","last_fullsized":"2024-03-14T09:04:09.785007+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778254+0000","last_change":"2024-03-14T09:03:56.769911+0000","last_active":"2024-03-14T09:04:09.778254+0000","last_peered":"2024-03-14T09:04:09.778254+0000","last_clean":"2024-03-14T09:04:09.778254+0000","last_became_active":"2024-03-14T09:03:56.769598+0000","last_became_peered":"2024-03-14T09:03:56.769598+0000","last_unstale":"2024-03-14T09:04:09.778254+0000","last_undegraded":"2024-03-14T09:04:09.778254+0000","last_fullsized":"2024-03-14T09:04:09.778254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401675+0000","last_change":"2024-03-14T09:04:09.789438+0000","last_active":"2024-03-14T09:04:11.401675+0000","last_peered":"2024-03-14T09:04:11.401675+0000","last_clean":"2024-03-14T09:04:11.401675+0000","last_became_active":"2024-03-14T09:04:09.789207+0000","last_became_peered":"2024-03-14T09:04:09.789207+0000","last_unstale":"2024-03-14T09:04:11.401675+0000","last_undegraded":"2024-03-14T09:04:11.401675+0000","last_fullsized":"2024-03-14T09:04:11.401675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:29:18.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400469+0000","last_change":"2024-03-14T09:04:09.378381+0000","last_active":"2024-03-14T09:04:11.400469+0000","last_peered":"2024-03-14T09:04:11.400469+0000","last_clean":"2024-03-14T09:04:11.400469+0000","last_became_active":"2024-03-14T09:04:09.378135+0000","last_became_peered":"2024-03-14T09:04:09.378135+0000","last_unstale":"2024-03-14T09:04:11.400469+0000","last_undegraded":"2024-03-14T09:04:11.400469+0000","last_fullsized":"2024-03-14T09:04:11.400469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:48.386429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.775095+0000","last_change":"2024-03-14T09:03:57.184589+0000","last_active":"2024-03-14T09:04:09.775095+0000","last_peered":"2024-03-14T09:04:09.775095+0000","last_clean":"2024-03-14T09:04:09.775095+0000","last_became_active":"2024-03-14T09:03:57.184303+0000","last_became_peered":"2024-03-14T09:03:57.184303+0000","last_unstale":"2024-03-14T09:04:09.775095+0000","last_undegraded":"2024-03-14T09:04:09.775095+0000","last_fullsized":"2024-03-14T09:04:09.775095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"69'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778906+0000","last_change":"2024-03-14T09:03:56.766306+0000","last_active":"2024-03-14T09:04:09.778906+0000","last_peered":"2024-03-14T09:04:09.778906+0000","last_clean":"2024-03-14T09:04:09.778906+0000","last_became_active":"2024-03-14T09:03:56.766045+0000","last_became_peered":"2024-03-14T09:03:56.766045+0000","last_unstale":"2024-03-14T09:04:09.778906+0000","last_undegraded":"2024-03-14T09:04:09.778906+0000","last_fullsized":"2024-03-14T09:04:09.778906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783580+0000","last_change":"2024-03-14T09:04:09.803111+0000","last_active":"2024-03-14T09:04:10.783580+0000","last_peered":"2024-03-14T09:04:10.783580+0000","last_clean":"2024-03-14T09:04:10.783580+0000","last_became_active":"2024-03-14T09:04:09.801957+0000","last_became_peered":"2024-03-14T09:04:09.801957+0000","last_unstale":"2024-03-14T09:04:10.783580+0000","last_undegraded":"2024-03-14T09:04:10.783580+0000","last_fullsized":"2024-03-14T09:04:10.783580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:43.806093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784034+0000","last_change":"2024-03-14T09:04:09.381721+0000","last_active":"2024-03-14T09:04:10.784034+0000","last_peered":"2024-03-14T09:04:10.784034+0000","last_clean":"2024-03-14T09:04:10.784034+0000","last_became_active":"2024-03-14T09:04:09.381428+0000","last_became_peered":"2024-03-14T09:04:09.381428+0000","last_unstale":"2024-03-14T09:04:10.784034+0000","last_undegraded":"2024-03-14T09:04:10.784034+0000","last_fullsized":"2024-03-14T09:04:10.784034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:50.175976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796183+0000","last_change":"2024-03-14T09:03:57.189394+0000","last_active":"2024-03-14T09:04:09.796183+0000","last_peered":"2024-03-14T09:04:09.796183+0000","last_clean":"2024-03-14T09:04:09.796183+0000","last_became_active":"2024-03-14T09:03:57.189162+0000","last_became_peered":"2024-03-14T09:03:57.189162+0000","last_unstale":"2024-03-14T09:04:09.796183+0000","last_undegraded":"2024-03-14T09:04:09.796183+0000","last_fullsized":"2024-03-14T09:04:09.796183+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"70'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784554+0000","last_change":"2024-03-14T09:03:56.769295+0000","last_active":"2024-03-14T09:04:09.784554+0000","last_peered":"2024-03-14T09:04:09.784554+0000","last_clean":"2024-03-14T09:04:09.784554+0000","last_became_active":"2024-03-14T09:03:56.768952+0000","last_became_peered":"2024-03-14T09:03:56.768952+0000","last_unstale":"2024-03-14T09:04:09.784554+0000","last_undegraded":"2024-03-14T09:04:09.784554+0000","last_fullsized":"2024-03-14T09:04:09.784554+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782666+0000","last_change":"2024-03-14T09:04:09.797259+0000","last_active":"2024-03-14T09:04:10.782666+0000","last_peered":"2024-03-14T09:04:10.782666+0000","last_clean":"2024-03-14T09:04:10.782666+0000","last_became_active":"2024-03-14T09:04:09.796798+0000","last_became_peered":"2024-03-14T09:04:09.796798+0000","last_unstale":"2024-03-14T09:04:10.782666+0000","last_undegraded":"2024-03-14T09:04:10.782666+0000","last_fullsized":"2024-03-14T09:04:10.782666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:42:01.168724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782720+0000","last_change":"2024-03-14T09:04:09.375024+0000","last_active":"2024-03-14T09:04:10.782720+0000","last_peered":"2024-03-14T09:04:10.782720+0000","last_clean":"2024-03-14T09:04:10.782720+0000","last_became_active":"2024-03-14T09:04:09.372740+0000","last_became_peered":"2024-03-14T09:04:09.372740+0000","last_unstale":"2024-03-14T09:04:10.782720+0000","last_undegraded":"2024-03-14T09:04:10.782720+0000","last_fullsized":"2024-03-14T09:04:10.782720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:14.302651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772985+0000","last_change":"2024-03-14T09:03:57.188359+0000","last_active":"2024-03-14T09:04:09.772985+0000","last_peered":"2024-03-14T09:04:09.772985+0000","last_clean":"2024-03-14T09:04:09.772985+0000","last_became_active":"2024-03-14T09:03:57.188093+0000","last_became_peered":"2024-03-14T09:03:57.188093+0000","last_unstale":"2024-03-14T09:04:09.772985+0000","last_undegraded":"2024-03-14T09:04:09.772985+0000","last_fullsized":"2024-03-14T09:04:09.772985+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783266+0000","last_change":"2024-03-14T09:03:56.769589+0000","last_active":"2024-03-14T09:04:09.783266+0000","last_peered":"2024-03-14T09:04:09.783266+0000","last_clean":"2024-03-14T09:04:09.783266+0000","last_became_active":"2024-03-14T09:03:56.767656+0000","last_became_peered":"2024-03-14T09:03:56.767656+0000","last_unstale":"2024-03-14T09:04:09.783266+0000","last_undegraded":"2024-03-14T09:04:09.783266+0000","last_fullsized":"2024-03-14T09:04:09.783266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:31.954923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783967+0000","last_change":"2024-03-14T09:04:09.801012+0000","last_active":"2024-03-14T09:04:10.783967+0000","last_peered":"2024-03-14T09:04:10.783967+0000","last_clean":"2024-03-14T09:04:10.783967+0000","last_became_active":"2024-03-14T09:04:09.800759+0000","last_became_peered":"2024-03-14T09:04:09.800759+0000","last_unstale":"2024-03-14T09:04:10.783967+0000","last_undegraded":"2024-03-14T09:04:10.783967+0000","last_fullsized":"2024-03-14T09:04:10.783967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:21.346758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"73'8","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400939+0000","last_change":"2024-03-14T09:04:09.376339+0000","last_active":"2024-03-14T09:04:11.400939+0000","last_peered":"2024-03-14T09:04:11.400939+0000","last_clean":"2024-03-14T09:04:11.400939+0000","last_became_active":"2024-03-14T09:04:09.376055+0000","last_became_peered":"2024-03-14T09:04:09.376055+0000","last_unstale":"2024-03-14T09:04:11.400939+0000","last_undegraded":"2024-03-14T09:04:11.400939+0000","last_fullsized":"2024-03-14T09:04:11.400939+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:34.350871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779954+0000","last_change":"2024-03-14T09:03:57.187466+0000","last_active":"2024-03-14T09:04:09.779954+0000","last_peered":"2024-03-14T09:04:09.779954+0000","last_clean":"2024-03-14T09:04:09.779954+0000","last_became_active":"2024-03-14T09:03:57.186590+0000","last_became_peered":"2024-03-14T09:03:57.186590+0000","last_unstale":"2024-03-14T09:04:09.779954+0000","last_undegraded":"2024-03-14T09:04:09.779954+0000","last_fullsized":"2024-03-14T09:04:09.779954+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:04.118845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777088+0000","last_change":"2024-03-14T09:03:56.762545+0000","last_active":"2024-03-14T09:04:09.777088+0000","last_peered":"2024-03-14T09:04:09.777088+0000","last_clean":"2024-03-14T09:04:09.777088+0000","last_became_active":"2024-03-14T09:03:56.762292+0000","last_became_peered":"2024-03-14T09:03:56.762292+0000","last_unstale":"2024-03-14T09:04:09.777088+0000","last_undegraded":"2024-03-14T09:04:09.777088+0000","last_fullsized":"2024-03-14T09:04:09.777088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:03.013048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782972+0000","last_change":"2024-03-14T09:04:09.784017+0000","last_active":"2024-03-14T09:04:10.782972+0000","last_peered":"2024-03-14T09:04:10.782972+0000","last_clean":"2024-03-14T09:04:10.782972+0000","last_became_active":"2024-03-14T09:04:09.783753+0000","last_became_peered":"2024-03-14T09:04:09.783753+0000","last_unstale":"2024-03-14T09:04:10.782972+0000","last_undegraded":"2024-03-14T09:04:10.782972+0000","last_fullsized":"2024-03-14T09:04:10.782972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:47:13.189988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784705+0000","last_change":"2024-03-14T09:04:09.379876+0000","last_active":"2024-03-14T09:04:10.784705+0000","last_peered":"2024-03-14T09:04:10.784705+0000","last_clean":"2024-03-14T09:04:10.784705+0000","last_became_active":"2024-03-14T09:04:09.379335+0000","last_became_peered":"2024-03-14T09:04:09.379335+0000","last_unstale":"2024-03-14T09:04:10.784705+0000","last_undegraded":"2024-03-14T09:04:10.784705+0000","last_fullsized":"2024-03-14T09:04:10.784705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:20:12.537175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780758+0000","last_change":"2024-03-14T09:03:57.184740+0000","last_active":"2024-03-14T09:04:09.780758+0000","last_peered":"2024-03-14T09:04:09.780758+0000","last_clean":"2024-03-14T09:04:09.780758+0000","last_became_active":"2024-03-14T09:03:57.184407+0000","last_became_peered":"2024-03-14T09:03:57.184407+0000","last_unstale":"2024-03-14T09:04:09.780758+0000","last_undegraded":"2024-03-14T09:04:09.780758+0000","last_fullsized":"2024-03-14T09:04:09.780758+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:12.744182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783972+0000","last_change":"2024-03-14T09:03:56.773362+0000","last_active":"2024-03-14T09:04:09.783972+0000","last_peered":"2024-03-14T09:04:09.783972+0000","last_clean":"2024-03-14T09:04:09.783972+0000","last_became_active":"2024-03-14T09:03:56.773012+0000","last_became_peered":"2024-03-14T09:03:56.773012+0000","last_unstale":"2024-03-14T09:04:09.783972+0000","last_undegraded":"2024-03-14T09:04:09.783972+0000","last_fullsized":"2024-03-14T09:04:09.783972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:38.243214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400833+0000","last_change":"2024-03-14T09:04:09.796325+0000","last_active":"2024-03-14T09:04:11.400833+0000","last_peered":"2024-03-14T09:04:11.400833+0000","last_clean":"2024-03-14T09:04:11.400833+0000","last_became_active":"2024-03-14T09:04:09.796077+0000","last_became_peered":"2024-03-14T09:04:09.796077+0000","last_unstale":"2024-03-14T09:04:11.400833+0000","last_undegraded":"2024-03-14T09:04:11.400833+0000","last_fullsized":"2024-03-14T09:04:11.400833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:17:29.957958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784549+0000","last_change":"2024-03-14T09:04:09.379863+0000","last_active":"2024-03-14T09:04:10.784549+0000","last_peered":"2024-03-14T09:04:10.784549+0000","last_clean":"2024-03-14T09:04:10.784549+0000","last_became_active":"2024-03-14T09:04:09.379592+0000","last_became_peered":"2024-03-14T09:04:09.379592+0000","last_unstale":"2024-03-14T09:04:10.784549+0000","last_undegraded":"2024-03-14T09:04:10.784549+0000","last_fullsized":"2024-03-14T09:04:10.784549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:46.907970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778330+0000","last_change":"2024-03-14T09:03:57.187459+0000","last_active":"2024-03-14T09:04:09.778330+0000","last_peered":"2024-03-14T09:04:09.778330+0000","last_clean":"2024-03-14T09:04:09.778330+0000","last_became_active":"2024-03-14T09:03:57.187221+0000","last_became_peered":"2024-03-14T09:03:57.187221+0000","last_unstale":"2024-03-14T09:04:09.778330+0000","last_undegraded":"2024-03-14T09:04:09.778330+0000","last_fullsized":"2024-03-14T09:04:09.778330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:42.577661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"69'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772415+0000","last_change":"2024-03-14T09:03:56.770808+0000","last_active":"2024-03-14T09:04:09.772415+0000","last_peered":"2024-03-14T09:04:09.772415+0000","last_clean":"2024-03-14T09:04:09.772415+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:04:09.772415+0000","last_undegraded":"2024-03-14T09:04:09.772415+0000","last_fullsized":"2024-03-14T09:04:09.772415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:29.358695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400943+0000","last_change":"2024-03-14T09:04:09.790683+0000","last_active":"2024-03-14T09:04:11.400943+0000","last_peered":"2024-03-14T09:04:11.400943+0000","last_clean":"2024-03-14T09:04:11.400943+0000","last_became_active":"2024-03-14T09:04:09.790387+0000","last_became_peered":"2024-03-14T09:04:09.790387+0000","last_unstale":"2024-03-14T09:04:11.400943+0000","last_undegraded":"2024-03-14T09:04:11.400943+0000","last_fullsized":"2024-03-14T09:04:11.400943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:55.431455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402135+0000","last_change":"2024-03-14T09:04:09.372636+0000","last_active":"2024-03-14T09:04:11.402135+0000","last_peered":"2024-03-14T09:04:11.402135+0000","last_clean":"2024-03-14T09:04:11.402135+0000","last_became_active":"2024-03-14T09:04:09.371868+0000","last_became_peered":"2024-03-14T09:04:09.371868+0000","last_unstale":"2024-03-14T09:04:11.402135+0000","last_undegraded":"2024-03-14T09:04:11.402135+0000","last_fullsized":"2024-03-14T09:04:11.402135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:38.539159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780538+0000","last_change":"2024-03-14T09:03:57.182905+0000","last_active":"2024-03-14T09:04:09.780538+0000","last_peered":"2024-03-14T09:04:09.780538+0000","last_clean":"2024-03-14T09:04:09.780538+0000","last_became_active":"2024-03-14T09:03:57.182631+0000","last_became_peered":"2024-03-14T09:03:57.182631+0000","last_unstale":"2024-03-14T09:04:09.780538+0000","last_undegraded":"2024-03-14T09:04:09.780538+0000","last_fullsized":"2024-03-14T09:04:09.780538+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:05.353470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779835+0000","last_change":"2024-03-14T09:03:56.770834+0000","last_active":"2024-03-14T09:04:09.779835+0000","last_peered":"2024-03-14T09:04:09.779835+0000","last_clean":"2024-03-14T09:04:09.779835+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:04:09.779835+0000","last_undegraded":"2024-03-14T09:04:09.779835+0000","last_fullsized":"2024-03-14T09:04:09.779835+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401250+0000","last_change":"2024-03-14T09:04:09.801869+0000","last_active":"2024-03-14T09:04:11.401250+0000","last_peered":"2024-03-14T09:04:11.401250+0000","last_clean":"2024-03-14T09:04:11.401250+0000","last_became_active":"2024-03-14T09:04:09.801150+0000","last_became_peered":"2024-03-14T09:04:09.801150+0000","last_unstale":"2024-03-14T09:04:11.401250+0000","last_undegraded":"2024-03-14T09:04:11.401250+0000","last_fullsized":"2024-03-14T09:04:11.401250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:14.657757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401261+0000","last_change":"2024-03-14T09:04:09.366718+0000","last_active":"2024-03-14T09:04:11.401261+0000","last_peered":"2024-03-14T09:04:11.401261+0000","last_clean":"2024-03-14T09:04:11.401261+0000","last_became_active":"2024-03-14T09:04:09.366449+0000","last_became_peered":"2024-03-14T09:04:09.366449+0000","last_unstale":"2024-03-14T09:04:11.401261+0000","last_undegraded":"2024-03-14T09:04:11.401261+0000","last_fullsized":"2024-03-14T09:04:11.401261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:01.109463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779396+0000","last_change":"2024-03-14T09:03:57.180017+0000","last_active":"2024-03-14T09:04:09.779396+0000","last_peered":"2024-03-14T09:04:09.779396+0000","last_clean":"2024-03-14T09:04:09.779396+0000","last_became_active":"2024-03-14T09:03:57.179589+0000","last_became_peered":"2024-03-14T09:03:57.179589+0000","last_unstale":"2024-03-14T09:04:09.779396+0000","last_undegraded":"2024-03-14T09:04:09.779396+0000","last_fullsized":"2024-03-14T09:04:09.779396+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.774987+0000","last_change":"2024-03-14T09:03:56.769821+0000","last_active":"2024-03-14T09:04:09.774987+0000","last_peered":"2024-03-14T09:04:09.774987+0000","last_clean":"2024-03-14T09:04:09.774987+0000","last_became_active":"2024-03-14T09:03:56.769597+0000","last_became_peered":"2024-03-14T09:03:56.769597+0000","last_unstale":"2024-03-14T09:04:09.774987+0000","last_undegraded":"2024-03-14T09:04:09.774987+0000","last_fullsized":"2024-03-14T09:04:09.774987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:01.018014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402518+0000","last_change":"2024-03-14T09:04:09.800352+0000","last_active":"2024-03-14T09:04:11.402518+0000","last_peered":"2024-03-14T09:04:11.402518+0000","last_clean":"2024-03-14T09:04:11.402518+0000","last_became_active":"2024-03-14T09:04:09.800092+0000","last_became_peered":"2024-03-14T09:04:09.800092+0000","last_unstale":"2024-03-14T09:04:11.402518+0000","last_undegraded":"2024-03-14T09:04:11.402518+0000","last_fullsized":"2024-03-14T09:04:11.402518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:49.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783445+0000","last_change":"2024-03-14T09:04:09.382043+0000","last_active":"2024-03-14T09:04:10.783445+0000","last_peered":"2024-03-14T09:04:10.783445+0000","last_clean":"2024-03-14T09:04:10.783445+0000","last_became_active":"2024-03-14T09:04:09.381757+0000","last_became_peered":"2024-03-14T09:04:09.381757+0000","last_unstale":"2024-03-14T09:04:10.783445+0000","last_undegraded":"2024-03-14T09:04:10.783445+0000","last_fullsized":"2024-03-14T09:04:10.783445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:27.468551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796062+0000","last_change":"2024-03-14T09:03:57.187274+0000","last_active":"2024-03-14T09:04:09.796062+0000","last_peered":"2024-03-14T09:04:09.796062+0000","last_clean":"2024-03-14T09:04:09.796062+0000","last_became_active":"2024-03-14T09:03:57.187035+0000","last_became_peered":"2024-03-14T09:03:57.187035+0000","last_unstale":"2024-03-14T09:04:09.796062+0000","last_undegraded":"2024-03-14T09:04:09.796062+0000","last_fullsized":"2024-03-14T09:04:09.796062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772554+0000","last_change":"2024-03-14T09:03:56.772665+0000","last_active":"2024-03-14T09:04:09.772554+0000","last_peered":"2024-03-14T09:04:09.772554+0000","last_clean":"2024-03-14T09:04:09.772554+0000","last_became_active":"2024-03-14T09:03:56.772464+0000","last_became_peered":"2024-03-14T09:03:56.772464+0000","last_unstale":"2024-03-14T09:04:09.772554+0000","last_undegraded":"2024-03-14T09:04:09.772554+0000","last_fullsized":"2024-03-14T09:04:09.772554+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:22.876530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783733+0000","last_change":"2024-03-14T09:04:09.785512+0000","last_active":"2024-03-14T09:04:10.783733+0000","last_peered":"2024-03-14T09:04:10.783733+0000","last_clean":"2024-03-14T09:04:10.783733+0000","last_became_active":"2024-03-14T09:04:09.785267+0000","last_became_peered":"2024-03-14T09:04:09.785267+0000","last_unstale":"2024-03-14T09:04:10.783733+0000","last_undegraded":"2024-03-14T09:04:10.783733+0000","last_fullsized":"2024-03-14T09:04:10.783733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:54.708813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.399916+0000","last_change":"2024-03-14T09:04:09.380582+0000","last_active":"2024-03-14T09:04:11.399916+0000","last_peered":"2024-03-14T09:04:11.399916+0000","last_clean":"2024-03-14T09:04:11.399916+0000","last_became_active":"2024-03-14T09:04:09.380312+0000","last_became_peered":"2024-03-14T09:04:09.380312+0000","last_unstale":"2024-03-14T09:04:11.399916+0000","last_undegraded":"2024-03-14T09:04:11.399916+0000","last_fullsized":"2024-03-14T09:04:11.399916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:33.894772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785884+0000","last_change":"2024-03-14T09:03:57.191774+0000","last_active":"2024-03-14T09:04:09.785884+0000","last_peered":"2024-03-14T09:04:09.785884+0000","last_clean":"2024-03-14T09:04:09.785884+0000","last_became_active":"2024-03-14T09:03:57.191391+0000","last_became_peered":"2024-03-14T09:03:57.191391+0000","last_unstale":"2024-03-14T09:04:09.785884+0000","last_undegraded":"2024-03-14T09:04:09.785884+0000","last_fullsized":"2024-03-14T09:04:09.785884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:22.069879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779049+0000","last_change":"2024-03-14T09:03:56.759918+0000","last_active":"2024-03-14T09:04:09.779049+0000","last_peered":"2024-03-14T09:04:09.779049+0000","last_clean":"2024-03-14T09:04:09.779049+0000","last_became_active":"2024-03-14T09:03:56.759495+0000","last_became_peered":"2024-03-14T09:03:56.759495+0000","last_unstale":"2024-03-14T09:04:09.779049+0000","last_undegraded":"2024-03-14T09:04:09.779049+0000","last_fullsized":"2024-03-14T09:04:09.779049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:04.348438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400229+0000","last_change":"2024-03-14T09:04:09.784010+0000","last_active":"2024-03-14T09:04:11.400229+0000","last_peered":"2024-03-14T09:04:11.400229+0000","last_clean":"2024-03-14T09:04:11.400229+0000","last_became_active":"2024-03-14T09:04:09.783599+0000","last_became_peered":"2024-03-14T09:04:09.783599+0000","last_unstale":"2024-03-14T09:04:11.400229+0000","last_undegraded":"2024-03-14T09:04:11.400229+0000","last_fullsized":"2024-03-14T09:04:11.400229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:45:35.790856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400280+0000","last_change":"2024-03-14T09:04:09.373852+0000","last_active":"2024-03-14T09:04:11.400280+0000","last_peered":"2024-03-14T09:04:11.400280+0000","last_clean":"2024-03-14T09:04:11.400280+0000","last_became_active":"2024-03-14T09:04:09.373535+0000","last_became_peered":"2024-03-14T09:04:09.373535+0000","last_unstale":"2024-03-14T09:04:11.400280+0000","last_undegraded":"2024-03-14T09:04:11.400280+0000","last_fullsized":"2024-03-14T09:04:11.400280+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:13.620690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784382+0000","last_change":"2024-03-14T09:03:57.181268+0000","last_active":"2024-03-14T09:04:09.784382+0000","last_peered":"2024-03-14T09:04:09.784382+0000","last_clean":"2024-03-14T09:04:09.784382+0000","last_became_active":"2024-03-14T09:03:57.180967+0000","last_became_peered":"2024-03-14T09:03:57.180967+0000","last_unstale":"2024-03-14T09:04:09.784382+0000","last_undegraded":"2024-03-14T09:04:09.784382+0000","last_fullsized":"2024-03-14T09:04:09.784382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.786001+0000","last_change":"2024-03-14T09:03:56.768053+0000","last_active":"2024-03-14T09:04:09.786001+0000","last_peered":"2024-03-14T09:04:09.786001+0000","last_clean":"2024-03-14T09:04:09.786001+0000","last_became_active":"2024-03-14T09:03:56.767780+0000","last_became_peered":"2024-03-14T09:03:56.767780+0000","last_unstale":"2024-03-14T09:04:09.786001+0000","last_undegraded":"2024-03-14T09:04:09.786001+0000","last_fullsized":"2024-03-14T09:04:09.786001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:00:23.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784387+0000","last_change":"2024-03-14T09:04:09.794287+0000","last_active":"2024-03-14T09:04:10.784387+0000","last_peered":"2024-03-14T09:04:10.784387+0000","last_clean":"2024-03-14T09:04:10.784387+0000","last_became_active":"2024-03-14T09:04:09.794047+0000","last_became_peered":"2024-03-14T09:04:09.794047+0000","last_unstale":"2024-03-14T09:04:10.784387+0000","last_undegraded":"2024-03-14T09:04:10.784387+0000","last_fullsized":"2024-03-14T09:04:10.784387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:49.393489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401086+0000","last_change":"2024-03-14T09:04:09.380471+0000","last_active":"2024-03-14T09:04:11.401086+0000","last_peered":"2024-03-14T09:04:11.401086+0000","last_clean":"2024-03-14T09:04:11.401086+0000","last_became_active":"2024-03-14T09:04:09.380219+0000","last_became_peered":"2024-03-14T09:04:09.380219+0000","last_unstale":"2024-03-14T09:04:11.401086+0000","last_undegraded":"2024-03-14T09:04:11.401086+0000","last_fullsized":"2024-03-14T09:04:11.401086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:50.175976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780176+0000","last_change":"2024-03-14T09:03:57.177141+0000","last_active":"2024-03-14T09:04:09.780176+0000","last_peered":"2024-03-14T09:04:09.780176+0000","last_clean":"2024-03-14T09:04:09.780176+0000","last_became_active":"2024-03-14T09:03:57.176843+0000","last_became_peered":"2024-03-14T09:03:57.176843+0000","last_unstale":"2024-03-14T09:04:09.780176+0000","last_undegraded":"2024-03-14T09:04:09.780176+0000","last_fullsized":"2024-03-14T09:04:09.780176+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:57.185084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780207+0000","last_change":"2024-03-14T09:03:56.764413+0000","last_active":"2024-03-14T09:04:09.780207+0000","last_peered":"2024-03-14T09:04:09.780207+0000","last_clean":"2024-03-14T09:04:09.780207+0000","last_became_active":"2024-03-14T09:03:56.764156+0000","last_became_peered":"2024-03-14T09:03:56.764156+0000","last_unstale":"2024-03-14T09:04:09.780207+0000","last_undegraded":"2024-03-14T09:04:09.780207+0000","last_fullsized":"2024-03-14T09:04:09.780207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:21:29.624660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784985+0000","last_change":"2024-03-14T09:04:09.799707+0000","last_active":"2024-03-14T09:04:10.784985+0000","last_peered":"2024-03-14T09:04:10.784985+0000","last_clean":"2024-03-14T09:04:10.784985+0000","last_became_active":"2024-03-14T09:04:09.799456+0000","last_became_peered":"2024-03-14T09:04:09.799456+0000","last_unstale":"2024-03-14T09:04:10.784985+0000","last_undegraded":"2024-03-14T09:04:10.784985+0000","last_fullsized":"2024-03-14T09:04:10.784985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:51.043997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400971+0000","last_change":"2024-03-14T09:04:09.379642+0000","last_active":"2024-03-14T09:04:11.400971+0000","last_peered":"2024-03-14T09:04:11.400971+0000","last_clean":"2024-03-14T09:04:11.400971+0000","last_became_active":"2024-03-14T09:04:09.379275+0000","last_became_peered":"2024-03-14T09:04:09.379275+0000","last_unstale":"2024-03-14T09:04:11.400971+0000","last_undegraded":"2024-03-14T09:04:11.400971+0000","last_fullsized":"2024-03-14T09:04:11.400971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:15.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784143+0000","last_change":"2024-03-14T09:03:57.187334+0000","last_active":"2024-03-14T09:04:09.784143+0000","last_peered":"2024-03-14T09:04:09.784143+0000","last_clean":"2024-03-14T09:04:09.784143+0000","last_became_active":"2024-03-14T09:03:57.187054+0000","last_became_peered":"2024-03-14T09:03:57.187054+0000","last_unstale":"2024-03-14T09:04:09.784143+0000","last_undegraded":"2024-03-14T09:04:09.784143+0000","last_fullsized":"2024-03-14T09:04:09.784143+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776720+0000","last_change":"2024-03-14T09:03:56.759664+0000","last_active":"2024-03-14T09:04:09.776720+0000","last_peered":"2024-03-14T09:04:09.776720+0000","last_clean":"2024-03-14T09:04:09.776720+0000","last_became_active":"2024-03-14T09:03:56.759378+0000","last_became_peered":"2024-03-14T09:03:56.759378+0000","last_unstale":"2024-03-14T09:04:09.776720+0000","last_undegraded":"2024-03-14T09:04:09.776720+0000","last_fullsized":"2024-03-14T09:04:09.776720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:03.113813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400635+0000","last_change":"2024-03-14T09:04:09.787270+0000","last_active":"2024-03-14T09:04:11.400635+0000","last_peered":"2024-03-14T09:04:11.400635+0000","last_clean":"2024-03-14T09:04:11.400635+0000","last_became_active":"2024-03-14T09:04:09.787007+0000","last_became_peered":"2024-03-14T09:04:09.787007+0000","last_unstale":"2024-03-14T09:04:11.400635+0000","last_undegraded":"2024-03-14T09:04:11.400635+0000","last_fullsized":"2024-03-14T09:04:11.400635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:16.528447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401489+0000","last_change":"2024-03-14T09:04:09.367225+0000","last_active":"2024-03-14T09:04:11.401489+0000","last_peered":"2024-03-14T09:04:11.401489+0000","last_clean":"2024-03-14T09:04:11.401489+0000","last_became_active":"2024-03-14T09:04:09.366961+0000","last_became_peered":"2024-03-14T09:04:09.366961+0000","last_unstale":"2024-03-14T09:04:11.401489+0000","last_undegraded":"2024-03-14T09:04:11.401489+0000","last_fullsized":"2024-03-14T09:04:11.401489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:42:00.161664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.795987+0000","last_change":"2024-03-14T09:03:57.177315+0000","last_active":"2024-03-14T09:04:09.795987+0000","last_peered":"2024-03-14T09:04:09.795987+0000","last_clean":"2024-03-14T09:04:09.795987+0000","last_became_active":"2024-03-14T09:03:57.177045+0000","last_became_peered":"2024-03-14T09:03:57.177045+0000","last_unstale":"2024-03-14T09:04:09.795987+0000","last_undegraded":"2024-03-14T09:04:09.795987+0000","last_fullsized":"2024-03-14T09:04:09.795987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782430+0000","last_change":"2024-03-14T09:03:56.764563+0000","last_active":"2024-03-14T09:04:09.782430+0000","last_peered":"2024-03-14T09:04:09.782430+0000","last_clean":"2024-03-14T09:04:09.782430+0000","last_became_active":"2024-03-14T09:03:56.764299+0000","last_became_peered":"2024-03-14T09:03:56.764299+0000","last_unstale":"2024-03-14T09:04:09.782430+0000","last_undegraded":"2024-03-14T09:04:09.782430+0000","last_fullsized":"2024-03-14T09:04:09.782430+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:22.457392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402011+0000","last_change":"2024-03-14T09:04:09.786880+0000","last_active":"2024-03-14T09:04:11.402011+0000","last_peered":"2024-03-14T09:04:11.402011+0000","last_clean":"2024-03-14T09:04:11.402011+0000","last_became_active":"2024-03-14T09:04:09.786581+0000","last_became_peered":"2024-03-14T09:04:09.786581+0000","last_unstale":"2024-03-14T09:04:11.402011+0000","last_undegraded":"2024-03-14T09:04:11.402011+0000","last_fullsized":"2024-03-14T09:04:11.402011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:43:14.433260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401213+0000","last_change":"2024-03-14T09:04:09.373374+0000","last_active":"2024-03-14T09:04:11.401213+0000","last_peered":"2024-03-14T09:04:11.401213+0000","last_clean":"2024-03-14T09:04:11.401213+0000","last_became_active":"2024-03-14T09:04:09.372987+0000","last_became_peered":"2024-03-14T09:04:09.372987+0000","last_unstale":"2024-03-14T09:04:11.401213+0000","last_undegraded":"2024-03-14T09:04:11.401213+0000","last_fullsized":"2024-03-14T09:04:11.401213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:54.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780681+0000","last_change":"2024-03-14T09:03:57.182451+0000","last_active":"2024-03-14T09:04:09.780681+0000","last_peered":"2024-03-14T09:04:09.780681+0000","last_clean":"2024-03-14T09:04:09.780681+0000","last_became_active":"2024-03-14T09:03:57.182007+0000","last_became_peered":"2024-03-14T09:03:57.182007+0000","last_unstale":"2024-03-14T09:04:09.780681+0000","last_undegraded":"2024-03-14T09:04:09.780681+0000","last_fullsized":"2024-03-14T09:04:09.780681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"70'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781298+0000","last_change":"2024-03-14T09:03:56.768285+0000","last_active":"2024-03-14T09:04:09.781298+0000","last_peered":"2024-03-14T09:04:09.781298+0000","last_clean":"2024-03-14T09:04:09.781298+0000","last_became_active":"2024-03-14T09:03:56.768060+0000","last_became_peered":"2024-03-14T09:03:56.768060+0000","last_unstale":"2024-03-14T09:04:09.781298+0000","last_undegraded":"2024-03-14T09:04:09.781298+0000","last_fullsized":"2024-03-14T09:04:09.781298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:56.180051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401564+0000","last_change":"2024-03-14T09:04:09.788779+0000","last_active":"2024-03-14T09:04:11.401564+0000","last_peered":"2024-03-14T09:04:11.401564+0000","last_clean":"2024-03-14T09:04:11.401564+0000","last_became_active":"2024-03-14T09:04:09.788165+0000","last_became_peered":"2024-03-14T09:04:09.788165+0000","last_unstale":"2024-03-14T09:04:11.401564+0000","last_undegraded":"2024-03-14T09:04:11.401564+0000","last_fullsized":"2024-03-14T09:04:11.401564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401219+0000","last_change":"2024-03-14T09:04:09.372263+0000","last_active":"2024-03-14T09:04:11.401219+0000","last_peered":"2024-03-14T09:04:11.401219+0000","last_clean":"2024-03-14T09:04:11.401219+0000","last_became_active":"2024-03-14T09:04:09.371457+0000","last_became_peered":"2024-03-14T09:04:09.371457+0000","last_unstale":"2024-03-14T09:04:11.401219+0000","last_undegraded":"2024-03-14T09:04:11.401219+0000","last_fullsized":"2024-03-14T09:04:11.401219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:17:28.950898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777523+0000","last_change":"2024-03-14T09:03:57.188009+0000","last_active":"2024-03-14T09:04:09.777523+0000","last_peered":"2024-03-14T09:04:09.777523+0000","last_clean":"2024-03-14T09:04:09.777523+0000","last_became_active":"2024-03-14T09:03:57.187745+0000","last_became_peered":"2024-03-14T09:03:57.187745+0000","last_unstale":"2024-03-14T09:04:09.777523+0000","last_undegraded":"2024-03-14T09:04:09.777523+0000","last_fullsized":"2024-03-14T09:04:09.777523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779017+0000","last_change":"2024-03-14T09:03:56.764742+0000","last_active":"2024-03-14T09:04:09.779017+0000","last_peered":"2024-03-14T09:04:09.779017+0000","last_clean":"2024-03-14T09:04:09.779017+0000","last_became_active":"2024-03-14T09:03:56.764467+0000","last_became_peered":"2024-03-14T09:03:56.764467+0000","last_unstale":"2024-03-14T09:04:09.779017+0000","last_undegraded":"2024-03-14T09:04:09.779017+0000","last_fullsized":"2024-03-14T09:04:09.779017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400386+0000","last_change":"2024-03-14T09:04:09.785569+0000","last_active":"2024-03-14T09:04:11.400386+0000","last_peered":"2024-03-14T09:04:11.400386+0000","last_clean":"2024-03-14T09:04:11.400386+0000","last_became_active":"2024-03-14T09:04:09.785326+0000","last_became_peered":"2024-03-14T09:04:09.785326+0000","last_unstale":"2024-03-14T09:04:11.400386+0000","last_undegraded":"2024-03-14T09:04:11.400386+0000","last_fullsized":"2024-03-14T09:04:11.400386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:15.309710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"74'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.693625+0000","last_change":"2024-03-14T09:04:09.382499+0000","last_active":"2024-03-14T09:04:11.693625+0000","last_peered":"2024-03-14T09:04:11.693625+0000","last_clean":"2024-03-14T09:04:11.693625+0000","last_became_active":"2024-03-14T09:04:09.381995+0000","last_became_peered":"2024-03-14T09:04:09.381995+0000","last_unstale":"2024-03-14T09:04:11.693625+0000","last_undegraded":"2024-03-14T09:04:11.693625+0000","last_fullsized":"2024-03-14T09:04:11.693625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:42.799033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.795972+0000","last_change":"2024-03-14T09:03:57.189309+0000","last_active":"2024-03-14T09:04:09.795972+0000","last_peered":"2024-03-14T09:04:09.795972+0000","last_clean":"2024-03-14T09:04:09.795972+0000","last_became_active":"2024-03-14T09:03:57.189127+0000","last_became_peered":"2024-03-14T09:03:57.189127+0000","last_unstale":"2024-03-14T09:04:09.795972+0000","last_undegraded":"2024-03-14T09:04:09.795972+0000","last_fullsized":"2024-03-14T09:04:09.795972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777807+0000","last_change":"2024-03-14T09:03:56.770115+0000","last_active":"2024-03-14T09:04:09.777807+0000","last_peered":"2024-03-14T09:04:09.777807+0000","last_clean":"2024-03-14T09:04:09.777807+0000","last_became_active":"2024-03-14T09:03:56.769621+0000","last_became_peered":"2024-03-14T09:03:56.769621+0000","last_unstale":"2024-03-14T09:04:09.777807+0000","last_undegraded":"2024-03-14T09:04:09.777807+0000","last_fullsized":"2024-03-14T09:04:09.777807+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783829+0000","last_change":"2024-03-14T09:04:09.796371+0000","last_active":"2024-03-14T09:04:10.783829+0000","last_peered":"2024-03-14T09:04:10.783829+0000","last_clean":"2024-03-14T09:04:10.783829+0000","last_became_active":"2024-03-14T09:04:09.796095+0000","last_became_peered":"2024-03-14T09:04:09.796095+0000","last_unstale":"2024-03-14T09:04:10.783829+0000","last_undegraded":"2024-03-14T09:04:10.783829+0000","last_fullsized":"2024-03-14T09:04:10.783829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:35.357930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783170+0000","last_change":"2024-03-14T09:04:09.379745+0000","last_active":"2024-03-14T09:04:10.783170+0000","last_peered":"2024-03-14T09:04:10.783170+0000","last_clean":"2024-03-14T09:04:10.783170+0000","last_became_active":"2024-03-14T09:04:09.379484+0000","last_became_peered":"2024-03-14T09:04:09.379484+0000","last_unstale":"2024-03-14T09:04:10.783170+0000","last_undegraded":"2024-03-14T09:04:10.783170+0000","last_fullsized":"2024-03-14T09:04:10.783170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:42.799033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784375+0000","last_change":"2024-03-14T09:03:57.180976+0000","last_active":"2024-03-14T09:04:09.784375+0000","last_peered":"2024-03-14T09:04:09.784375+0000","last_clean":"2024-03-14T09:04:09.784375+0000","last_became_active":"2024-03-14T09:03:57.180689+0000","last_became_peered":"2024-03-14T09:03:57.180689+0000","last_unstale":"2024-03-14T09:04:09.784375+0000","last_undegraded":"2024-03-14T09:04:09.784375+0000","last_fullsized":"2024-03-14T09:04:09.784375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:02.023046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773163+0000","last_change":"2024-03-14T09:03:56.770777+0000","last_active":"2024-03-14T09:04:09.773163+0000","last_peered":"2024-03-14T09:04:09.773163+0000","last_clean":"2024-03-14T09:04:09.773163+0000","last_became_active":"2024-03-14T09:03:56.770466+0000","last_became_peered":"2024-03-14T09:03:56.770466+0000","last_unstale":"2024-03-14T09:04:09.773163+0000","last_undegraded":"2024-03-14T09:04:09.773163+0000","last_fullsized":"2024-03-14T09:04:09.773163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:41.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401051+0000","last_change":"2024-03-14T09:04:09.787284+0000","last_active":"2024-03-14T09:04:11.401051+0000","last_peered":"2024-03-14T09:04:11.401051+0000","last_clean":"2024-03-14T09:04:11.401051+0000","last_became_active":"2024-03-14T09:04:09.787007+0000","last_became_peered":"2024-03-14T09:04:09.787007+0000","last_unstale":"2024-03-14T09:04:11.401051+0000","last_undegraded":"2024-03-14T09:04:11.401051+0000","last_fullsized":"2024-03-14T09:04:11.401051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:00.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783909+0000","last_change":"2024-03-14T09:04:09.377195+0000","last_active":"2024-03-14T09:04:10.783909+0000","last_peered":"2024-03-14T09:04:10.783909+0000","last_clean":"2024-03-14T09:04:10.783909+0000","last_became_active":"2024-03-14T09:04:09.376963+0000","last_became_peered":"2024-03-14T09:04:09.376963+0000","last_unstale":"2024-03-14T09:04:10.783909+0000","last_undegraded":"2024-03-14T09:04:10.783909+0000","last_fullsized":"2024-03-14T09:04:10.783909+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:47:06.421840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773018+0000","last_change":"2024-03-14T09:03:57.184709+0000","last_active":"2024-03-14T09:04:09.773018+0000","last_peered":"2024-03-14T09:04:09.773018+0000","last_clean":"2024-03-14T09:04:09.773018+0000","last_became_active":"2024-03-14T09:03:57.184380+0000","last_became_peered":"2024-03-14T09:03:57.184380+0000","last_unstale":"2024-03-14T09:04:09.773018+0000","last_undegraded":"2024-03-14T09:04:09.773018+0000","last_fullsized":"2024-03-14T09:04:09.773018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793909+0000","last_change":"2024-03-14T09:03:56.769939+0000","last_active":"2024-03-14T09:04:09.793909+0000","last_peered":"2024-03-14T09:04:09.793909+0000","last_clean":"2024-03-14T09:04:09.793909+0000","last_became_active":"2024-03-14T09:03:56.769670+0000","last_became_peered":"2024-03-14T09:03:56.769670+0000","last_unstale":"2024-03-14T09:04:09.793909+0000","last_undegraded":"2024-03-14T09:04:09.793909+0000","last_fullsized":"2024-03-14T09:04:09.793909+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:50.421546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784637+0000","last_change":"2024-03-14T09:04:09.796184+0000","last_active":"2024-03-14T09:04:10.784637+0000","last_peered":"2024-03-14T09:04:10.784637+0000","last_clean":"2024-03-14T09:04:10.784637+0000","last_became_active":"2024-03-14T09:04:09.795957+0000","last_became_peered":"2024-03-14T09:04:09.795957+0000","last_unstale":"2024-03-14T09:04:10.784637+0000","last_undegraded":"2024-03-14T09:04:10.784637+0000","last_fullsized":"2024-03-14T09:04:10.784637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:43.674655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401111+0000","last_change":"2024-03-14T09:04:09.369844+0000","last_active":"2024-03-14T09:04:11.401111+0000","last_peered":"2024-03-14T09:04:11.401111+0000","last_clean":"2024-03-14T09:04:11.401111+0000","last_became_active":"2024-03-14T09:04:09.369584+0000","last_became_peered":"2024-03-14T09:04:09.369584+0000","last_unstale":"2024-03-14T09:04:11.401111+0000","last_undegraded":"2024-03-14T09:04:11.401111+0000","last_fullsized":"2024-03-14T09:04:11.401111+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:48.812333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781410+0000","last_change":"2024-03-14T09:03:57.187259+0000","last_active":"2024-03-14T09:04:09.781410+0000","last_peered":"2024-03-14T09:04:09.781410+0000","last_clean":"2024-03-14T09:04:09.781410+0000","last_became_active":"2024-03-14T09:03:57.186972+0000","last_became_peered":"2024-03-14T09:03:57.186972+0000","last_unstale":"2024-03-14T09:04:09.781410+0000","last_undegraded":"2024-03-14T09:04:09.781410+0000","last_fullsized":"2024-03-14T09:04:09.781410+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777363+0000","last_change":"2024-03-14T09:03:56.771166+0000","last_active":"2024-03-14T09:04:09.777363+0000","last_peered":"2024-03-14T09:04:09.777363+0000","last_clean":"2024-03-14T09:04:09.777363+0000","last_became_active":"2024-03-14T09:03:56.769678+0000","last_became_peered":"2024-03-14T09:03:56.769678+0000","last_unstale":"2024-03-14T09:04:09.777363+0000","last_undegraded":"2024-03-14T09:04:09.777363+0000","last_fullsized":"2024-03-14T09:04:09.777363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:06.742278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783402+0000","last_change":"2024-03-14T09:04:09.796461+0000","last_active":"2024-03-14T09:04:10.783402+0000","last_peered":"2024-03-14T09:04:10.783402+0000","last_clean":"2024-03-14T09:04:10.783402+0000","last_became_active":"2024-03-14T09:04:09.796193+0000","last_became_peered":"2024-03-14T09:04:09.796193+0000","last_unstale":"2024-03-14T09:04:10.783402+0000","last_undegraded":"2024-03-14T09:04:10.783402+0000","last_fullsized":"2024-03-14T09:04:10.783402+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:14.951521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400276+0000","last_change":"2024-03-14T09:04:09.382491+0000","last_active":"2024-03-14T09:04:11.400276+0000","last_peered":"2024-03-14T09:04:11.400276+0000","last_clean":"2024-03-14T09:04:11.400276+0000","last_became_active":"2024-03-14T09:04:09.382211+0000","last_became_peered":"2024-03-14T09:04:09.382211+0000","last_unstale":"2024-03-14T09:04:11.400276+0000","last_undegraded":"2024-03-14T09:04:11.400276+0000","last_fullsized":"2024-03-14T09:04:11.400276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:49.293407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777276+0000","last_change":"2024-03-14T09:03:57.181824+0000","last_active":"2024-03-14T09:04:09.777276+0000","last_peered":"2024-03-14T09:04:09.777276+0000","last_clean":"2024-03-14T09:04:09.777276+0000","last_became_active":"2024-03-14T09:03:57.181573+0000","last_became_peered":"2024-03-14T09:03:57.181573+0000","last_unstale":"2024-03-14T09:04:09.777276+0000","last_undegraded":"2024-03-14T09:04:09.777276+0000","last_fullsized":"2024-03-14T09:04:09.777276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793755+0000","last_change":"2024-03-14T09:03:56.770803+0000","last_active":"2024-03-14T09:04:09.793755+0000","last_peered":"2024-03-14T09:04:09.793755+0000","last_clean":"2024-03-14T09:04:09.793755+0000","last_became_active":"2024-03-14T09:03:56.770311+0000","last_became_peered":"2024-03-14T09:03:56.770311+0000","last_unstale":"2024-03-14T09:04:09.793755+0000","last_undegraded":"2024-03-14T09:04:09.793755+0000","last_fullsized":"2024-03-14T09:04:09.793755+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.785884+0000","last_change":"2024-03-14T09:04:09.793536+0000","last_active":"2024-03-14T09:04:10.785884+0000","last_peered":"2024-03-14T09:04:10.785884+0000","last_clean":"2024-03-14T09:04:10.785884+0000","last_became_active":"2024-03-14T09:04:09.793283+0000","last_became_peered":"2024-03-14T09:04:09.793283+0000","last_unstale":"2024-03-14T09:04:10.785884+0000","last_undegraded":"2024-03-14T09:04:10.785884+0000","last_fullsized":"2024-03-14T09:04:10.785884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:33.968294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401320+0000","last_change":"2024-03-14T09:04:09.380665+0000","last_active":"2024-03-14T09:04:11.401320+0000","last_peered":"2024-03-14T09:04:11.401320+0000","last_clean":"2024-03-14T09:04:11.401320+0000","last_became_active":"2024-03-14T09:04:09.380369+0000","last_became_peered":"2024-03-14T09:04:09.380369+0000","last_unstale":"2024-03-14T09:04:11.401320+0000","last_undegraded":"2024-03-14T09:04:11.401320+0000","last_fullsized":"2024-03-14T09:04:11.401320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:33:47.940938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781811+0000","last_change":"2024-03-14T09:03:57.186670+0000","last_active":"2024-03-14T09:04:09.781811+0000","last_peered":"2024-03-14T09:04:09.781811+0000","last_clean":"2024-03-14T09:04:09.781811+0000","last_became_active":"2024-03-14T09:03:57.186398+0000","last_became_peered":"2024-03-14T09:03:57.186398+0000","last_unstale":"2024-03-14T09:04:09.781811+0000","last_undegraded":"2024-03-14T09:04:09.781811+0000","last_fullsized":"2024-03-14T09:04:09.781811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:15.878140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781053+0000","last_change":"2024-03-14T09:03:56.766595+0000","last_active":"2024-03-14T09:04:09.781053+0000","last_peered":"2024-03-14T09:04:09.781053+0000","last_clean":"2024-03-14T09:04:09.781053+0000","last_became_active":"2024-03-14T09:03:56.766269+0000","last_became_peered":"2024-03-14T09:03:56.766269+0000","last_unstale":"2024-03-14T09:04:09.781053+0000","last_undegraded":"2024-03-14T09:04:09.781053+0000","last_fullsized":"2024-03-14T09:04:09.781053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781780+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:09.781780+0000","last_peered":"2024-03-14T09:04:09.781780+0000","last_clean":"2024-03-14T09:04:09.781780+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:09.781780+0000","last_undegraded":"2024-03-14T09:04:09.781780+0000","last_fullsized":"2024-03-14T09:04:09.781780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":994685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.85699999999999998,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":6.5,"15min":6.5},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53100000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.432},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68100000000000005,"5min":0.878,"15min":0.878},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":405481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.627},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.87,"15min":0.87},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.437}]},{"osd":7,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":404329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33900000000000002,"15min":0.34000000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.45700000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39400000000000002,"15min":0.39500000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.497,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.377}]},{"osd":4,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.626}]},{"osd":6,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":995119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.71599999999999997,"15min":0.78200000000000003},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.41799999999999998,"5min":7.383,"15min":7.383},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.79200000000000004,"15min":0.86599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51800000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.81699999999999995,"15min":0.89200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.48899999999999999,"5min":7.944,"15min":7.944},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.77200000000000002,"15min":0.83599999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0349999999999999,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.312}]},{"osd":4,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.98399999999999999,"15min":1.0489999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0509999999999999,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80300000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.36499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":404751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34200000000000003,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.40400000000000003,"5min":1.111,"15min":1.111},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.375}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.67300000000000004,"15min":0.72999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.41199999999999998,"5min":7.5,"15min":7.5},"last":0.255},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.71599999999999997,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.48699999999999999,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":1.151,"15min":1.151},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":1.179,"15min":1.179},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71799999999999997,"5min":0.751,"15min":0.751},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.75900000000000001,"5min":1.165,"15min":1.165},"last":0.628},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64900000000000002,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83199999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":408796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.311,"15min":0.308},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.307},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.42799999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.373,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.45400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.378},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.45800000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.498},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75700000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76200000000000001,"5min":1.087,"15min":1.087},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":405823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51100000000000001,"15min":0.51600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.68999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.755}]},{"osd":2,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.751,"15min":0.751},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.89100000000000001,"15min":0.94699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76000000000000001,"5min":7.702,"15min":7.702},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.753,"15min":0.753},"last":0.624},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.84899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":407302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.434}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67900000000000005,"5min":0.876,"15min":0.876},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.624}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.499},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:13.900 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:14.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.051+0000 7f4943ebe700 1 Processor -- start 2024-03-14T09:04:14.052 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 -- start start 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f493c14c5c0 0x7f493c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c072290 0x7f493c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f493c0c1fd0 0x7f493c0c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c0c1bb0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c0c6170 con 0x7f493c14c5c0 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.052+0000 7f4943ebe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c070e10 con 0x7f493c072290 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f493c0c1fd0 0x7f493c0c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f493c0c1fd0 0x7f493c0c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43592/0 (socket says 172.21.15.154:43592) 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 -- 172.21.15.154:0/762233258 learned_addr learned my addr 172.21.15.154:0/762233258 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f4941459700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c072290 0x7f493c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f4941c5a700 1 --2- 172.21.15.154:0/762233258 >> v2:172.21.15.191:3300/0 conn(0x7f493c14c5c0 0x7f493c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 -- 172.21.15.154:0/762233258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c072290 msgr2=0x7f493c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c072290 0x7f493c072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 -- 172.21.15.154:0/762233258 >> v2:172.21.15.191:3300/0 conn(0x7f493c14c5c0 msgr2=0x7f493c152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 --2- 172.21.15.154:0/762233258 >> v2:172.21.15.191:3300/0 conn(0x7f493c14c5c0 0x7f493c152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 -- 172.21.15.154:0/762233258 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c14c9a0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f494245b700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c0c2460 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a9340 tx=0x7f493400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d98593863b30a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f4940c58700 1 -- 172.21.15.154:0/762233258 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400e070 con 0x7f493c0c1fd0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f4940c58700 1 -- 172.21.15.154:0/762233258 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493400b9c0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.053+0000 7f4940c58700 1 -- 172.21.15.154:0/762233258 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400bd70 con 0x7f493c0c1fd0 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 -- 172.21.15.154:0/762233258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 msgr2=0x7f493c0c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c0c2460 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a9340 tx=0x7f493400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 -- 172.21.15.154:0/762233258 shutdown_connections 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c0c2460 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 --2- 172.21.15.154:0/762233258 >> v2:172.21.15.191:3300/0 conn(0x7f493c14c5c0 0x7f493c152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 --2- 172.21.15.154:0/762233258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c072290 0x7f493c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 -- 172.21.15.154:0/762233258 >> 172.21.15.154:0/762233258 conn(0x7f493c06a770 msgr2=0x7f493c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:14.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 -- 172.21.15.154:0/762233258 shutdown_connections 2024-03-14T09:04:14.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.054+0000 7f4943ebe700 1 -- 172.21.15.154:0/762233258 wait complete. 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.055+0000 7f4943ebe700 1 Processor -- start 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.055+0000 7f4943ebe700 1 -- start start 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f493c072290 0x7f493c0aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 0x7f493c14b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1dc090 con 0x7f493c0c1fd0 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1df610 con 0x7f493c072290 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c57b0 con 0x7f493c14c5c0 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43600/0 (socket says 172.21.15.154:43600) 2024-03-14T09:04:14.059 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 -- 172.21.15.154:0/1451678988 learned_addr learned my addr 172.21.15.154:0/1451678988 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941c5a700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f493c072290 0x7f493c0aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f494245b700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 0x7f493c14b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 -- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 msgr2=0x7f493c14b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 0x7f493c14b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 -- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f493c072290 msgr2=0x7f493c0aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f493c072290 0x7f493c0aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 -- 172.21.15.154:0/1451678988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934009b30 con 0x7f493c0c1fd0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f494245b700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 0x7f493c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4941459700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f493800c7f0 tx=0x7f493800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938004220 con 0x7f493c0c1fd0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4938004380 con 0x7f493c0c1fd0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c14dae0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380036a0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.060 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.056+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c14d800 con 0x7f493c0c1fd0 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.057+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c0697d0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.058+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4938003800 con 0x7f493c0c1fd0 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.058+0000 7f492e7fc700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 0x7f492006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.058+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f4938088be0 con 0x7f493c0c1fd0 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.058+0000 7f4941c5a700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 0x7f492006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.061 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.059+0000 7f4941c5a700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 0x7f492006cd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4930006fd0 tx=0x7f4930008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.062 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.063+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4938053c90 con 0x7f493c0c1fd0 2024-03-14T09:04:14.252 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.252+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f493c072d80 con 0x7f492006a860 2024-03-14T09:04:14.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.267+0000 7f492e7fc700 1 -- 172.21.15.154:0/1451678988 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528102 (secure 0 0 0) 0x7f493000c040 con 0x7f492006a860 2024-03-14T09:04:14.273 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:14.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 msgr2=0x7f492006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 0x7f492006cd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4930006fd0 tx=0x7f4930008040 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 msgr2=0x7f493c14b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f493800c7f0 tx=0x7f493800cb00 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 shutdown_connections 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f493c14c5c0 0x7f493c14b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f493c072290 0x7f493c0aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f492006a860 0x7f492006cd20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 --2- 172.21.15.154:0/1451678988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c0c1fd0 0x7f493c14b340 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 >> 172.21.15.154:0/1451678988 conn(0x7f493c06a770 msgr2=0x7f493c155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 shutdown_connections 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.271+0000 7f4943ebe700 1 -- 172.21.15.154:0/1451678988 wait complete. 2024-03-14T09:04:14.278 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:14.283 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-14T09:04:13.558917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596603,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":336,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224552,"kb_used_data":8392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750168,"statfs":{"total":773094113280,"available":772864172032,"internally_reserved":0,"allocated":8593408,"data_stored":4426286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.199295"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400182+0000","last_change":"2024-03-14T09:04:09.795463+0000","last_active":"2024-03-14T09:04:11.400182+0000","last_peered":"2024-03-14T09:04:11.400182+0000","last_clean":"2024-03-14T09:04:11.400182+0000","last_became_active":"2024-03-14T09:04:09.795259+0000","last_became_peered":"2024-03-14T09:04:09.795259+0000","last_unstale":"2024-03-14T09:04:11.400182+0000","last_undegraded":"2024-03-14T09:04:11.400182+0000","last_fullsized":"2024-03-14T09:04:11.400182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:37.967825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779210+0000","last_change":"2024-03-14T09:03:57.183867+0000","last_active":"2024-03-14T09:04:09.779210+0000","last_peered":"2024-03-14T09:04:09.779210+0000","last_clean":"2024-03-14T09:04:09.779210+0000","last_became_active":"2024-03-14T09:03:57.183371+0000","last_became_peered":"2024-03-14T09:03:57.183371+0000","last_unstale":"2024-03-14T09:04:09.779210+0000","last_undegraded":"2024-03-14T09:04:09.779210+0000","last_fullsized":"2024-03-14T09:04:09.779210+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783772+0000","last_change":"2024-03-14T09:04:09.791872+0000","last_active":"2024-03-14T09:04:10.783772+0000","last_peered":"2024-03-14T09:04:10.783772+0000","last_clean":"2024-03-14T09:04:10.783772+0000","last_became_active":"2024-03-14T09:04:09.791341+0000","last_became_peered":"2024-03-14T09:04:09.791341+0000","last_unstale":"2024-03-14T09:04:10.783772+0000","last_undegraded":"2024-03-14T09:04:10.783772+0000","last_fullsized":"2024-03-14T09:04:10.783772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:07.293563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781438+0000","last_change":"2024-03-14T09:03:57.187973+0000","last_active":"2024-03-14T09:04:09.781438+0000","last_peered":"2024-03-14T09:04:09.781438+0000","last_clean":"2024-03-14T09:04:09.781438+0000","last_became_active":"2024-03-14T09:03:57.187737+0000","last_became_peered":"2024-03-14T09:03:57.187737+0000","last_unstale":"2024-03-14T09:04:09.781438+0000","last_undegraded":"2024-03-14T09:04:09.781438+0000","last_fullsized":"2024-03-14T09:04:09.781438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:39.248247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400288+0000","last_change":"2024-03-14T09:04:09.789743+0000","last_active":"2024-03-14T09:04:11.400288+0000","last_peered":"2024-03-14T09:04:11.400288+0000","last_clean":"2024-03-14T09:04:11.400288+0000","last_became_active":"2024-03-14T09:04:09.789396+0000","last_became_peered":"2024-03-14T09:04:09.789396+0000","last_unstale":"2024-03-14T09:04:11.400288+0000","last_undegraded":"2024-03-14T09:04:11.400288+0000","last_fullsized":"2024-03-14T09:04:11.400288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:31.883445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780964+0000","last_change":"2024-03-14T09:03:57.180121+0000","last_active":"2024-03-14T09:04:09.780964+0000","last_peered":"2024-03-14T09:04:09.780964+0000","last_clean":"2024-03-14T09:04:09.780964+0000","last_became_active":"2024-03-14T09:03:57.179465+0000","last_became_peered":"2024-03-14T09:03:57.179465+0000","last_unstale":"2024-03-14T09:04:09.780964+0000","last_undegraded":"2024-03-14T09:04:09.780964+0000","last_fullsized":"2024-03-14T09:04:09.780964+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401365+0000","last_change":"2024-03-14T09:04:09.788938+0000","last_active":"2024-03-14T09:04:11.401365+0000","last_peered":"2024-03-14T09:04:11.401365+0000","last_clean":"2024-03-14T09:04:11.401365+0000","last_became_active":"2024-03-14T09:04:09.788510+0000","last_became_peered":"2024-03-14T09:04:09.788510+0000","last_unstale":"2024-03-14T09:04:11.401365+0000","last_undegraded":"2024-03-14T09:04:11.401365+0000","last_fullsized":"2024-03-14T09:04:11.401365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:03:21.360800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783372+0000","last_change":"2024-03-14T09:03:57.191808+0000","last_active":"2024-03-14T09:04:09.783372+0000","last_peered":"2024-03-14T09:04:09.783372+0000","last_clean":"2024-03-14T09:04:09.783372+0000","last_became_active":"2024-03-14T09:03:57.187786+0000","last_became_peered":"2024-03-14T09:03:57.187786+0000","last_unstale":"2024-03-14T09:04:09.783372+0000","last_undegraded":"2024-03-14T09:04:09.783372+0000","last_fullsized":"2024-03-14T09:04:09.783372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:23.462424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400340+0000","last_change":"2024-03-14T09:04:09.789579+0000","last_active":"2024-03-14T09:04:11.400340+0000","last_peered":"2024-03-14T09:04:11.400340+0000","last_clean":"2024-03-14T09:04:11.400340+0000","last_became_active":"2024-03-14T09:04:09.789295+0000","last_became_peered":"2024-03-14T09:04:09.789295+0000","last_unstale":"2024-03-14T09:04:11.400340+0000","last_undegraded":"2024-03-14T09:04:11.400340+0000","last_fullsized":"2024-03-14T09:04:11.400340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:28:07.483738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793680+0000","last_change":"2024-03-14T09:03:57.186814+0000","last_active":"2024-03-14T09:04:09.793680+0000","last_peered":"2024-03-14T09:04:09.793680+0000","last_clean":"2024-03-14T09:04:09.793680+0000","last_became_active":"2024-03-14T09:03:57.186480+0000","last_became_peered":"2024-03-14T09:03:57.186480+0000","last_unstale":"2024-03-14T09:04:09.793680+0000","last_undegraded":"2024-03-14T09:04:09.793680+0000","last_fullsized":"2024-03-14T09:04:09.793680+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400422+0000","last_change":"2024-03-14T09:04:09.788892+0000","last_active":"2024-03-14T09:04:11.400422+0000","last_peered":"2024-03-14T09:04:11.400422+0000","last_clean":"2024-03-14T09:04:11.400422+0000","last_became_active":"2024-03-14T09:04:09.788657+0000","last_became_peered":"2024-03-14T09:04:09.788657+0000","last_unstale":"2024-03-14T09:04:11.400422+0000","last_undegraded":"2024-03-14T09:04:11.400422+0000","last_fullsized":"2024-03-14T09:04:11.400422+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:47:13.189988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784282+0000","last_change":"2024-03-14T09:03:57.179120+0000","last_active":"2024-03-14T09:04:09.784282+0000","last_peered":"2024-03-14T09:04:09.784282+0000","last_clean":"2024-03-14T09:04:09.784282+0000","last_became_active":"2024-03-14T09:03:57.178824+0000","last_became_peered":"2024-03-14T09:03:57.178824+0000","last_unstale":"2024-03-14T09:04:09.784282+0000","last_undegraded":"2024-03-14T09:04:09.784282+0000","last_fullsized":"2024-03-14T09:04:09.784282+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400767+0000","last_change":"2024-03-14T09:04:09.786125+0000","last_active":"2024-03-14T09:04:11.400767+0000","last_peered":"2024-03-14T09:04:11.400767+0000","last_clean":"2024-03-14T09:04:11.400767+0000","last_became_active":"2024-03-14T09:04:09.785805+0000","last_became_peered":"2024-03-14T09:04:09.785805+0000","last_unstale":"2024-03-14T09:04:11.400767+0000","last_undegraded":"2024-03-14T09:04:11.400767+0000","last_fullsized":"2024-03-14T09:04:11.400767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:11:07.367499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780045+0000","last_change":"2024-03-14T09:03:57.176658+0000","last_active":"2024-03-14T09:04:09.780045+0000","last_peered":"2024-03-14T09:04:09.780045+0000","last_clean":"2024-03-14T09:04:09.780045+0000","last_became_active":"2024-03-14T09:03:57.176352+0000","last_became_peered":"2024-03-14T09:03:57.176352+0000","last_unstale":"2024-03-14T09:04:09.780045+0000","last_undegraded":"2024-03-14T09:04:09.780045+0000","last_fullsized":"2024-03-14T09:04:09.780045+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:51:09.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401267+0000","last_change":"2024-03-14T09:04:09.790488+0000","last_active":"2024-03-14T09:04:11.401267+0000","last_peered":"2024-03-14T09:04:11.401267+0000","last_clean":"2024-03-14T09:04:11.401267+0000","last_became_active":"2024-03-14T09:04:09.790221+0000","last_became_peered":"2024-03-14T09:04:09.790221+0000","last_unstale":"2024-03-14T09:04:11.401267+0000","last_undegraded":"2024-03-14T09:04:11.401267+0000","last_fullsized":"2024-03-14T09:04:11.401267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:16:19.679079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783314+0000","last_change":"2024-03-14T09:03:57.186734+0000","last_active":"2024-03-14T09:04:09.783314+0000","last_peered":"2024-03-14T09:04:09.783314+0000","last_clean":"2024-03-14T09:04:09.783314+0000","last_became_active":"2024-03-14T09:03:57.186455+0000","last_became_peered":"2024-03-14T09:03:57.186455+0000","last_unstale":"2024-03-14T09:04:09.783314+0000","last_undegraded":"2024-03-14T09:04:09.783314+0000","last_fullsized":"2024-03-14T09:04:09.783314+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400975+0000","last_change":"2024-03-14T09:04:09.801347+0000","last_active":"2024-03-14T09:04:11.400975+0000","last_peered":"2024-03-14T09:04:11.400975+0000","last_clean":"2024-03-14T09:04:11.400975+0000","last_became_active":"2024-03-14T09:04:09.801060+0000","last_became_peered":"2024-03-14T09:04:09.801060+0000","last_unstale":"2024-03-14T09:04:11.400975+0000","last_undegraded":"2024-03-14T09:04:11.400975+0000","last_fullsized":"2024-03-14T09:04:11.400975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:29:40.282609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784810+0000","last_change":"2024-03-14T09:03:57.186757+0000","last_active":"2024-03-14T09:04:09.784810+0000","last_peered":"2024-03-14T09:04:09.784810+0000","last_clean":"2024-03-14T09:04:09.784810+0000","last_became_active":"2024-03-14T09:03:57.186486+0000","last_became_peered":"2024-03-14T09:03:57.186486+0000","last_unstale":"2024-03-14T09:04:09.784810+0000","last_undegraded":"2024-03-14T09:04:09.784810+0000","last_fullsized":"2024-03-14T09:04:09.784810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782884+0000","last_change":"2024-03-14T09:04:09.799551+0000","last_active":"2024-03-14T09:04:10.782884+0000","last_peered":"2024-03-14T09:04:10.782884+0000","last_clean":"2024-03-14T09:04:10.782884+0000","last_became_active":"2024-03-14T09:04:09.799287+0000","last_became_peered":"2024-03-14T09:04:09.799287+0000","last_unstale":"2024-03-14T09:04:10.782884+0000","last_undegraded":"2024-03-14T09:04:10.782884+0000","last_fullsized":"2024-03-14T09:04:10.782884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778187+0000","last_change":"2024-03-14T09:03:57.180488+0000","last_active":"2024-03-14T09:04:09.778187+0000","last_peered":"2024-03-14T09:04:09.778187+0000","last_clean":"2024-03-14T09:04:09.778187+0000","last_became_active":"2024-03-14T09:03:57.180202+0000","last_became_peered":"2024-03-14T09:03:57.180202+0000","last_unstale":"2024-03-14T09:04:09.778187+0000","last_undegraded":"2024-03-14T09:04:09.778187+0000","last_fullsized":"2024-03-14T09:04:09.778187+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783557+0000","last_change":"2024-03-14T09:04:09.800401+0000","last_active":"2024-03-14T09:04:10.783557+0000","last_peered":"2024-03-14T09:04:10.783557+0000","last_clean":"2024-03-14T09:04:10.783557+0000","last_became_active":"2024-03-14T09:04:09.800120+0000","last_became_peered":"2024-03-14T09:04:09.800120+0000","last_unstale":"2024-03-14T09:04:10.783557+0000","last_undegraded":"2024-03-14T09:04:10.783557+0000","last_fullsized":"2024-03-14T09:04:10.783557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:00.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783772+0000","last_change":"2024-03-14T09:04:09.374689+0000","last_active":"2024-03-14T09:04:10.783772+0000","last_peered":"2024-03-14T09:04:10.783772+0000","last_clean":"2024-03-14T09:04:10.783772+0000","last_became_active":"2024-03-14T09:04:09.374377+0000","last_became_peered":"2024-03-14T09:04:09.374377+0000","last_unstale":"2024-03-14T09:04:10.783772+0000","last_undegraded":"2024-03-14T09:04:10.783772+0000","last_fullsized":"2024-03-14T09:04:10.783772+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:37.206868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400509+0000","last_change":"2024-03-14T09:04:09.793291+0000","last_active":"2024-03-14T09:04:11.400509+0000","last_peered":"2024-03-14T09:04:11.400509+0000","last_clean":"2024-03-14T09:04:11.400509+0000","last_became_active":"2024-03-14T09:04:09.793025+0000","last_became_peered":"2024-03-14T09:04:09.793025+0000","last_unstale":"2024-03-14T09:04:11.400509+0000","last_undegraded":"2024-03-14T09:04:11.400509+0000","last_fullsized":"2024-03-14T09:04:11.400509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:45:57.150041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776759+0000","last_change":"2024-03-14T09:03:57.182863+0000","last_active":"2024-03-14T09:04:09.776759+0000","last_peered":"2024-03-14T09:04:09.776759+0000","last_clean":"2024-03-14T09:04:09.776759+0000","last_became_active":"2024-03-14T09:03:57.181034+0000","last_became_peered":"2024-03-14T09:03:57.181034+0000","last_unstale":"2024-03-14T09:04:09.776759+0000","last_undegraded":"2024-03-14T09:04:09.776759+0000","last_fullsized":"2024-03-14T09:04:09.776759+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783014+0000","last_change":"2024-03-14T09:04:09.792625+0000","last_active":"2024-03-14T09:04:10.783014+0000","last_peered":"2024-03-14T09:04:10.783014+0000","last_clean":"2024-03-14T09:04:10.783014+0000","last_became_active":"2024-03-14T09:04:09.792370+0000","last_became_peered":"2024-03-14T09:04:09.792370+0000","last_unstale":"2024-03-14T09:04:10.783014+0000","last_undegraded":"2024-03-14T09:04:10.783014+0000","last_fullsized":"2024-03-14T09:04:10.783014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:20:45.664696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779881+0000","last_change":"2024-03-14T09:03:57.177255+0000","last_active":"2024-03-14T09:04:09.779881+0000","last_peered":"2024-03-14T09:04:09.779881+0000","last_clean":"2024-03-14T09:04:09.779881+0000","last_became_active":"2024-03-14T09:03:57.176972+0000","last_became_peered":"2024-03-14T09:03:57.176972+0000","last_unstale":"2024-03-14T09:04:09.779881+0000","last_undegraded":"2024-03-14T09:04:09.779881+0000","last_fullsized":"2024-03-14T09:04:09.779881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:41.881683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400979+0000","last_change":"2024-03-14T09:04:09.799979+0000","last_active":"2024-03-14T09:04:11.400979+0000","last_peered":"2024-03-14T09:04:11.400979+0000","last_clean":"2024-03-14T09:04:11.400979+0000","last_became_active":"2024-03-14T09:04:09.799709+0000","last_became_peered":"2024-03-14T09:04:09.799709+0000","last_unstale":"2024-03-14T09:04:11.400979+0000","last_undegraded":"2024-03-14T09:04:11.400979+0000","last_fullsized":"2024-03-14T09:04:11.400979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785807+0000","last_change":"2024-03-14T09:03:57.191926+0000","last_active":"2024-03-14T09:04:09.785807+0000","last_peered":"2024-03-14T09:04:09.785807+0000","last_clean":"2024-03-14T09:04:09.785807+0000","last_became_active":"2024-03-14T09:03:57.191416+0000","last_became_peered":"2024-03-14T09:03:57.191416+0000","last_unstale":"2024-03-14T09:04:09.785807+0000","last_undegraded":"2024-03-14T09:04:09.785807+0000","last_fullsized":"2024-03-14T09:04:09.785807+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:25.082584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783412+0000","last_change":"2024-03-14T09:04:09.801660+0000","last_active":"2024-03-14T09:04:10.783412+0000","last_peered":"2024-03-14T09:04:10.783412+0000","last_clean":"2024-03-14T09:04:10.783412+0000","last_became_active":"2024-03-14T09:04:09.801416+0000","last_became_peered":"2024-03-14T09:04:09.801416+0000","last_unstale":"2024-03-14T09:04:10.783412+0000","last_undegraded":"2024-03-14T09:04:10.783412+0000","last_fullsized":"2024-03-14T09:04:10.783412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:29:18.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782475+0000","last_change":"2024-03-14T09:03:57.175406+0000","last_active":"2024-03-14T09:04:09.782475+0000","last_peered":"2024-03-14T09:04:09.782475+0000","last_clean":"2024-03-14T09:04:09.782475+0000","last_became_active":"2024-03-14T09:03:57.175155+0000","last_became_peered":"2024-03-14T09:03:57.175155+0000","last_unstale":"2024-03-14T09:04:09.782475+0000","last_undegraded":"2024-03-14T09:04:09.782475+0000","last_fullsized":"2024-03-14T09:04:09.782475+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:34:19.270920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784302+0000","last_change":"2024-03-14T09:04:09.793701+0000","last_active":"2024-03-14T09:04:10.784302+0000","last_peered":"2024-03-14T09:04:10.784302+0000","last_clean":"2024-03-14T09:04:10.784302+0000","last_became_active":"2024-03-14T09:04:09.792889+0000","last_became_peered":"2024-03-14T09:04:09.792889+0000","last_unstale":"2024-03-14T09:04:10.784302+0000","last_undegraded":"2024-03-14T09:04:10.784302+0000","last_fullsized":"2024-03-14T09:04:10.784302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:17.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793669+0000","last_change":"2024-03-14T09:03:57.181942+0000","last_active":"2024-03-14T09:04:09.793669+0000","last_peered":"2024-03-14T09:04:09.793669+0000","last_clean":"2024-03-14T09:04:09.793669+0000","last_became_active":"2024-03-14T09:03:57.181669+0000","last_became_peered":"2024-03-14T09:03:57.181669+0000","last_unstale":"2024-03-14T09:04:09.793669+0000","last_undegraded":"2024-03-14T09:04:09.793669+0000","last_fullsized":"2024-03-14T09:04:09.793669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:23.881563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784332+0000","last_change":"2024-03-14T09:04:09.797382+0000","last_active":"2024-03-14T09:04:10.784332+0000","last_peered":"2024-03-14T09:04:10.784332+0000","last_clean":"2024-03-14T09:04:10.784332+0000","last_became_active":"2024-03-14T09:04:09.797137+0000","last_became_peered":"2024-03-14T09:04:09.797137+0000","last_unstale":"2024-03-14T09:04:10.784332+0000","last_undegraded":"2024-03-14T09:04:10.784332+0000","last_fullsized":"2024-03-14T09:04:10.784332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:38:59.136969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796582+0000","last_change":"2024-03-14T09:03:57.177217+0000","last_active":"2024-03-14T09:04:09.796582+0000","last_peered":"2024-03-14T09:04:09.796582+0000","last_clean":"2024-03-14T09:04:09.796582+0000","last_became_active":"2024-03-14T09:03:57.176941+0000","last_became_peered":"2024-03-14T09:03:57.176941+0000","last_unstale":"2024-03-14T09:04:09.796582+0000","last_undegraded":"2024-03-14T09:04:09.796582+0000","last_fullsized":"2024-03-14T09:04:09.796582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:31.732618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783012+0000","last_change":"2024-03-14T09:04:09.796023+0000","last_active":"2024-03-14T09:04:10.783012+0000","last_peered":"2024-03-14T09:04:10.783012+0000","last_clean":"2024-03-14T09:04:10.783012+0000","last_became_active":"2024-03-14T09:04:09.794309+0000","last_became_peered":"2024-03-14T09:04:09.794309+0000","last_unstale":"2024-03-14T09:04:10.783012+0000","last_undegraded":"2024-03-14T09:04:10.783012+0000","last_fullsized":"2024-03-14T09:04:10.783012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:49.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784613+0000","last_change":"2024-03-14T09:03:57.176770+0000","last_active":"2024-03-14T09:04:09.784613+0000","last_peered":"2024-03-14T09:04:09.784613+0000","last_clean":"2024-03-14T09:04:09.784613+0000","last_became_active":"2024-03-14T09:03:57.176496+0000","last_became_peered":"2024-03-14T09:03:57.176496+0000","last_unstale":"2024-03-14T09:04:09.784613+0000","last_undegraded":"2024-03-14T09:04:09.784613+0000","last_fullsized":"2024-03-14T09:04:09.784613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:32.959955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401480+0000","last_change":"2024-03-14T09:04:09.791778+0000","last_active":"2024-03-14T09:04:11.401480+0000","last_peered":"2024-03-14T09:04:11.401480+0000","last_clean":"2024-03-14T09:04:11.401480+0000","last_became_active":"2024-03-14T09:04:09.791462+0000","last_became_peered":"2024-03-14T09:04:09.791462+0000","last_unstale":"2024-03-14T09:04:11.401480+0000","last_undegraded":"2024-03-14T09:04:11.401480+0000","last_fullsized":"2024-03-14T09:04:11.401480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:34:27.112960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773773+0000","last_change":"2024-03-14T09:03:57.188745+0000","last_active":"2024-03-14T09:04:09.773773+0000","last_peered":"2024-03-14T09:04:09.773773+0000","last_clean":"2024-03-14T09:04:09.773773+0000","last_became_active":"2024-03-14T09:03:57.187830+0000","last_became_peered":"2024-03-14T09:03:57.187830+0000","last_unstale":"2024-03-14T09:04:09.773773+0000","last_undegraded":"2024-03-14T09:04:09.773773+0000","last_fullsized":"2024-03-14T09:04:09.773773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783976+0000","last_change":"2024-03-14T09:04:09.790848+0000","last_active":"2024-03-14T09:04:10.783976+0000","last_peered":"2024-03-14T09:04:10.783976+0000","last_clean":"2024-03-14T09:04:10.783976+0000","last_became_active":"2024-03-14T09:04:09.790636+0000","last_became_peered":"2024-03-14T09:04:09.790636+0000","last_unstale":"2024-03-14T09:04:10.783976+0000","last_undegraded":"2024-03-14T09:04:10.783976+0000","last_fullsized":"2024-03-14T09:04:10.783976+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:53:48.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783521+0000","last_change":"2024-03-14T09:03:57.186877+0000","last_active":"2024-03-14T09:04:09.783521+0000","last_peered":"2024-03-14T09:04:09.783521+0000","last_clean":"2024-03-14T09:04:09.783521+0000","last_became_active":"2024-03-14T09:03:57.186581+0000","last_became_peered":"2024-03-14T09:03:57.186581+0000","last_unstale":"2024-03-14T09:04:09.783521+0000","last_undegraded":"2024-03-14T09:04:09.783521+0000","last_fullsized":"2024-03-14T09:04:09.783521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:51.324739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400204+0000","last_change":"2024-03-14T09:04:09.789552+0000","last_active":"2024-03-14T09:04:11.400204+0000","last_peered":"2024-03-14T09:04:11.400204+0000","last_clean":"2024-03-14T09:04:11.400204+0000","last_became_active":"2024-03-14T09:04:09.789296+0000","last_became_peered":"2024-03-14T09:04:09.789296+0000","last_unstale":"2024-03-14T09:04:11.400204+0000","last_undegraded":"2024-03-14T09:04:11.400204+0000","last_fullsized":"2024-03-14T09:04:11.400204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:20:45.664696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784704+0000","last_change":"2024-03-14T09:03:57.187026+0000","last_active":"2024-03-14T09:04:09.784704+0000","last_peered":"2024-03-14T09:04:09.784704+0000","last_clean":"2024-03-14T09:04:09.784704+0000","last_became_active":"2024-03-14T09:03:57.186774+0000","last_became_peered":"2024-03-14T09:03:57.186774+0000","last_unstale":"2024-03-14T09:04:09.784704+0000","last_undegraded":"2024-03-14T09:04:09.784704+0000","last_fullsized":"2024-03-14T09:04:09.784704+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:55:25.363869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400786+0000","last_change":"2024-03-14T09:04:09.792469+0000","last_active":"2024-03-14T09:04:11.400786+0000","last_peered":"2024-03-14T09:04:11.400786+0000","last_clean":"2024-03-14T09:04:11.400786+0000","last_became_active":"2024-03-14T09:04:09.791828+0000","last_became_peered":"2024-03-14T09:04:09.791828+0000","last_unstale":"2024-03-14T09:04:11.400786+0000","last_undegraded":"2024-03-14T09:04:11.400786+0000","last_fullsized":"2024-03-14T09:04:11.400786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777838+0000","last_change":"2024-03-14T09:03:57.189350+0000","last_active":"2024-03-14T09:04:09.777838+0000","last_peered":"2024-03-14T09:04:09.777838+0000","last_clean":"2024-03-14T09:04:09.777838+0000","last_became_active":"2024-03-14T09:03:57.189109+0000","last_became_peered":"2024-03-14T09:03:57.189109+0000","last_unstale":"2024-03-14T09:04:09.777838+0000","last_undegraded":"2024-03-14T09:04:09.777838+0000","last_fullsized":"2024-03-14T09:04:09.777838+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:18.878301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784555+0000","last_change":"2024-03-14T09:04:09.796033+0000","last_active":"2024-03-14T09:04:10.784555+0000","last_peered":"2024-03-14T09:04:10.784555+0000","last_clean":"2024-03-14T09:04:10.784555+0000","last_became_active":"2024-03-14T09:04:09.795784+0000","last_became_peered":"2024-03-14T09:04:09.795784+0000","last_unstale":"2024-03-14T09:04:10.784555+0000","last_undegraded":"2024-03-14T09:04:10.784555+0000","last_fullsized":"2024-03-14T09:04:10.784555+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:41:54.494207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777203+0000","last_change":"2024-03-14T09:03:57.186278+0000","last_active":"2024-03-14T09:04:09.777203+0000","last_peered":"2024-03-14T09:04:09.777203+0000","last_clean":"2024-03-14T09:04:09.777203+0000","last_became_active":"2024-03-14T09:03:57.186014+0000","last_became_peered":"2024-03-14T09:03:57.186014+0000","last_unstale":"2024-03-14T09:04:09.777203+0000","last_undegraded":"2024-03-14T09:04:09.777203+0000","last_fullsized":"2024-03-14T09:04:09.777203+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:42.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400096+0000","last_change":"2024-03-14T09:04:09.790562+0000","last_active":"2024-03-14T09:04:11.400096+0000","last_peered":"2024-03-14T09:04:11.400096+0000","last_clean":"2024-03-14T09:04:11.400096+0000","last_became_active":"2024-03-14T09:04:09.790288+0000","last_became_peered":"2024-03-14T09:04:09.790288+0000","last_unstale":"2024-03-14T09:04:11.400096+0000","last_undegraded":"2024-03-14T09:04:11.400096+0000","last_fullsized":"2024-03-14T09:04:11.400096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:33:45.300921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780089+0000","last_change":"2024-03-14T09:03:57.186749+0000","last_active":"2024-03-14T09:04:09.780089+0000","last_peered":"2024-03-14T09:04:09.780089+0000","last_clean":"2024-03-14T09:04:09.780089+0000","last_became_active":"2024-03-14T09:03:57.186502+0000","last_became_peered":"2024-03-14T09:03:57.186502+0000","last_unstale":"2024-03-14T09:04:09.780089+0000","last_undegraded":"2024-03-14T09:04:09.780089+0000","last_fullsized":"2024-03-14T09:04:09.780089+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:10.996899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401082+0000","last_change":"2024-03-14T09:04:09.797676+0000","last_active":"2024-03-14T09:04:11.401082+0000","last_peered":"2024-03-14T09:04:11.401082+0000","last_clean":"2024-03-14T09:04:11.401082+0000","last_became_active":"2024-03-14T09:04:09.797396+0000","last_became_peered":"2024-03-14T09:04:09.797396+0000","last_unstale":"2024-03-14T09:04:11.401082+0000","last_undegraded":"2024-03-14T09:04:11.401082+0000","last_fullsized":"2024-03-14T09:04:11.401082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:07.293563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782345+0000","last_change":"2024-03-14T09:03:57.185209+0000","last_active":"2024-03-14T09:04:09.782345+0000","last_peered":"2024-03-14T09:04:09.782345+0000","last_clean":"2024-03-14T09:04:09.782345+0000","last_became_active":"2024-03-14T09:03:57.184942+0000","last_became_peered":"2024-03-14T09:03:57.184942+0000","last_unstale":"2024-03-14T09:04:09.782345+0000","last_undegraded":"2024-03-14T09:04:09.782345+0000","last_fullsized":"2024-03-14T09:04:09.782345+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:37.052225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401410+0000","last_change":"2024-03-14T09:04:09.785193+0000","last_active":"2024-03-14T09:04:11.401410+0000","last_peered":"2024-03-14T09:04:11.401410+0000","last_clean":"2024-03-14T09:04:11.401410+0000","last_became_active":"2024-03-14T09:04:09.784910+0000","last_became_peered":"2024-03-14T09:04:09.784910+0000","last_unstale":"2024-03-14T09:04:11.401410+0000","last_undegraded":"2024-03-14T09:04:11.401410+0000","last_fullsized":"2024-03-14T09:04:11.401410+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780425+0000","last_change":"2024-03-14T09:03:57.189881+0000","last_active":"2024-03-14T09:04:09.780425+0000","last_peered":"2024-03-14T09:04:09.780425+0000","last_clean":"2024-03-14T09:04:09.780425+0000","last_became_active":"2024-03-14T09:03:57.189632+0000","last_became_peered":"2024-03-14T09:03:57.189632+0000","last_unstale":"2024-03-14T09:04:09.780425+0000","last_undegraded":"2024-03-14T09:04:09.780425+0000","last_fullsized":"2024-03-14T09:04:09.780425+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402505+0000","last_change":"2024-03-14T09:04:09.795512+0000","last_active":"2024-03-14T09:04:11.402505+0000","last_peered":"2024-03-14T09:04:11.402505+0000","last_clean":"2024-03-14T09:04:11.402505+0000","last_became_active":"2024-03-14T09:04:09.795282+0000","last_became_peered":"2024-03-14T09:04:09.795282+0000","last_unstale":"2024-03-14T09:04:11.402505+0000","last_undegraded":"2024-03-14T09:04:11.402505+0000","last_fullsized":"2024-03-14T09:04:11.402505+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:43.806093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796548+0000","last_change":"2024-03-14T09:03:57.168978+0000","last_active":"2024-03-14T09:04:09.796548+0000","last_peered":"2024-03-14T09:04:09.796548+0000","last_clean":"2024-03-14T09:04:09.796548+0000","last_became_active":"2024-03-14T09:03:57.167802+0000","last_became_peered":"2024-03-14T09:03:57.167802+0000","last_unstale":"2024-03-14T09:04:09.796548+0000","last_undegraded":"2024-03-14T09:04:09.796548+0000","last_fullsized":"2024-03-14T09:04:09.796548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401609+0000","last_change":"2024-03-14T09:04:09.793855+0000","last_active":"2024-03-14T09:04:11.401609+0000","last_peered":"2024-03-14T09:04:11.401609+0000","last_clean":"2024-03-14T09:04:11.401609+0000","last_became_active":"2024-03-14T09:04:09.793402+0000","last_became_peered":"2024-03-14T09:04:09.793402+0000","last_unstale":"2024-03-14T09:04:11.401609+0000","last_undegraded":"2024-03-14T09:04:11.401609+0000","last_fullsized":"2024-03-14T09:04:11.401609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:53:48.239622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777708+0000","last_change":"2024-03-14T09:03:57.183439+0000","last_active":"2024-03-14T09:04:09.777708+0000","last_peered":"2024-03-14T09:04:09.777708+0000","last_clean":"2024-03-14T09:04:09.777708+0000","last_became_active":"2024-03-14T09:03:57.183150+0000","last_became_peered":"2024-03-14T09:03:57.183150+0000","last_unstale":"2024-03-14T09:04:09.777708+0000","last_undegraded":"2024-03-14T09:04:09.777708+0000","last_fullsized":"2024-03-14T09:04:09.777708+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783103+0000","last_change":"2024-03-14T09:04:09.784256+0000","last_active":"2024-03-14T09:04:10.783103+0000","last_peered":"2024-03-14T09:04:10.783103+0000","last_clean":"2024-03-14T09:04:10.783103+0000","last_became_active":"2024-03-14T09:04:09.784014+0000","last_became_peered":"2024-03-14T09:04:09.784014+0000","last_unstale":"2024-03-14T09:04:10.783103+0000","last_undegraded":"2024-03-14T09:04:10.783103+0000","last_fullsized":"2024-03-14T09:04:10.783103+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:34:27.112960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783362+0000","last_change":"2024-03-14T09:03:57.184782+0000","last_active":"2024-03-14T09:04:09.783362+0000","last_peered":"2024-03-14T09:04:09.783362+0000","last_clean":"2024-03-14T09:04:09.783362+0000","last_became_active":"2024-03-14T09:03:57.184507+0000","last_became_peered":"2024-03-14T09:03:57.184507+0000","last_unstale":"2024-03-14T09:04:09.783362+0000","last_undegraded":"2024-03-14T09:04:09.783362+0000","last_fullsized":"2024-03-14T09:04:09.783362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:48.425158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400912+0000","last_change":"2024-03-14T09:04:09.800044+0000","last_active":"2024-03-14T09:04:11.400912+0000","last_peered":"2024-03-14T09:04:11.400912+0000","last_clean":"2024-03-14T09:04:11.400912+0000","last_became_active":"2024-03-14T09:04:09.799798+0000","last_became_peered":"2024-03-14T09:04:09.799798+0000","last_unstale":"2024-03-14T09:04:11.400912+0000","last_undegraded":"2024-03-14T09:04:11.400912+0000","last_fullsized":"2024-03-14T09:04:11.400912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:30.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777727+0000","last_change":"2024-03-14T09:03:57.185432+0000","last_active":"2024-03-14T09:04:09.777727+0000","last_peered":"2024-03-14T09:04:09.777727+0000","last_clean":"2024-03-14T09:04:09.777727+0000","last_became_active":"2024-03-14T09:03:57.184879+0000","last_became_peered":"2024-03-14T09:03:57.184879+0000","last_unstale":"2024-03-14T09:04:09.777727+0000","last_undegraded":"2024-03-14T09:04:09.777727+0000","last_fullsized":"2024-03-14T09:04:09.777727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:22.429275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400959+0000","last_change":"2024-03-14T09:04:09.791951+0000","last_active":"2024-03-14T09:04:11.400959+0000","last_peered":"2024-03-14T09:04:11.400959+0000","last_clean":"2024-03-14T09:04:11.400959+0000","last_became_active":"2024-03-14T09:04:09.791710+0000","last_became_peered":"2024-03-14T09:04:09.791710+0000","last_unstale":"2024-03-14T09:04:11.400959+0000","last_undegraded":"2024-03-14T09:04:11.400959+0000","last_fullsized":"2024-03-14T09:04:11.400959+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:53:59.403833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793523+0000","last_change":"2024-03-14T09:03:57.186461+0000","last_active":"2024-03-14T09:04:09.793523+0000","last_peered":"2024-03-14T09:04:09.793523+0000","last_clean":"2024-03-14T09:04:09.793523+0000","last_became_active":"2024-03-14T09:03:57.186176+0000","last_became_peered":"2024-03-14T09:03:57.186176+0000","last_unstale":"2024-03-14T09:04:09.793523+0000","last_undegraded":"2024-03-14T09:04:09.793523+0000","last_fullsized":"2024-03-14T09:04:09.793523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784476+0000","last_change":"2024-03-14T09:04:09.792952+0000","last_active":"2024-03-14T09:04:10.784476+0000","last_peered":"2024-03-14T09:04:10.784476+0000","last_clean":"2024-03-14T09:04:10.784476+0000","last_became_active":"2024-03-14T09:04:09.792695+0000","last_became_peered":"2024-03-14T09:04:09.792695+0000","last_unstale":"2024-03-14T09:04:10.784476+0000","last_undegraded":"2024-03-14T09:04:10.784476+0000","last_fullsized":"2024-03-14T09:04:10.784476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:06:30.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776622+0000","last_change":"2024-03-14T09:03:57.185583+0000","last_active":"2024-03-14T09:04:09.776622+0000","last_peered":"2024-03-14T09:04:09.776622+0000","last_clean":"2024-03-14T09:04:09.776622+0000","last_became_active":"2024-03-14T09:03:57.185292+0000","last_became_peered":"2024-03-14T09:03:57.185292+0000","last_unstale":"2024-03-14T09:04:09.776622+0000","last_undegraded":"2024-03-14T09:04:09.776622+0000","last_fullsized":"2024-03-14T09:04:09.776622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400517+0000","last_change":"2024-03-14T09:04:09.800474+0000","last_active":"2024-03-14T09:04:11.400517+0000","last_peered":"2024-03-14T09:04:11.400517+0000","last_clean":"2024-03-14T09:04:11.400517+0000","last_became_active":"2024-03-14T09:04:09.800187+0000","last_became_peered":"2024-03-14T09:04:09.800187+0000","last_unstale":"2024-03-14T09:04:11.400517+0000","last_undegraded":"2024-03-14T09:04:11.400517+0000","last_fullsized":"2024-03-14T09:04:11.400517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:34.901831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401535+0000","last_change":"2024-03-14T09:04:09.375153+0000","last_active":"2024-03-14T09:04:11.401535+0000","last_peered":"2024-03-14T09:04:11.401535+0000","last_clean":"2024-03-14T09:04:11.401535+0000","last_became_active":"2024-03-14T09:04:09.374755+0000","last_became_peered":"2024-03-14T09:04:09.374755+0000","last_unstale":"2024-03-14T09:04:11.401535+0000","last_undegraded":"2024-03-14T09:04:11.401535+0000","last_fullsized":"2024-03-14T09:04:11.401535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:48:49.117112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796295+0000","last_change":"2024-03-14T09:03:57.182704+0000","last_active":"2024-03-14T09:04:09.796295+0000","last_peered":"2024-03-14T09:04:09.796295+0000","last_clean":"2024-03-14T09:04:09.796295+0000","last_became_active":"2024-03-14T09:03:57.182438+0000","last_became_peered":"2024-03-14T09:03:57.182438+0000","last_unstale":"2024-03-14T09:04:09.796295+0000","last_undegraded":"2024-03-14T09:04:09.796295+0000","last_fullsized":"2024-03-14T09:04:09.796295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"69'4","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779504+0000","last_change":"2024-03-14T09:03:56.771973+0000","last_active":"2024-03-14T09:04:09.779504+0000","last_peered":"2024-03-14T09:04:09.779504+0000","last_clean":"2024-03-14T09:04:09.779504+0000","last_became_active":"2024-03-14T09:03:56.771714+0000","last_became_peered":"2024-03-14T09:03:56.771714+0000","last_unstale":"2024-03-14T09:04:09.779504+0000","last_undegraded":"2024-03-14T09:04:09.779504+0000","last_fullsized":"2024-03-14T09:04:09.779504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:17.873268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783195+0000","last_change":"2024-03-14T09:04:09.803323+0000","last_active":"2024-03-14T09:04:10.783195+0000","last_peered":"2024-03-14T09:04:10.783195+0000","last_clean":"2024-03-14T09:04:10.783195+0000","last_became_active":"2024-03-14T09:04:09.801887+0000","last_became_peered":"2024-03-14T09:04:09.801887+0000","last_unstale":"2024-03-14T09:04:10.783195+0000","last_undegraded":"2024-03-14T09:04:10.783195+0000","last_fullsized":"2024-03-14T09:04:10.783195+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:02.116523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401359+0000","last_change":"2024-03-14T09:04:09.366923+0000","last_active":"2024-03-14T09:04:11.401359+0000","last_peered":"2024-03-14T09:04:11.401359+0000","last_clean":"2024-03-14T09:04:11.401359+0000","last_became_active":"2024-03-14T09:04:09.366662+0000","last_became_peered":"2024-03-14T09:04:09.366662+0000","last_unstale":"2024-03-14T09:04:11.401359+0000","last_undegraded":"2024-03-14T09:04:11.401359+0000","last_fullsized":"2024-03-14T09:04:11.401359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:13.944461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779306+0000","last_change":"2024-03-14T09:03:57.185961+0000","last_active":"2024-03-14T09:04:09.779306+0000","last_peered":"2024-03-14T09:04:09.779306+0000","last_clean":"2024-03-14T09:04:09.779306+0000","last_became_active":"2024-03-14T09:03:57.185658+0000","last_became_peered":"2024-03-14T09:03:57.185658+0000","last_unstale":"2024-03-14T09:04:09.779306+0000","last_undegraded":"2024-03-14T09:04:09.779306+0000","last_fullsized":"2024-03-14T09:04:09.779306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796199+0000","last_change":"2024-03-14T09:03:56.769740+0000","last_active":"2024-03-14T09:04:09.796199+0000","last_peered":"2024-03-14T09:04:09.796199+0000","last_clean":"2024-03-14T09:04:09.796199+0000","last_became_active":"2024-03-14T09:03:56.769429+0000","last_became_peered":"2024-03-14T09:03:56.769429+0000","last_unstale":"2024-03-14T09:04:09.796199+0000","last_undegraded":"2024-03-14T09:04:09.796199+0000","last_fullsized":"2024-03-14T09:04:09.796199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.785834+0000","last_change":"2024-03-14T09:04:09.792199+0000","last_active":"2024-03-14T09:04:10.785834+0000","last_peered":"2024-03-14T09:04:10.785834+0000","last_clean":"2024-03-14T09:04:10.785834+0000","last_became_active":"2024-03-14T09:04:09.791888+0000","last_became_peered":"2024-03-14T09:04:09.791888+0000","last_unstale":"2024-03-14T09:04:10.785834+0000","last_undegraded":"2024-03-14T09:04:10.785834+0000","last_fullsized":"2024-03-14T09:04:10.785834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:16:57.041712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"73'4","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401441+0000","last_change":"2024-03-14T09:04:09.375907+0000","last_active":"2024-03-14T09:04:11.401441+0000","last_peered":"2024-03-14T09:04:11.401441+0000","last_clean":"2024-03-14T09:04:11.401441+0000","last_became_active":"2024-03-14T09:04:09.375650+0000","last_became_peered":"2024-03-14T09:04:09.375650+0000","last_unstale":"2024-03-14T09:04:11.401441+0000","last_undegraded":"2024-03-14T09:04:11.401441+0000","last_fullsized":"2024-03-14T09:04:11.401441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:33:47.940938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777616+0000","last_change":"2024-03-14T09:03:57.188083+0000","last_active":"2024-03-14T09:04:09.777616+0000","last_peered":"2024-03-14T09:04:09.777616+0000","last_clean":"2024-03-14T09:04:09.777616+0000","last_became_active":"2024-03-14T09:03:57.187832+0000","last_became_peered":"2024-03-14T09:03:57.187832+0000","last_unstale":"2024-03-14T09:04:09.777616+0000","last_undegraded":"2024-03-14T09:04:09.777616+0000","last_fullsized":"2024-03-14T09:04:09.777616+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773860+0000","last_change":"2024-03-14T09:03:56.771927+0000","last_active":"2024-03-14T09:04:09.773860+0000","last_peered":"2024-03-14T09:04:09.773860+0000","last_clean":"2024-03-14T09:04:09.773860+0000","last_became_active":"2024-03-14T09:03:56.771684+0000","last_became_peered":"2024-03-14T09:03:56.771684+0000","last_unstale":"2024-03-14T09:04:09.773860+0000","last_undegraded":"2024-03-14T09:04:09.773860+0000","last_fullsized":"2024-03-14T09:04:09.773860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784332+0000","last_change":"2024-03-14T09:04:09.790695+0000","last_active":"2024-03-14T09:04:10.784332+0000","last_peered":"2024-03-14T09:04:10.784332+0000","last_clean":"2024-03-14T09:04:10.784332+0000","last_became_active":"2024-03-14T09:04:09.790184+0000","last_became_peered":"2024-03-14T09:04:09.790184+0000","last_unstale":"2024-03-14T09:04:10.784332+0000","last_undegraded":"2024-03-14T09:04:10.784332+0000","last_fullsized":"2024-03-14T09:04:10.784332+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:53:59.403833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400936+0000","last_change":"2024-03-14T09:04:09.379525+0000","last_active":"2024-03-14T09:04:11.400936+0000","last_peered":"2024-03-14T09:04:11.400936+0000","last_clean":"2024-03-14T09:04:11.400936+0000","last_became_active":"2024-03-14T09:04:09.379216+0000","last_became_peered":"2024-03-14T09:04:09.379216+0000","last_unstale":"2024-03-14T09:04:11.400936+0000","last_undegraded":"2024-03-14T09:04:11.400936+0000","last_fullsized":"2024-03-14T09:04:11.400936+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:13.620690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781015+0000","last_change":"2024-03-14T09:03:57.180285+0000","last_active":"2024-03-14T09:04:09.781015+0000","last_peered":"2024-03-14T09:04:09.781015+0000","last_clean":"2024-03-14T09:04:09.781015+0000","last_became_active":"2024-03-14T09:03:57.179838+0000","last_became_peered":"2024-03-14T09:03:57.179838+0000","last_unstale":"2024-03-14T09:04:09.781015+0000","last_undegraded":"2024-03-14T09:04:09.781015+0000","last_fullsized":"2024-03-14T09:04:09.781015+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.775275+0000","last_change":"2024-03-14T09:03:56.772021+0000","last_active":"2024-03-14T09:04:09.775275+0000","last_peered":"2024-03-14T09:04:09.775275+0000","last_clean":"2024-03-14T09:04:09.775275+0000","last_became_active":"2024-03-14T09:03:56.771764+0000","last_became_peered":"2024-03-14T09:03:56.771764+0000","last_unstale":"2024-03-14T09:04:09.775275+0000","last_undegraded":"2024-03-14T09:04:09.775275+0000","last_fullsized":"2024-03-14T09:04:09.775275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:49:30.727586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782933+0000","last_change":"2024-03-14T09:04:09.791770+0000","last_active":"2024-03-14T09:04:10.782933+0000","last_peered":"2024-03-14T09:04:10.782933+0000","last_clean":"2024-03-14T09:04:10.782933+0000","last_became_active":"2024-03-14T09:04:09.791566+0000","last_became_peered":"2024-03-14T09:04:09.791566+0000","last_unstale":"2024-03-14T09:04:10.782933+0000","last_undegraded":"2024-03-14T09:04:10.782933+0000","last_fullsized":"2024-03-14T09:04:10.782933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:33:45.300921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400327+0000","last_change":"2024-03-14T09:04:09.381693+0000","last_active":"2024-03-14T09:04:11.400327+0000","last_peered":"2024-03-14T09:04:11.400327+0000","last_clean":"2024-03-14T09:04:11.400327+0000","last_became_active":"2024-03-14T09:04:09.381368+0000","last_became_peered":"2024-03-14T09:04:09.381368+0000","last_unstale":"2024-03-14T09:04:11.400327+0000","last_undegraded":"2024-03-14T09:04:11.400327+0000","last_fullsized":"2024-03-14T09:04:11.400327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:30:54.608231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777577+0000","last_change":"2024-03-14T09:03:57.185267+0000","last_active":"2024-03-14T09:04:09.777577+0000","last_peered":"2024-03-14T09:04:09.777577+0000","last_clean":"2024-03-14T09:04:09.777577+0000","last_became_active":"2024-03-14T09:03:57.184614+0000","last_became_peered":"2024-03-14T09:03:57.184614+0000","last_unstale":"2024-03-14T09:04:09.777577+0000","last_undegraded":"2024-03-14T09:04:09.777577+0000","last_fullsized":"2024-03-14T09:04:09.777577+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:25.314389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793428+0000","last_change":"2024-03-14T09:03:56.770949+0000","last_active":"2024-03-14T09:04:09.793428+0000","last_peered":"2024-03-14T09:04:09.793428+0000","last_clean":"2024-03-14T09:04:09.793428+0000","last_became_active":"2024-03-14T09:03:56.770678+0000","last_became_peered":"2024-03-14T09:03:56.770678+0000","last_unstale":"2024-03-14T09:04:09.793428+0000","last_undegraded":"2024-03-14T09:04:09.793428+0000","last_fullsized":"2024-03-14T09:04:09.793428+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783187+0000","last_change":"2024-03-14T09:04:09.800588+0000","last_active":"2024-03-14T09:04:10.783187+0000","last_peered":"2024-03-14T09:04:10.783187+0000","last_clean":"2024-03-14T09:04:10.783187+0000","last_became_active":"2024-03-14T09:04:09.800315+0000","last_became_peered":"2024-03-14T09:04:09.800315+0000","last_unstale":"2024-03-14T09:04:10.783187+0000","last_undegraded":"2024-03-14T09:04:10.783187+0000","last_fullsized":"2024-03-14T09:04:10.783187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:21:41.207805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.399939+0000","last_change":"2024-03-14T09:04:09.377623+0000","last_active":"2024-03-14T09:04:11.399939+0000","last_peered":"2024-03-14T09:04:11.399939+0000","last_clean":"2024-03-14T09:04:11.399939+0000","last_became_active":"2024-03-14T09:04:09.377333+0000","last_became_peered":"2024-03-14T09:04:09.377333+0000","last_unstale":"2024-03-14T09:04:11.399939+0000","last_undegraded":"2024-03-14T09:04:11.399939+0000","last_fullsized":"2024-03-14T09:04:11.399939+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:53.701753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793884+0000","last_change":"2024-03-14T09:03:57.178070+0000","last_active":"2024-03-14T09:04:09.793884+0000","last_peered":"2024-03-14T09:04:09.793884+0000","last_clean":"2024-03-14T09:04:09.793884+0000","last_became_active":"2024-03-14T09:03:57.177621+0000","last_became_peered":"2024-03-14T09:03:57.177621+0000","last_unstale":"2024-03-14T09:04:09.793884+0000","last_undegraded":"2024-03-14T09:04:09.793884+0000","last_fullsized":"2024-03-14T09:04:09.793884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:55.259556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793461+0000","last_change":"2024-03-14T09:03:56.768242+0000","last_active":"2024-03-14T09:04:09.793461+0000","last_peered":"2024-03-14T09:04:09.793461+0000","last_clean":"2024-03-14T09:04:09.793461+0000","last_became_active":"2024-03-14T09:03:56.767960+0000","last_became_peered":"2024-03-14T09:03:56.767960+0000","last_unstale":"2024-03-14T09:04:09.793461+0000","last_undegraded":"2024-03-14T09:04:09.793461+0000","last_fullsized":"2024-03-14T09:04:09.793461+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400802+0000","last_change":"2024-03-14T09:04:09.797741+0000","last_active":"2024-03-14T09:04:11.400802+0000","last_peered":"2024-03-14T09:04:11.400802+0000","last_clean":"2024-03-14T09:04:11.400802+0000","last_became_active":"2024-03-14T09:04:09.797472+0000","last_became_peered":"2024-03-14T09:04:09.797472+0000","last_unstale":"2024-03-14T09:04:11.400802+0000","last_undegraded":"2024-03-14T09:04:11.400802+0000","last_fullsized":"2024-03-14T09:04:11.400802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:50.300466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400473+0000","last_change":"2024-03-14T09:04:09.379004+0000","last_active":"2024-03-14T09:04:11.400473+0000","last_peered":"2024-03-14T09:04:11.400473+0000","last_clean":"2024-03-14T09:04:11.400473+0000","last_became_active":"2024-03-14T09:04:09.378729+0000","last_became_peered":"2024-03-14T09:04:09.378729+0000","last_unstale":"2024-03-14T09:04:11.400473+0000","last_undegraded":"2024-03-14T09:04:11.400473+0000","last_fullsized":"2024-03-14T09:04:11.400473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:20.339699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781239+0000","last_change":"2024-03-14T09:03:57.187305+0000","last_active":"2024-03-14T09:04:09.781239+0000","last_peered":"2024-03-14T09:04:09.781239+0000","last_clean":"2024-03-14T09:04:09.781239+0000","last_became_active":"2024-03-14T09:03:57.186346+0000","last_became_peered":"2024-03-14T09:03:57.186346+0000","last_unstale":"2024-03-14T09:04:09.781239+0000","last_undegraded":"2024-03-14T09:04:09.781239+0000","last_fullsized":"2024-03-14T09:04:09.781239+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:04.018081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773576+0000","last_change":"2024-03-14T09:03:56.773036+0000","last_active":"2024-03-14T09:04:09.773576+0000","last_peered":"2024-03-14T09:04:09.773576+0000","last_clean":"2024-03-14T09:04:09.773576+0000","last_became_active":"2024-03-14T09:03:56.772740+0000","last_became_peered":"2024-03-14T09:03:56.772740+0000","last_unstale":"2024-03-14T09:04:09.773576+0000","last_undegraded":"2024-03-14T09:04:09.773576+0000","last_fullsized":"2024-03-14T09:04:09.773576+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:44.982623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400200+0000","last_change":"2024-03-14T09:04:09.787975+0000","last_active":"2024-03-14T09:04:11.400200+0000","last_peered":"2024-03-14T09:04:11.400200+0000","last_clean":"2024-03-14T09:04:11.400200+0000","last_became_active":"2024-03-14T09:04:09.787646+0000","last_became_peered":"2024-03-14T09:04:09.787646+0000","last_unstale":"2024-03-14T09:04:11.400200+0000","last_undegraded":"2024-03-14T09:04:11.400200+0000","last_fullsized":"2024-03-14T09:04:11.400200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:45:57.150041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400982+0000","last_change":"2024-03-14T09:04:09.381315+0000","last_active":"2024-03-14T09:04:11.400982+0000","last_peered":"2024-03-14T09:04:11.400982+0000","last_clean":"2024-03-14T09:04:11.400982+0000","last_became_active":"2024-03-14T09:04:09.381017+0000","last_became_peered":"2024-03-14T09:04:09.381017+0000","last_unstale":"2024-03-14T09:04:11.400982+0000","last_undegraded":"2024-03-14T09:04:11.400982+0000","last_fullsized":"2024-03-14T09:04:11.400982+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:04:59.084242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782553+0000","last_change":"2024-03-14T09:03:57.172128+0000","last_active":"2024-03-14T09:04:09.782553+0000","last_peered":"2024-03-14T09:04:09.782553+0000","last_clean":"2024-03-14T09:04:09.782553+0000","last_became_active":"2024-03-14T09:03:57.171877+0000","last_became_peered":"2024-03-14T09:03:57.171877+0000","last_unstale":"2024-03-14T09:04:09.782553+0000","last_undegraded":"2024-03-14T09:04:09.782553+0000","last_fullsized":"2024-03-14T09:04:09.782553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:26:25.355300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780677+0000","last_change":"2024-03-14T09:03:56.769464+0000","last_active":"2024-03-14T09:04:09.780677+0000","last_peered":"2024-03-14T09:04:09.780677+0000","last_clean":"2024-03-14T09:04:09.780677+0000","last_became_active":"2024-03-14T09:03:56.769231+0000","last_became_peered":"2024-03-14T09:03:56.769231+0000","last_unstale":"2024-03-14T09:04:09.780677+0000","last_undegraded":"2024-03-14T09:04:09.780677+0000","last_fullsized":"2024-03-14T09:04:09.780677+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:50.319706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401132+0000","last_change":"2024-03-14T09:04:09.779677+0000","last_active":"2024-03-14T09:04:11.401132+0000","last_peered":"2024-03-14T09:04:11.401132+0000","last_clean":"2024-03-14T09:04:11.401132+0000","last_became_active":"2024-03-14T09:04:09.779363+0000","last_became_peered":"2024-03-14T09:04:09.779363+0000","last_unstale":"2024-03-14T09:04:11.401132+0000","last_undegraded":"2024-03-14T09:04:11.401132+0000","last_fullsized":"2024-03-14T09:04:11.401132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:51.183035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"73'3","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401357+0000","last_change":"2024-03-14T09:04:09.377329+0000","last_active":"2024-03-14T09:04:11.401357+0000","last_peered":"2024-03-14T09:04:11.401357+0000","last_clean":"2024-03-14T09:04:11.401357+0000","last_became_active":"2024-03-14T09:04:09.377014+0000","last_became_peered":"2024-03-14T09:04:09.377014+0000","last_unstale":"2024-03-14T09:04:11.401357+0000","last_undegraded":"2024-03-14T09:04:11.401357+0000","last_fullsized":"2024-03-14T09:04:11.401357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:48:49.117112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779309+0000","last_change":"2024-03-14T09:03:57.188282+0000","last_active":"2024-03-14T09:04:09.779309+0000","last_peered":"2024-03-14T09:04:09.779309+0000","last_clean":"2024-03-14T09:04:09.779309+0000","last_became_active":"2024-03-14T09:03:57.188018+0000","last_became_peered":"2024-03-14T09:03:57.188018+0000","last_unstale":"2024-03-14T09:04:09.779309+0000","last_undegraded":"2024-03-14T09:04:09.779309+0000","last_fullsized":"2024-03-14T09:04:09.779309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:22.289306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779742+0000","last_change":"2024-03-14T09:03:56.768124+0000","last_active":"2024-03-14T09:04:09.779742+0000","last_peered":"2024-03-14T09:04:09.779742+0000","last_clean":"2024-03-14T09:04:09.779742+0000","last_became_active":"2024-03-14T09:03:56.766669+0000","last_became_peered":"2024-03-14T09:03:56.766669+0000","last_unstale":"2024-03-14T09:04:09.779742+0000","last_undegraded":"2024-03-14T09:04:09.779742+0000","last_fullsized":"2024-03-14T09:04:09.779742+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:11.739150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401238+0000","last_change":"2024-03-14T09:04:09.799468+0000","last_active":"2024-03-14T09:04:11.401238+0000","last_peered":"2024-03-14T09:04:11.401238+0000","last_clean":"2024-03-14T09:04:11.401238+0000","last_became_active":"2024-03-14T09:04:09.799145+0000","last_became_peered":"2024-03-14T09:04:09.799145+0000","last_unstale":"2024-03-14T09:04:11.401238+0000","last_undegraded":"2024-03-14T09:04:11.401238+0000","last_fullsized":"2024-03-14T09:04:11.401238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:04:01.338280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"73'6","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783315+0000","last_change":"2024-03-14T09:04:09.376557+0000","last_active":"2024-03-14T09:04:10.783315+0000","last_peered":"2024-03-14T09:04:10.783315+0000","last_clean":"2024-03-14T09:04:10.783315+0000","last_became_active":"2024-03-14T09:04:09.376141+0000","last_became_peered":"2024-03-14T09:04:09.376141+0000","last_unstale":"2024-03-14T09:04:10.783315+0000","last_undegraded":"2024-03-14T09:04:10.783315+0000","last_fullsized":"2024-03-14T09:04:10.783315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:30:20.802531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785007+0000","last_change":"2024-03-14T09:03:57.179078+0000","last_active":"2024-03-14T09:04:09.785007+0000","last_peered":"2024-03-14T09:04:09.785007+0000","last_clean":"2024-03-14T09:04:09.785007+0000","last_became_active":"2024-03-14T09:03:57.178782+0000","last_became_peered":"2024-03-14T09:03:57.178782+0000","last_unstale":"2024-03-14T09:04:09.785007+0000","last_undegraded":"2024-03-14T09:04:09.785007+0000","last_fullsized":"2024-03-14T09:04:09.785007+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:30.363727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778254+0000","last_change":"2024-03-14T09:03:56.769911+0000","last_active":"2024-03-14T09:04:09.778254+0000","last_peered":"2024-03-14T09:04:09.778254+0000","last_clean":"2024-03-14T09:04:09.778254+0000","last_became_active":"2024-03-14T09:03:56.769598+0000","last_became_peered":"2024-03-14T09:03:56.769598+0000","last_unstale":"2024-03-14T09:04:09.778254+0000","last_undegraded":"2024-03-14T09:04:09.778254+0000","last_fullsized":"2024-03-14T09:04:09.778254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:14.873107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401675+0000","last_change":"2024-03-14T09:04:09.789438+0000","last_active":"2024-03-14T09:04:11.401675+0000","last_peered":"2024-03-14T09:04:11.401675+0000","last_clean":"2024-03-14T09:04:11.401675+0000","last_became_active":"2024-03-14T09:04:09.789207+0000","last_became_peered":"2024-03-14T09:04:09.789207+0000","last_unstale":"2024-03-14T09:04:11.401675+0000","last_undegraded":"2024-03-14T09:04:11.401675+0000","last_fullsized":"2024-03-14T09:04:11.401675+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:29:18.431413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400469+0000","last_change":"2024-03-14T09:04:09.378381+0000","last_active":"2024-03-14T09:04:11.400469+0000","last_peered":"2024-03-14T09:04:11.400469+0000","last_clean":"2024-03-14T09:04:11.400469+0000","last_became_active":"2024-03-14T09:04:09.378135+0000","last_became_peered":"2024-03-14T09:04:09.378135+0000","last_unstale":"2024-03-14T09:04:11.400469+0000","last_undegraded":"2024-03-14T09:04:11.400469+0000","last_fullsized":"2024-03-14T09:04:11.400469+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:48.386429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.775095+0000","last_change":"2024-03-14T09:03:57.184589+0000","last_active":"2024-03-14T09:04:09.775095+0000","last_peered":"2024-03-14T09:04:09.775095+0000","last_clean":"2024-03-14T09:04:09.775095+0000","last_became_active":"2024-03-14T09:03:57.184303+0000","last_became_peered":"2024-03-14T09:03:57.184303+0000","last_unstale":"2024-03-14T09:04:09.775095+0000","last_undegraded":"2024-03-14T09:04:09.775095+0000","last_fullsized":"2024-03-14T09:04:09.775095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:38.431473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"69'3","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778906+0000","last_change":"2024-03-14T09:03:56.766306+0000","last_active":"2024-03-14T09:04:09.778906+0000","last_peered":"2024-03-14T09:04:09.778906+0000","last_clean":"2024-03-14T09:04:09.778906+0000","last_became_active":"2024-03-14T09:03:56.766045+0000","last_became_peered":"2024-03-14T09:03:56.766045+0000","last_unstale":"2024-03-14T09:04:09.778906+0000","last_undegraded":"2024-03-14T09:04:09.778906+0000","last_fullsized":"2024-03-14T09:04:09.778906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783580+0000","last_change":"2024-03-14T09:04:09.803111+0000","last_active":"2024-03-14T09:04:10.783580+0000","last_peered":"2024-03-14T09:04:10.783580+0000","last_clean":"2024-03-14T09:04:10.783580+0000","last_became_active":"2024-03-14T09:04:09.801957+0000","last_became_peered":"2024-03-14T09:04:09.801957+0000","last_unstale":"2024-03-14T09:04:10.783580+0000","last_undegraded":"2024-03-14T09:04:10.783580+0000","last_fullsized":"2024-03-14T09:04:10.783580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:43.806093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784034+0000","last_change":"2024-03-14T09:04:09.381721+0000","last_active":"2024-03-14T09:04:10.784034+0000","last_peered":"2024-03-14T09:04:10.784034+0000","last_clean":"2024-03-14T09:04:10.784034+0000","last_became_active":"2024-03-14T09:04:09.381428+0000","last_became_peered":"2024-03-14T09:04:09.381428+0000","last_unstale":"2024-03-14T09:04:10.784034+0000","last_undegraded":"2024-03-14T09:04:10.784034+0000","last_fullsized":"2024-03-14T09:04:10.784034+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:50.175976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796183+0000","last_change":"2024-03-14T09:03:57.189394+0000","last_active":"2024-03-14T09:04:09.796183+0000","last_peered":"2024-03-14T09:04:09.796183+0000","last_clean":"2024-03-14T09:04:09.796183+0000","last_became_active":"2024-03-14T09:03:57.189162+0000","last_became_peered":"2024-03-14T09:03:57.189162+0000","last_unstale":"2024-03-14T09:04:09.796183+0000","last_undegraded":"2024-03-14T09:04:09.796183+0000","last_fullsized":"2024-03-14T09:04:09.796183+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"70'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784554+0000","last_change":"2024-03-14T09:03:56.769295+0000","last_active":"2024-03-14T09:04:09.784554+0000","last_peered":"2024-03-14T09:04:09.784554+0000","last_clean":"2024-03-14T09:04:09.784554+0000","last_became_active":"2024-03-14T09:03:56.768952+0000","last_became_peered":"2024-03-14T09:03:56.768952+0000","last_unstale":"2024-03-14T09:04:09.784554+0000","last_undegraded":"2024-03-14T09:04:09.784554+0000","last_fullsized":"2024-03-14T09:04:09.784554+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:46:36.047193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3075,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782666+0000","last_change":"2024-03-14T09:04:09.797259+0000","last_active":"2024-03-14T09:04:10.782666+0000","last_peered":"2024-03-14T09:04:10.782666+0000","last_clean":"2024-03-14T09:04:10.782666+0000","last_became_active":"2024-03-14T09:04:09.796798+0000","last_became_peered":"2024-03-14T09:04:09.796798+0000","last_unstale":"2024-03-14T09:04:10.782666+0000","last_undegraded":"2024-03-14T09:04:10.782666+0000","last_fullsized":"2024-03-14T09:04:10.782666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:42:01.168724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782720+0000","last_change":"2024-03-14T09:04:09.375024+0000","last_active":"2024-03-14T09:04:10.782720+0000","last_peered":"2024-03-14T09:04:10.782720+0000","last_clean":"2024-03-14T09:04:10.782720+0000","last_became_active":"2024-03-14T09:04:09.372740+0000","last_became_peered":"2024-03-14T09:04:09.372740+0000","last_unstale":"2024-03-14T09:04:10.782720+0000","last_undegraded":"2024-03-14T09:04:10.782720+0000","last_fullsized":"2024-03-14T09:04:10.782720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:14.302651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772985+0000","last_change":"2024-03-14T09:03:57.188359+0000","last_active":"2024-03-14T09:04:09.772985+0000","last_peered":"2024-03-14T09:04:09.772985+0000","last_clean":"2024-03-14T09:04:09.772985+0000","last_became_active":"2024-03-14T09:03:57.188093+0000","last_became_peered":"2024-03-14T09:03:57.188093+0000","last_unstale":"2024-03-14T09:04:09.772985+0000","last_undegraded":"2024-03-14T09:04:09.772985+0000","last_fullsized":"2024-03-14T09:04:09.772985+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:22.745406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783266+0000","last_change":"2024-03-14T09:03:56.769589+0000","last_active":"2024-03-14T09:04:09.783266+0000","last_peered":"2024-03-14T09:04:09.783266+0000","last_clean":"2024-03-14T09:04:09.783266+0000","last_became_active":"2024-03-14T09:03:56.767656+0000","last_became_peered":"2024-03-14T09:03:56.767656+0000","last_unstale":"2024-03-14T09:04:09.783266+0000","last_undegraded":"2024-03-14T09:04:09.783266+0000","last_fullsized":"2024-03-14T09:04:09.783266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:31:31.954923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783967+0000","last_change":"2024-03-14T09:04:09.801012+0000","last_active":"2024-03-14T09:04:10.783967+0000","last_peered":"2024-03-14T09:04:10.783967+0000","last_clean":"2024-03-14T09:04:10.783967+0000","last_became_active":"2024-03-14T09:04:09.800759+0000","last_became_peered":"2024-03-14T09:04:09.800759+0000","last_unstale":"2024-03-14T09:04:10.783967+0000","last_undegraded":"2024-03-14T09:04:10.783967+0000","last_fullsized":"2024-03-14T09:04:10.783967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:21.346758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"73'8","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400939+0000","last_change":"2024-03-14T09:04:09.376339+0000","last_active":"2024-03-14T09:04:11.400939+0000","last_peered":"2024-03-14T09:04:11.400939+0000","last_clean":"2024-03-14T09:04:11.400939+0000","last_became_active":"2024-03-14T09:04:09.376055+0000","last_became_peered":"2024-03-14T09:04:09.376055+0000","last_unstale":"2024-03-14T09:04:11.400939+0000","last_undegraded":"2024-03-14T09:04:11.400939+0000","last_fullsized":"2024-03-14T09:04:11.400939+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:34.350871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779954+0000","last_change":"2024-03-14T09:03:57.187466+0000","last_active":"2024-03-14T09:04:09.779954+0000","last_peered":"2024-03-14T09:04:09.779954+0000","last_clean":"2024-03-14T09:04:09.779954+0000","last_became_active":"2024-03-14T09:03:57.186590+0000","last_became_peered":"2024-03-14T09:03:57.186590+0000","last_unstale":"2024-03-14T09:04:09.779954+0000","last_undegraded":"2024-03-14T09:04:09.779954+0000","last_fullsized":"2024-03-14T09:04:09.779954+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:04.118845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777088+0000","last_change":"2024-03-14T09:03:56.762545+0000","last_active":"2024-03-14T09:04:09.777088+0000","last_peered":"2024-03-14T09:04:09.777088+0000","last_clean":"2024-03-14T09:04:09.777088+0000","last_became_active":"2024-03-14T09:03:56.762292+0000","last_became_peered":"2024-03-14T09:03:56.762292+0000","last_unstale":"2024-03-14T09:04:09.777088+0000","last_undegraded":"2024-03-14T09:04:09.777088+0000","last_fullsized":"2024-03-14T09:04:09.777088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:10:03.013048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.782972+0000","last_change":"2024-03-14T09:04:09.784017+0000","last_active":"2024-03-14T09:04:10.782972+0000","last_peered":"2024-03-14T09:04:10.782972+0000","last_clean":"2024-03-14T09:04:10.782972+0000","last_became_active":"2024-03-14T09:04:09.783753+0000","last_became_peered":"2024-03-14T09:04:09.783753+0000","last_unstale":"2024-03-14T09:04:10.782972+0000","last_undegraded":"2024-03-14T09:04:10.782972+0000","last_fullsized":"2024-03-14T09:04:10.782972+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:47:13.189988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784705+0000","last_change":"2024-03-14T09:04:09.379876+0000","last_active":"2024-03-14T09:04:10.784705+0000","last_peered":"2024-03-14T09:04:10.784705+0000","last_clean":"2024-03-14T09:04:10.784705+0000","last_became_active":"2024-03-14T09:04:09.379335+0000","last_became_peered":"2024-03-14T09:04:09.379335+0000","last_unstale":"2024-03-14T09:04:10.784705+0000","last_undegraded":"2024-03-14T09:04:10.784705+0000","last_fullsized":"2024-03-14T09:04:10.784705+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:20:12.537175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780758+0000","last_change":"2024-03-14T09:03:57.184740+0000","last_active":"2024-03-14T09:04:09.780758+0000","last_peered":"2024-03-14T09:04:09.780758+0000","last_clean":"2024-03-14T09:04:09.780758+0000","last_became_active":"2024-03-14T09:03:57.184407+0000","last_became_peered":"2024-03-14T09:03:57.184407+0000","last_unstale":"2024-03-14T09:04:09.780758+0000","last_undegraded":"2024-03-14T09:04:09.780758+0000","last_fullsized":"2024-03-14T09:04:09.780758+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:17:12.744182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.783972+0000","last_change":"2024-03-14T09:03:56.773362+0000","last_active":"2024-03-14T09:04:09.783972+0000","last_peered":"2024-03-14T09:04:09.783972+0000","last_clean":"2024-03-14T09:04:09.783972+0000","last_became_active":"2024-03-14T09:03:56.773012+0000","last_became_peered":"2024-03-14T09:03:56.773012+0000","last_unstale":"2024-03-14T09:04:09.783972+0000","last_undegraded":"2024-03-14T09:04:09.783972+0000","last_fullsized":"2024-03-14T09:04:09.783972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:38.243214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400833+0000","last_change":"2024-03-14T09:04:09.796325+0000","last_active":"2024-03-14T09:04:11.400833+0000","last_peered":"2024-03-14T09:04:11.400833+0000","last_clean":"2024-03-14T09:04:11.400833+0000","last_became_active":"2024-03-14T09:04:09.796077+0000","last_became_peered":"2024-03-14T09:04:09.796077+0000","last_unstale":"2024-03-14T09:04:11.400833+0000","last_undegraded":"2024-03-14T09:04:11.400833+0000","last_fullsized":"2024-03-14T09:04:11.400833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:17:29.957958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784549+0000","last_change":"2024-03-14T09:04:09.379863+0000","last_active":"2024-03-14T09:04:10.784549+0000","last_peered":"2024-03-14T09:04:10.784549+0000","last_clean":"2024-03-14T09:04:10.784549+0000","last_became_active":"2024-03-14T09:04:09.379592+0000","last_became_peered":"2024-03-14T09:04:09.379592+0000","last_unstale":"2024-03-14T09:04:10.784549+0000","last_undegraded":"2024-03-14T09:04:10.784549+0000","last_fullsized":"2024-03-14T09:04:10.784549+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:21:46.907970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.778330+0000","last_change":"2024-03-14T09:03:57.187459+0000","last_active":"2024-03-14T09:04:09.778330+0000","last_peered":"2024-03-14T09:04:09.778330+0000","last_clean":"2024-03-14T09:04:09.778330+0000","last_became_active":"2024-03-14T09:03:57.187221+0000","last_became_peered":"2024-03-14T09:03:57.187221+0000","last_unstale":"2024-03-14T09:04:09.778330+0000","last_undegraded":"2024-03-14T09:04:09.778330+0000","last_fullsized":"2024-03-14T09:04:09.778330+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:04:42.577661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"69'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772415+0000","last_change":"2024-03-14T09:03:56.770808+0000","last_active":"2024-03-14T09:04:09.772415+0000","last_peered":"2024-03-14T09:04:09.772415+0000","last_clean":"2024-03-14T09:04:09.772415+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:04:09.772415+0000","last_undegraded":"2024-03-14T09:04:09.772415+0000","last_fullsized":"2024-03-14T09:04:09.772415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:50:29.358695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400943+0000","last_change":"2024-03-14T09:04:09.790683+0000","last_active":"2024-03-14T09:04:11.400943+0000","last_peered":"2024-03-14T09:04:11.400943+0000","last_clean":"2024-03-14T09:04:11.400943+0000","last_became_active":"2024-03-14T09:04:09.790387+0000","last_became_peered":"2024-03-14T09:04:09.790387+0000","last_unstale":"2024-03-14T09:04:11.400943+0000","last_undegraded":"2024-03-14T09:04:11.400943+0000","last_fullsized":"2024-03-14T09:04:11.400943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:55.431455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402135+0000","last_change":"2024-03-14T09:04:09.372636+0000","last_active":"2024-03-14T09:04:11.402135+0000","last_peered":"2024-03-14T09:04:11.402135+0000","last_clean":"2024-03-14T09:04:11.402135+0000","last_became_active":"2024-03-14T09:04:09.371868+0000","last_became_peered":"2024-03-14T09:04:09.371868+0000","last_unstale":"2024-03-14T09:04:11.402135+0000","last_undegraded":"2024-03-14T09:04:11.402135+0000","last_fullsized":"2024-03-14T09:04:11.402135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:39:38.539159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780538+0000","last_change":"2024-03-14T09:03:57.182905+0000","last_active":"2024-03-14T09:04:09.780538+0000","last_peered":"2024-03-14T09:04:09.780538+0000","last_clean":"2024-03-14T09:04:09.780538+0000","last_became_active":"2024-03-14T09:03:57.182631+0000","last_became_peered":"2024-03-14T09:03:57.182631+0000","last_unstale":"2024-03-14T09:04:09.780538+0000","last_undegraded":"2024-03-14T09:04:09.780538+0000","last_fullsized":"2024-03-14T09:04:09.780538+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:05.353470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779835+0000","last_change":"2024-03-14T09:03:56.770834+0000","last_active":"2024-03-14T09:04:09.779835+0000","last_peered":"2024-03-14T09:04:09.779835+0000","last_clean":"2024-03-14T09:04:09.779835+0000","last_became_active":"2024-03-14T09:03:56.770551+0000","last_became_peered":"2024-03-14T09:03:56.770551+0000","last_unstale":"2024-03-14T09:04:09.779835+0000","last_undegraded":"2024-03-14T09:04:09.779835+0000","last_fullsized":"2024-03-14T09:04:09.779835+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401250+0000","last_change":"2024-03-14T09:04:09.801869+0000","last_active":"2024-03-14T09:04:11.401250+0000","last_peered":"2024-03-14T09:04:11.401250+0000","last_clean":"2024-03-14T09:04:11.401250+0000","last_became_active":"2024-03-14T09:04:09.801150+0000","last_became_peered":"2024-03-14T09:04:09.801150+0000","last_unstale":"2024-03-14T09:04:11.401250+0000","last_undegraded":"2024-03-14T09:04:11.401250+0000","last_fullsized":"2024-03-14T09:04:11.401250+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:08:14.657757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401261+0000","last_change":"2024-03-14T09:04:09.366718+0000","last_active":"2024-03-14T09:04:11.401261+0000","last_peered":"2024-03-14T09:04:11.401261+0000","last_clean":"2024-03-14T09:04:11.401261+0000","last_became_active":"2024-03-14T09:04:09.366449+0000","last_became_peered":"2024-03-14T09:04:09.366449+0000","last_unstale":"2024-03-14T09:04:11.401261+0000","last_undegraded":"2024-03-14T09:04:11.401261+0000","last_fullsized":"2024-03-14T09:04:11.401261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:32:01.109463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779396+0000","last_change":"2024-03-14T09:03:57.180017+0000","last_active":"2024-03-14T09:04:09.779396+0000","last_peered":"2024-03-14T09:04:09.779396+0000","last_clean":"2024-03-14T09:04:09.779396+0000","last_became_active":"2024-03-14T09:03:57.179589+0000","last_became_peered":"2024-03-14T09:03:57.179589+0000","last_unstale":"2024-03-14T09:04:09.779396+0000","last_undegraded":"2024-03-14T09:04:09.779396+0000","last_fullsized":"2024-03-14T09:04:09.779396+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.774987+0000","last_change":"2024-03-14T09:03:56.769821+0000","last_active":"2024-03-14T09:04:09.774987+0000","last_peered":"2024-03-14T09:04:09.774987+0000","last_clean":"2024-03-14T09:04:09.774987+0000","last_became_active":"2024-03-14T09:03:56.769597+0000","last_became_peered":"2024-03-14T09:03:56.769597+0000","last_unstale":"2024-03-14T09:04:09.774987+0000","last_undegraded":"2024-03-14T09:04:09.774987+0000","last_fullsized":"2024-03-14T09:04:09.774987+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:01.018014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402518+0000","last_change":"2024-03-14T09:04:09.800352+0000","last_active":"2024-03-14T09:04:11.402518+0000","last_peered":"2024-03-14T09:04:11.402518+0000","last_clean":"2024-03-14T09:04:11.402518+0000","last_became_active":"2024-03-14T09:04:09.800092+0000","last_became_peered":"2024-03-14T09:04:09.800092+0000","last_unstale":"2024-03-14T09:04:11.402518+0000","last_undegraded":"2024-03-14T09:04:11.402518+0000","last_fullsized":"2024-03-14T09:04:11.402518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:49.819393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783445+0000","last_change":"2024-03-14T09:04:09.382043+0000","last_active":"2024-03-14T09:04:10.783445+0000","last_peered":"2024-03-14T09:04:10.783445+0000","last_clean":"2024-03-14T09:04:10.783445+0000","last_became_active":"2024-03-14T09:04:09.381757+0000","last_became_peered":"2024-03-14T09:04:09.381757+0000","last_unstale":"2024-03-14T09:04:10.783445+0000","last_undegraded":"2024-03-14T09:04:10.783445+0000","last_fullsized":"2024-03-14T09:04:10.783445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:28:27.468551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.796062+0000","last_change":"2024-03-14T09:03:57.187274+0000","last_active":"2024-03-14T09:04:09.796062+0000","last_peered":"2024-03-14T09:04:09.796062+0000","last_clean":"2024-03-14T09:04:09.796062+0000","last_became_active":"2024-03-14T09:03:57.187035+0000","last_became_peered":"2024-03-14T09:03:57.187035+0000","last_unstale":"2024-03-14T09:04:09.796062+0000","last_undegraded":"2024-03-14T09:04:09.796062+0000","last_fullsized":"2024-03-14T09:04:09.796062+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:52:31.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.772554+0000","last_change":"2024-03-14T09:03:56.772665+0000","last_active":"2024-03-14T09:04:09.772554+0000","last_peered":"2024-03-14T09:04:09.772554+0000","last_clean":"2024-03-14T09:04:09.772554+0000","last_became_active":"2024-03-14T09:03:56.772464+0000","last_became_peered":"2024-03-14T09:03:56.772464+0000","last_unstale":"2024-03-14T09:04:09.772554+0000","last_undegraded":"2024-03-14T09:04:09.772554+0000","last_fullsized":"2024-03-14T09:04:09.772554+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:06:22.876530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783733+0000","last_change":"2024-03-14T09:04:09.785512+0000","last_active":"2024-03-14T09:04:10.783733+0000","last_peered":"2024-03-14T09:04:10.783733+0000","last_clean":"2024-03-14T09:04:10.783733+0000","last_became_active":"2024-03-14T09:04:09.785267+0000","last_became_peered":"2024-03-14T09:04:09.785267+0000","last_unstale":"2024-03-14T09:04:10.783733+0000","last_undegraded":"2024-03-14T09:04:10.783733+0000","last_fullsized":"2024-03-14T09:04:10.783733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:54.708813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.399916+0000","last_change":"2024-03-14T09:04:09.380582+0000","last_active":"2024-03-14T09:04:11.399916+0000","last_peered":"2024-03-14T09:04:11.399916+0000","last_clean":"2024-03-14T09:04:11.399916+0000","last_became_active":"2024-03-14T09:04:09.380312+0000","last_became_peered":"2024-03-14T09:04:09.380312+0000","last_unstale":"2024-03-14T09:04:11.399916+0000","last_undegraded":"2024-03-14T09:04:11.399916+0000","last_fullsized":"2024-03-14T09:04:11.399916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:22:33.894772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.785884+0000","last_change":"2024-03-14T09:03:57.191774+0000","last_active":"2024-03-14T09:04:09.785884+0000","last_peered":"2024-03-14T09:04:09.785884+0000","last_clean":"2024-03-14T09:04:09.785884+0000","last_became_active":"2024-03-14T09:03:57.191391+0000","last_became_peered":"2024-03-14T09:03:57.191391+0000","last_unstale":"2024-03-14T09:04:09.785884+0000","last_undegraded":"2024-03-14T09:04:09.785884+0000","last_fullsized":"2024-03-14T09:04:09.785884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:51:22.069879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779049+0000","last_change":"2024-03-14T09:03:56.759918+0000","last_active":"2024-03-14T09:04:09.779049+0000","last_peered":"2024-03-14T09:04:09.779049+0000","last_clean":"2024-03-14T09:04:09.779049+0000","last_became_active":"2024-03-14T09:03:56.759495+0000","last_became_peered":"2024-03-14T09:03:56.759495+0000","last_unstale":"2024-03-14T09:04:09.779049+0000","last_undegraded":"2024-03-14T09:04:09.779049+0000","last_fullsized":"2024-03-14T09:04:09.779049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:38:04.348438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400229+0000","last_change":"2024-03-14T09:04:09.784010+0000","last_active":"2024-03-14T09:04:11.400229+0000","last_peered":"2024-03-14T09:04:11.400229+0000","last_clean":"2024-03-14T09:04:11.400229+0000","last_became_active":"2024-03-14T09:04:09.783599+0000","last_became_peered":"2024-03-14T09:04:09.783599+0000","last_unstale":"2024-03-14T09:04:11.400229+0000","last_undegraded":"2024-03-14T09:04:11.400229+0000","last_fullsized":"2024-03-14T09:04:11.400229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:45:35.790856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400280+0000","last_change":"2024-03-14T09:04:09.373852+0000","last_active":"2024-03-14T09:04:11.400280+0000","last_peered":"2024-03-14T09:04:11.400280+0000","last_clean":"2024-03-14T09:04:11.400280+0000","last_became_active":"2024-03-14T09:04:09.373535+0000","last_became_peered":"2024-03-14T09:04:09.373535+0000","last_unstale":"2024-03-14T09:04:11.400280+0000","last_undegraded":"2024-03-14T09:04:11.400280+0000","last_fullsized":"2024-03-14T09:04:11.400280+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:28:13.620690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784382+0000","last_change":"2024-03-14T09:03:57.181268+0000","last_active":"2024-03-14T09:04:09.784382+0000","last_peered":"2024-03-14T09:04:09.784382+0000","last_clean":"2024-03-14T09:04:09.784382+0000","last_became_active":"2024-03-14T09:03:57.180967+0000","last_became_peered":"2024-03-14T09:03:57.180967+0000","last_unstale":"2024-03-14T09:04:09.784382+0000","last_undegraded":"2024-03-14T09:04:09.784382+0000","last_fullsized":"2024-03-14T09:04:09.784382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:45.987655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.786001+0000","last_change":"2024-03-14T09:03:56.768053+0000","last_active":"2024-03-14T09:04:09.786001+0000","last_peered":"2024-03-14T09:04:09.786001+0000","last_clean":"2024-03-14T09:04:09.786001+0000","last_became_active":"2024-03-14T09:03:56.767780+0000","last_became_peered":"2024-03-14T09:03:56.767780+0000","last_unstale":"2024-03-14T09:04:09.786001+0000","last_undegraded":"2024-03-14T09:04:09.786001+0000","last_fullsized":"2024-03-14T09:04:09.786001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:00:23.189832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784387+0000","last_change":"2024-03-14T09:04:09.794287+0000","last_active":"2024-03-14T09:04:10.784387+0000","last_peered":"2024-03-14T09:04:10.784387+0000","last_clean":"2024-03-14T09:04:10.784387+0000","last_became_active":"2024-03-14T09:04:09.794047+0000","last_became_peered":"2024-03-14T09:04:09.794047+0000","last_unstale":"2024-03-14T09:04:10.784387+0000","last_undegraded":"2024-03-14T09:04:10.784387+0000","last_fullsized":"2024-03-14T09:04:10.784387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:49.393489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401086+0000","last_change":"2024-03-14T09:04:09.380471+0000","last_active":"2024-03-14T09:04:11.401086+0000","last_peered":"2024-03-14T09:04:11.401086+0000","last_clean":"2024-03-14T09:04:11.401086+0000","last_became_active":"2024-03-14T09:04:09.380219+0000","last_became_peered":"2024-03-14T09:04:09.380219+0000","last_unstale":"2024-03-14T09:04:11.401086+0000","last_undegraded":"2024-03-14T09:04:11.401086+0000","last_fullsized":"2024-03-14T09:04:11.401086+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:50.175976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780176+0000","last_change":"2024-03-14T09:03:57.177141+0000","last_active":"2024-03-14T09:04:09.780176+0000","last_peered":"2024-03-14T09:04:09.780176+0000","last_clean":"2024-03-14T09:04:09.780176+0000","last_became_active":"2024-03-14T09:03:57.176843+0000","last_became_peered":"2024-03-14T09:03:57.176843+0000","last_unstale":"2024-03-14T09:04:09.780176+0000","last_undegraded":"2024-03-14T09:04:09.780176+0000","last_fullsized":"2024-03-14T09:04:09.780176+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:57.185084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780207+0000","last_change":"2024-03-14T09:03:56.764413+0000","last_active":"2024-03-14T09:04:09.780207+0000","last_peered":"2024-03-14T09:04:09.780207+0000","last_clean":"2024-03-14T09:04:09.780207+0000","last_became_active":"2024-03-14T09:03:56.764156+0000","last_became_peered":"2024-03-14T09:03:56.764156+0000","last_unstale":"2024-03-14T09:04:09.780207+0000","last_undegraded":"2024-03-14T09:04:09.780207+0000","last_fullsized":"2024-03-14T09:04:09.780207+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:21:29.624660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784985+0000","last_change":"2024-03-14T09:04:09.799707+0000","last_active":"2024-03-14T09:04:10.784985+0000","last_peered":"2024-03-14T09:04:10.784985+0000","last_clean":"2024-03-14T09:04:10.784985+0000","last_became_active":"2024-03-14T09:04:09.799456+0000","last_became_peered":"2024-03-14T09:04:09.799456+0000","last_unstale":"2024-03-14T09:04:10.784985+0000","last_undegraded":"2024-03-14T09:04:10.784985+0000","last_fullsized":"2024-03-14T09:04:10.784985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:54:51.043997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400971+0000","last_change":"2024-03-14T09:04:09.379642+0000","last_active":"2024-03-14T09:04:11.400971+0000","last_peered":"2024-03-14T09:04:11.400971+0000","last_clean":"2024-03-14T09:04:11.400971+0000","last_became_active":"2024-03-14T09:04:09.379275+0000","last_became_peered":"2024-03-14T09:04:09.379275+0000","last_unstale":"2024-03-14T09:04:11.400971+0000","last_undegraded":"2024-03-14T09:04:11.400971+0000","last_fullsized":"2024-03-14T09:04:11.400971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:15.521387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784143+0000","last_change":"2024-03-14T09:03:57.187334+0000","last_active":"2024-03-14T09:04:09.784143+0000","last_peered":"2024-03-14T09:04:09.784143+0000","last_clean":"2024-03-14T09:04:09.784143+0000","last_became_active":"2024-03-14T09:03:57.187054+0000","last_became_peered":"2024-03-14T09:03:57.187054+0000","last_unstale":"2024-03-14T09:04:09.784143+0000","last_undegraded":"2024-03-14T09:04:09.784143+0000","last_fullsized":"2024-03-14T09:04:09.784143+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:19:46.177123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.776720+0000","last_change":"2024-03-14T09:03:56.759664+0000","last_active":"2024-03-14T09:04:09.776720+0000","last_peered":"2024-03-14T09:04:09.776720+0000","last_clean":"2024-03-14T09:04:09.776720+0000","last_became_active":"2024-03-14T09:03:56.759378+0000","last_became_peered":"2024-03-14T09:03:56.759378+0000","last_unstale":"2024-03-14T09:04:09.776720+0000","last_undegraded":"2024-03-14T09:04:09.776720+0000","last_fullsized":"2024-03-14T09:04:09.776720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:34:03.113813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400635+0000","last_change":"2024-03-14T09:04:09.787270+0000","last_active":"2024-03-14T09:04:11.400635+0000","last_peered":"2024-03-14T09:04:11.400635+0000","last_clean":"2024-03-14T09:04:11.400635+0000","last_became_active":"2024-03-14T09:04:09.787007+0000","last_became_peered":"2024-03-14T09:04:09.787007+0000","last_unstale":"2024-03-14T09:04:11.400635+0000","last_undegraded":"2024-03-14T09:04:11.400635+0000","last_fullsized":"2024-03-14T09:04:11.400635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:44:16.528447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401489+0000","last_change":"2024-03-14T09:04:09.367225+0000","last_active":"2024-03-14T09:04:11.401489+0000","last_peered":"2024-03-14T09:04:11.401489+0000","last_clean":"2024-03-14T09:04:11.401489+0000","last_became_active":"2024-03-14T09:04:09.366961+0000","last_became_peered":"2024-03-14T09:04:09.366961+0000","last_unstale":"2024-03-14T09:04:11.401489+0000","last_undegraded":"2024-03-14T09:04:11.401489+0000","last_fullsized":"2024-03-14T09:04:11.401489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:42:00.161664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.795987+0000","last_change":"2024-03-14T09:03:57.177315+0000","last_active":"2024-03-14T09:04:09.795987+0000","last_peered":"2024-03-14T09:04:09.795987+0000","last_clean":"2024-03-14T09:04:09.795987+0000","last_became_active":"2024-03-14T09:03:57.177045+0000","last_became_peered":"2024-03-14T09:03:57.177045+0000","last_unstale":"2024-03-14T09:04:09.795987+0000","last_undegraded":"2024-03-14T09:04:09.795987+0000","last_fullsized":"2024-03-14T09:04:09.795987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:31.062131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"69'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.782430+0000","last_change":"2024-03-14T09:03:56.764563+0000","last_active":"2024-03-14T09:04:09.782430+0000","last_peered":"2024-03-14T09:04:09.782430+0000","last_clean":"2024-03-14T09:04:09.782430+0000","last_became_active":"2024-03-14T09:03:56.764299+0000","last_became_peered":"2024-03-14T09:03:56.764299+0000","last_unstale":"2024-03-14T09:04:09.782430+0000","last_undegraded":"2024-03-14T09:04:09.782430+0000","last_fullsized":"2024-03-14T09:04:09.782430+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:32:22.457392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.402011+0000","last_change":"2024-03-14T09:04:09.786880+0000","last_active":"2024-03-14T09:04:11.402011+0000","last_peered":"2024-03-14T09:04:11.402011+0000","last_clean":"2024-03-14T09:04:11.402011+0000","last_became_active":"2024-03-14T09:04:09.786581+0000","last_became_peered":"2024-03-14T09:04:09.786581+0000","last_unstale":"2024-03-14T09:04:11.402011+0000","last_undegraded":"2024-03-14T09:04:11.402011+0000","last_fullsized":"2024-03-14T09:04:11.402011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:43:14.433260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401213+0000","last_change":"2024-03-14T09:04:09.373374+0000","last_active":"2024-03-14T09:04:11.401213+0000","last_peered":"2024-03-14T09:04:11.401213+0000","last_clean":"2024-03-14T09:04:11.401213+0000","last_became_active":"2024-03-14T09:04:09.372987+0000","last_became_peered":"2024-03-14T09:04:09.372987+0000","last_unstale":"2024-03-14T09:04:11.401213+0000","last_undegraded":"2024-03-14T09:04:11.401213+0000","last_fullsized":"2024-03-14T09:04:11.401213+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:16:54.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.780681+0000","last_change":"2024-03-14T09:03:57.182451+0000","last_active":"2024-03-14T09:04:09.780681+0000","last_peered":"2024-03-14T09:04:09.780681+0000","last_clean":"2024-03-14T09:04:09.780681+0000","last_became_active":"2024-03-14T09:03:57.182007+0000","last_became_peered":"2024-03-14T09:03:57.182007+0000","last_unstale":"2024-03-14T09:04:09.780681+0000","last_undegraded":"2024-03-14T09:04:09.780681+0000","last_fullsized":"2024-03-14T09:04:09.780681+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:47:08.734234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"70'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781298+0000","last_change":"2024-03-14T09:03:56.768285+0000","last_active":"2024-03-14T09:04:09.781298+0000","last_peered":"2024-03-14T09:04:09.781298+0000","last_clean":"2024-03-14T09:04:09.781298+0000","last_became_active":"2024-03-14T09:03:56.768060+0000","last_became_peered":"2024-03-14T09:03:56.768060+0000","last_unstale":"2024-03-14T09:04:09.781298+0000","last_undegraded":"2024-03-14T09:04:09.781298+0000","last_fullsized":"2024-03-14T09:04:09.781298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T20:31:56.180051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401564+0000","last_change":"2024-03-14T09:04:09.788779+0000","last_active":"2024-03-14T09:04:11.401564+0000","last_peered":"2024-03-14T09:04:11.401564+0000","last_clean":"2024-03-14T09:04:11.401564+0000","last_became_active":"2024-03-14T09:04:09.788165+0000","last_became_peered":"2024-03-14T09:04:09.788165+0000","last_unstale":"2024-03-14T09:04:11.401564+0000","last_undegraded":"2024-03-14T09:04:11.401564+0000","last_fullsized":"2024-03-14T09:04:11.401564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:05:38.870340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"73'2","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401219+0000","last_change":"2024-03-14T09:04:09.372263+0000","last_active":"2024-03-14T09:04:11.401219+0000","last_peered":"2024-03-14T09:04:11.401219+0000","last_clean":"2024-03-14T09:04:11.401219+0000","last_became_active":"2024-03-14T09:04:09.371457+0000","last_became_peered":"2024-03-14T09:04:09.371457+0000","last_unstale":"2024-03-14T09:04:11.401219+0000","last_undegraded":"2024-03-14T09:04:11.401219+0000","last_fullsized":"2024-03-14T09:04:11.401219+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:17:28.950898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777523+0000","last_change":"2024-03-14T09:03:57.188009+0000","last_active":"2024-03-14T09:04:09.777523+0000","last_peered":"2024-03-14T09:04:09.777523+0000","last_clean":"2024-03-14T09:04:09.777523+0000","last_became_active":"2024-03-14T09:03:57.187745+0000","last_became_peered":"2024-03-14T09:03:57.187745+0000","last_unstale":"2024-03-14T09:04:09.777523+0000","last_undegraded":"2024-03-14T09:04:09.777523+0000","last_fullsized":"2024-03-14T09:04:09.777523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:06:01.815544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.779017+0000","last_change":"2024-03-14T09:03:56.764742+0000","last_active":"2024-03-14T09:04:09.779017+0000","last_peered":"2024-03-14T09:04:09.779017+0000","last_clean":"2024-03-14T09:04:09.779017+0000","last_became_active":"2024-03-14T09:03:56.764467+0000","last_became_peered":"2024-03-14T09:03:56.764467+0000","last_unstale":"2024-03-14T09:04:09.779017+0000","last_undegraded":"2024-03-14T09:04:09.779017+0000","last_fullsized":"2024-03-14T09:04:09.779017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T09:11:25.434069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400386+0000","last_change":"2024-03-14T09:04:09.785569+0000","last_active":"2024-03-14T09:04:11.400386+0000","last_peered":"2024-03-14T09:04:11.400386+0000","last_clean":"2024-03-14T09:04:11.400386+0000","last_became_active":"2024-03-14T09:04:09.785326+0000","last_became_peered":"2024-03-14T09:04:09.785326+0000","last_unstale":"2024-03-14T09:04:11.400386+0000","last_undegraded":"2024-03-14T09:04:11.400386+0000","last_fullsized":"2024-03-14T09:04:11.400386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:06:15.309710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"74'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.693625+0000","last_change":"2024-03-14T09:04:09.382499+0000","last_active":"2024-03-14T09:04:11.693625+0000","last_peered":"2024-03-14T09:04:11.693625+0000","last_clean":"2024-03-14T09:04:11.693625+0000","last_became_active":"2024-03-14T09:04:09.381995+0000","last_became_peered":"2024-03-14T09:04:09.381995+0000","last_unstale":"2024-03-14T09:04:11.693625+0000","last_undegraded":"2024-03-14T09:04:11.693625+0000","last_fullsized":"2024-03-14T09:04:11.693625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:42.799033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.795972+0000","last_change":"2024-03-14T09:03:57.189309+0000","last_active":"2024-03-14T09:04:09.795972+0000","last_peered":"2024-03-14T09:04:09.795972+0000","last_clean":"2024-03-14T09:04:09.795972+0000","last_became_active":"2024-03-14T09:03:57.189127+0000","last_became_peered":"2024-03-14T09:03:57.189127+0000","last_unstale":"2024-03-14T09:04:09.795972+0000","last_undegraded":"2024-03-14T09:04:09.795972+0000","last_fullsized":"2024-03-14T09:04:09.795972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:42.775808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777807+0000","last_change":"2024-03-14T09:03:56.770115+0000","last_active":"2024-03-14T09:04:09.777807+0000","last_peered":"2024-03-14T09:04:09.777807+0000","last_clean":"2024-03-14T09:04:09.777807+0000","last_became_active":"2024-03-14T09:03:56.769621+0000","last_became_peered":"2024-03-14T09:03:56.769621+0000","last_unstale":"2024-03-14T09:04:09.777807+0000","last_undegraded":"2024-03-14T09:04:09.777807+0000","last_fullsized":"2024-03-14T09:04:09.777807+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:43:21.424243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783829+0000","last_change":"2024-03-14T09:04:09.796371+0000","last_active":"2024-03-14T09:04:10.783829+0000","last_peered":"2024-03-14T09:04:10.783829+0000","last_clean":"2024-03-14T09:04:10.783829+0000","last_became_active":"2024-03-14T09:04:09.796095+0000","last_became_peered":"2024-03-14T09:04:09.796095+0000","last_unstale":"2024-03-14T09:04:10.783829+0000","last_undegraded":"2024-03-14T09:04:10.783829+0000","last_fullsized":"2024-03-14T09:04:10.783829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:57:35.357930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783170+0000","last_change":"2024-03-14T09:04:09.379745+0000","last_active":"2024-03-14T09:04:10.783170+0000","last_peered":"2024-03-14T09:04:10.783170+0000","last_clean":"2024-03-14T09:04:10.783170+0000","last_became_active":"2024-03-14T09:04:09.379484+0000","last_became_peered":"2024-03-14T09:04:09.379484+0000","last_unstale":"2024-03-14T09:04:10.783170+0000","last_undegraded":"2024-03-14T09:04:10.783170+0000","last_fullsized":"2024-03-14T09:04:10.783170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:22:42.799033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.784375+0000","last_change":"2024-03-14T09:03:57.180976+0000","last_active":"2024-03-14T09:04:09.784375+0000","last_peered":"2024-03-14T09:04:09.784375+0000","last_clean":"2024-03-14T09:04:09.784375+0000","last_became_active":"2024-03-14T09:03:57.180689+0000","last_became_peered":"2024-03-14T09:03:57.180689+0000","last_unstale":"2024-03-14T09:04:09.784375+0000","last_undegraded":"2024-03-14T09:04:09.784375+0000","last_fullsized":"2024-03-14T09:04:09.784375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:50:02.023046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773163+0000","last_change":"2024-03-14T09:03:56.770777+0000","last_active":"2024-03-14T09:04:09.773163+0000","last_peered":"2024-03-14T09:04:09.773163+0000","last_clean":"2024-03-14T09:04:09.773163+0000","last_became_active":"2024-03-14T09:03:56.770466+0000","last_became_peered":"2024-03-14T09:03:56.770466+0000","last_unstale":"2024-03-14T09:04:09.773163+0000","last_undegraded":"2024-03-14T09:04:09.773163+0000","last_fullsized":"2024-03-14T09:04:09.773163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:37:41.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401051+0000","last_change":"2024-03-14T09:04:09.787284+0000","last_active":"2024-03-14T09:04:11.401051+0000","last_peered":"2024-03-14T09:04:11.401051+0000","last_clean":"2024-03-14T09:04:11.401051+0000","last_became_active":"2024-03-14T09:04:09.787007+0000","last_became_peered":"2024-03-14T09:04:09.787007+0000","last_unstale":"2024-03-14T09:04:11.401051+0000","last_undegraded":"2024-03-14T09:04:11.401051+0000","last_fullsized":"2024-03-14T09:04:11.401051+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:05:00.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783909+0000","last_change":"2024-03-14T09:04:09.377195+0000","last_active":"2024-03-14T09:04:10.783909+0000","last_peered":"2024-03-14T09:04:10.783909+0000","last_clean":"2024-03-14T09:04:10.783909+0000","last_became_active":"2024-03-14T09:04:09.376963+0000","last_became_peered":"2024-03-14T09:04:09.376963+0000","last_unstale":"2024-03-14T09:04:10.783909+0000","last_undegraded":"2024-03-14T09:04:10.783909+0000","last_fullsized":"2024-03-14T09:04:10.783909+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:47:06.421840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.773018+0000","last_change":"2024-03-14T09:03:57.184709+0000","last_active":"2024-03-14T09:04:09.773018+0000","last_peered":"2024-03-14T09:04:09.773018+0000","last_clean":"2024-03-14T09:04:09.773018+0000","last_became_active":"2024-03-14T09:03:57.184380+0000","last_became_peered":"2024-03-14T09:03:57.184380+0000","last_unstale":"2024-03-14T09:04:09.773018+0000","last_undegraded":"2024-03-14T09:04:09.773018+0000","last_fullsized":"2024-03-14T09:04:09.773018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793909+0000","last_change":"2024-03-14T09:03:56.769939+0000","last_active":"2024-03-14T09:04:09.793909+0000","last_peered":"2024-03-14T09:04:09.793909+0000","last_clean":"2024-03-14T09:04:09.793909+0000","last_became_active":"2024-03-14T09:03:56.769670+0000","last_became_peered":"2024-03-14T09:03:56.769670+0000","last_unstale":"2024-03-14T09:04:09.793909+0000","last_undegraded":"2024-03-14T09:04:09.793909+0000","last_fullsized":"2024-03-14T09:04:09.793909+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:40:50.421546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.784637+0000","last_change":"2024-03-14T09:04:09.796184+0000","last_active":"2024-03-14T09:04:10.784637+0000","last_peered":"2024-03-14T09:04:10.784637+0000","last_clean":"2024-03-14T09:04:10.784637+0000","last_became_active":"2024-03-14T09:04:09.795957+0000","last_became_peered":"2024-03-14T09:04:09.795957+0000","last_unstale":"2024-03-14T09:04:10.784637+0000","last_undegraded":"2024-03-14T09:04:10.784637+0000","last_fullsized":"2024-03-14T09:04:10.784637+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T13:09:43.674655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401111+0000","last_change":"2024-03-14T09:04:09.369844+0000","last_active":"2024-03-14T09:04:11.401111+0000","last_peered":"2024-03-14T09:04:11.401111+0000","last_clean":"2024-03-14T09:04:11.401111+0000","last_became_active":"2024-03-14T09:04:09.369584+0000","last_became_peered":"2024-03-14T09:04:09.369584+0000","last_unstale":"2024-03-14T09:04:11.401111+0000","last_undegraded":"2024-03-14T09:04:11.401111+0000","last_fullsized":"2024-03-14T09:04:11.401111+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T17:55:48.812333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781410+0000","last_change":"2024-03-14T09:03:57.187259+0000","last_active":"2024-03-14T09:04:09.781410+0000","last_peered":"2024-03-14T09:04:09.781410+0000","last_clean":"2024-03-14T09:04:09.781410+0000","last_became_active":"2024-03-14T09:03:57.186972+0000","last_became_peered":"2024-03-14T09:03:57.186972+0000","last_unstale":"2024-03-14T09:04:09.781410+0000","last_undegraded":"2024-03-14T09:04:09.781410+0000","last_fullsized":"2024-03-14T09:04:09.781410+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T10:43:36.780964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777363+0000","last_change":"2024-03-14T09:03:56.771166+0000","last_active":"2024-03-14T09:04:09.777363+0000","last_peered":"2024-03-14T09:04:09.777363+0000","last_clean":"2024-03-14T09:04:09.777363+0000","last_became_active":"2024-03-14T09:03:56.769678+0000","last_became_peered":"2024-03-14T09:03:56.769678+0000","last_unstale":"2024-03-14T09:04:09.777363+0000","last_undegraded":"2024-03-14T09:04:09.777363+0000","last_fullsized":"2024-03-14T09:04:09.777363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:18:06.742278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.783402+0000","last_change":"2024-03-14T09:04:09.796461+0000","last_active":"2024-03-14T09:04:10.783402+0000","last_peered":"2024-03-14T09:04:10.783402+0000","last_clean":"2024-03-14T09:04:10.783402+0000","last_became_active":"2024-03-14T09:04:09.796193+0000","last_became_peered":"2024-03-14T09:04:09.796193+0000","last_unstale":"2024-03-14T09:04:10.783402+0000","last_undegraded":"2024-03-14T09:04:10.783402+0000","last_fullsized":"2024-03-14T09:04:10.783402+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:27:14.951521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.400276+0000","last_change":"2024-03-14T09:04:09.382491+0000","last_active":"2024-03-14T09:04:11.400276+0000","last_peered":"2024-03-14T09:04:11.400276+0000","last_clean":"2024-03-14T09:04:11.400276+0000","last_became_active":"2024-03-14T09:04:09.382211+0000","last_became_peered":"2024-03-14T09:04:09.382211+0000","last_unstale":"2024-03-14T09:04:11.400276+0000","last_undegraded":"2024-03-14T09:04:11.400276+0000","last_fullsized":"2024-03-14T09:04:11.400276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T16:51:49.293407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.777276+0000","last_change":"2024-03-14T09:03:57.181824+0000","last_active":"2024-03-14T09:04:09.777276+0000","last_peered":"2024-03-14T09:04:09.777276+0000","last_clean":"2024-03-14T09:04:09.777276+0000","last_became_active":"2024-03-14T09:03:57.181573+0000","last_became_peered":"2024-03-14T09:03:57.181573+0000","last_unstale":"2024-03-14T09:04:09.777276+0000","last_undegraded":"2024-03-14T09:04:09.777276+0000","last_fullsized":"2024-03-14T09:04:09.777276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T11:52:31.823414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.793755+0000","last_change":"2024-03-14T09:03:56.770803+0000","last_active":"2024-03-14T09:04:09.793755+0000","last_peered":"2024-03-14T09:04:09.793755+0000","last_clean":"2024-03-14T09:04:09.793755+0000","last_became_active":"2024-03-14T09:03:56.770311+0000","last_became_peered":"2024-03-14T09:03:56.770311+0000","last_unstale":"2024-03-14T09:04:09.793755+0000","last_undegraded":"2024-03-14T09:04:09.793755+0000","last_fullsized":"2024-03-14T09:04:09.793755+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T15:58:24.309356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:10.785884+0000","last_change":"2024-03-14T09:04:09.793536+0000","last_active":"2024-03-14T09:04:10.785884+0000","last_peered":"2024-03-14T09:04:10.785884+0000","last_clean":"2024-03-14T09:04:10.785884+0000","last_became_active":"2024-03-14T09:04:09.793283+0000","last_became_peered":"2024-03-14T09:04:09.793283+0000","last_unstale":"2024-03-14T09:04:10.785884+0000","last_undegraded":"2024-03-14T09:04:10.785884+0000","last_fullsized":"2024-03-14T09:04:10.785884+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:09.354407+0000","last_clean_scrub_stamp":"2024-03-14T09:04:09.354407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:51:33.968294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"73'1","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-14T09:04:11.401320+0000","last_change":"2024-03-14T09:04:09.380665+0000","last_active":"2024-03-14T09:04:11.401320+0000","last_peered":"2024-03-14T09:04:11.401320+0000","last_clean":"2024-03-14T09:04:11.401320+0000","last_became_active":"2024-03-14T09:04:09.380369+0000","last_became_peered":"2024-03-14T09:04:09.380369+0000","last_unstale":"2024-03-14T09:04:11.401320+0000","last_undegraded":"2024-03-14T09:04:11.401320+0000","last_fullsized":"2024-03-14T09:04:11.401320+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:04:08.347348+0000","last_clean_scrub_stamp":"2024-03-14T09:04:08.347348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:33:47.940938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781811+0000","last_change":"2024-03-14T09:03:57.186670+0000","last_active":"2024-03-14T09:04:09.781811+0000","last_peered":"2024-03-14T09:04:09.781811+0000","last_clean":"2024-03-14T09:04:09.781811+0000","last_became_active":"2024-03-14T09:03:57.186398+0000","last_became_peered":"2024-03-14T09:03:57.186398+0000","last_unstale":"2024-03-14T09:04:09.781811+0000","last_undegraded":"2024-03-14T09:04:09.781811+0000","last_fullsized":"2024-03-14T09:04:09.781811+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:56.741883+0000","last_clean_scrub_stamp":"2024-03-14T09:03:56.741883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T12:48:15.878140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"69'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781053+0000","last_change":"2024-03-14T09:03:56.766595+0000","last_active":"2024-03-14T09:04:09.781053+0000","last_peered":"2024-03-14T09:04:09.781053+0000","last_clean":"2024-03-14T09:04:09.781053+0000","last_became_active":"2024-03-14T09:03:56.766269+0000","last_became_peered":"2024-03-14T09:03:56.766269+0000","last_unstale":"2024-03-14T09:04:09.781053+0000","last_undegraded":"2024-03-14T09:04:09.781053+0000","last_fullsized":"2024-03-14T09:04:09.781053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:03:55.736850+0000","last_clean_scrub_stamp":"2024-03-14T09:03:55.736850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T14:14:54.254524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-14T09:04:09.781780+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:09.781780+0000","last_peered":"2024-03-14T09:04:09.781780+0000","last_clean":"2024-03-14T09:04:09.781780+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:09.781780+0000","last_undegraded":"2024-03-14T09:04:09.781780+0000","last_fullsized":"2024-03-14T09:04:09.781780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":3873,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343340,"statfs":{"total":96636764160,"available":96607580160,"internally_reserved":0,"allocated":1515520,"data_stored":994685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.497},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54900000000000004,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.85699999999999998,"15min":0.91300000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.66600000000000004,"5min":6.5,"15min":6.5},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.53100000000000003,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.432},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69099999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.36899999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.64100000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72199999999999998,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.68100000000000005,"5min":0.878,"15min":0.878},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61099999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":405481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.627},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69999999999999996,"5min":0.87,"15min":0.87},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68700000000000006,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81299999999999994,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.437}]},{"osd":7,"last update":"Thu Mar 14 09:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.64500000000000002,"15min":0.64900000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91400000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80200000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.52800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":404329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33900000000000002,"15min":0.34000000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.45700000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.34000000000000002,"15min":0.34200000000000003},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.28199999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39400000000000002,"15min":0.39500000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.497,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.377}]},{"osd":4,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.626}]},{"osd":6,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:03:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1511424,"data_stored":995119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.71599999999999997,"15min":0.78200000000000003},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.41799999999999998,"5min":7.383,"15min":7.383},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.44500000000000001,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52500000000000002,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.79200000000000004,"15min":0.86599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.51800000000000002,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.81699999999999995,"15min":0.89200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.48899999999999999,"5min":7.944,"15min":7.944},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.77200000000000002,"15min":0.83599999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0349999999999999,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.312}]},{"osd":4,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77300000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72199999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.94699999999999995,"15min":1.0029999999999999},"min":{"1min":0.41099999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.014,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.98399999999999999,"15min":1.0489999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0509999999999999,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:03:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80300000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.36499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":404751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.34200000000000003,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40799999999999997,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.40400000000000003,"5min":1.111,"15min":1.111},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.46400000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.41599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.375}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.67300000000000004,"15min":0.72999999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.41199999999999998,"5min":7.5,"15min":7.5},"last":0.255},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.71599999999999997,"15min":0.77000000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.48699999999999999,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81999999999999995,"5min":1.151,"15min":1.151},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.64500000000000002,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69999999999999996,"5min":1.179,"15min":1.179},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71799999999999997,"5min":0.751,"15min":0.751},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.75900000000000001,"5min":1.165,"15min":1.165},"last":0.628},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.64900000000000002,"15min":0.65800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83199999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":408796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.311,"15min":0.308},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"last":0.307},{"interface":"front","average":{"1min":0.30399999999999999,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.38300000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.39400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.42799999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.373,"15min":0.375},"min":{"1min":0.29099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.45400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.378},"min":{"1min":0.31,"5min":0.253,"15min":0.253},"max":{"1min":0.45800000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73799999999999999,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.498},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.435,"5min":0.375,"15min":0.375},"max":{"1min":0.80700000000000005,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.79400000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.75700000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:03:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76200000000000001,"5min":1.087,"15min":1.087},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":405823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.51100000000000001,"15min":0.51600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.68999999999999995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.755}]},{"osd":2,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.751,"15min":0.751},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71099999999999997,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64000000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.89100000000000001,"15min":0.94699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76000000000000001,"5min":7.702,"15min":7.702},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68600000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.71499999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:03:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.753,"15min":0.753},"last":0.624},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.61499999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.84899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":407302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73499999999999999,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.434}]},{"osd":1,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.67900000000000005,"5min":0.876,"15min":0.876},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68500000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53400000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.70399999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70499999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.624}]},{"osd":6,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.54000000000000004,"15min":0.53100000000000003},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.499},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:03:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.626,"15min":0.626},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79900000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:14.291 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-14T09:04:14.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mon 'allow r' osd 'allow rw pool=cephfs2_data' mds allow 2024-03-14T09:04:14.453 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.453+0000 7f40a7e14700 1 Processor -- start 2024-03-14T09:04:14.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 -- start start 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40a00c5d60 0x7f40a00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a00acd10 con 0x7f40a00702f0 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a0150110 con 0x7f40a00c5d60 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.454+0000 7f40a7e14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00ac7b0 con 0x7f40a00c4380 2024-03-14T09:04:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a53af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40a00c5d60 0x7f40a00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a53af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:47120/0 (socket says 172.21.15.154:47120) 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40a00c5d60 0x7f40a00c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:40594/0 (socket says 172.21.15.154:40594) 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a53af700 1 -- 172.21.15.154:0/1173237454 learned_addr learned my addr 172.21.15.154:0/1173237454 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a5bb0700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f40a00702f0 0x7f40a00c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:46558/0 (socket says 172.21.15.154:46558) 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 -- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 msgr2=0x7f40a014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 -- 172.21.15.154:0/1173237454 >> v1:172.21.15.191:6789/0 conn(0x7f40a00702f0 legacy=0x7f40a00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 -- 172.21.15.154:0/1173237454 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00ac9c0 con 0x7f40a00c5d60 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a53af700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a63b1700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00c5d60 0x7f40a00c61d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4098009d90 tx=0x7f409800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ff0b747c99beb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a4bae700 1 -- 172.21.15.154:0/1173237454 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800e070 con 0x7f40a00c5d60 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.455+0000 7f40a4bae700 1 -- 172.21.15.154:0/1173237454 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098003a20 con 0x7f40a00c5d60 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a4bae700 1 -- 172.21.15.154:0/1173237454 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098003dd0 con 0x7f40a00c5d60 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 -- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00c5d60 msgr2=0x7f40a00c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00c5d60 0x7f40a00c61d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4098009d90 tx=0x7f409800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 -- 172.21.15.154:0/1173237454 shutdown_connections 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00c5d60 0x7f40a00c61d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 --2- 172.21.15.154:0/1173237454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 -- 172.21.15.154:0/1173237454 >> 172.21.15.154:0/1173237454 conn(0x7f40a00a9bf0 msgr2=0x7f40a00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 -- 172.21.15.154:0/1173237454 shutdown_connections 2024-03-14T09:04:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.456+0000 7f40a7e14700 1 -- 172.21.15.154:0/1173237454 wait complete. 2024-03-14T09:04:14.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.457+0000 7f40a7e14700 1 Processor -- start 2024-03-14T09:04:14.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.457+0000 7f40a7e14700 1 -- start start 2024-03-14T09:04:14.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 0x7f40a0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.461 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a00c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0150ed0 con 0x7f40a00c5d60 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0152610 con 0x7f40a00702f0 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a7e14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0152030 con 0x7f40a00c4380 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a5bb0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 0x7f40a0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43608/0 (socket says 172.21.15.154:43608) 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 -- 172.21.15.154:0/4144084745 learned_addr learned my addr 172.21.15.154:0/4144084745 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a53af700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a00c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 -- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 msgr2=0x7f40a00c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a00c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 -- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 msgr2=0x7f40a0152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 0x7f40a0152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a63b1700 1 -- 172.21.15.154:0/4144084745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c009550 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.458+0000 7f40a5bb0700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 0x7f40a0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40a63b1700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f409800b680 tx=0x7f4098004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098009410 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098004580 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4098009a70 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098004930 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.459+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a01505e0 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.460+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a01527a0 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.460+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4098012430 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.460+0000 7f40967fc700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 0x7f408406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.461+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f409808a8f0 con 0x7f40a00c5d60 2024-03-14T09:04:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.461+0000 7f40a5bb0700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 0x7f408406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.461+0000 7f40a5bb0700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 0x7f408406cd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4090007c20 tx=0x7f4090007b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.465+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4098059430 con 0x7f40a00c5d60 2024-03-14T09:04:14.742 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.742+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]} v 0) v1 -- 0x7f40a00697d0 con 0x7f40a00c5d60 2024-03-14T09:04:14.745 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.746+0000 7f40967fc700 1 -- 172.21.15.154:0/4144084745 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs2_data", "mds", "allow"]}]=0 updated caps for client.1 v24) v1 ==== 185+0+0 (secure 0 0 0) 0x7f4098058fc0 con 0x7f40a00c5d60 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 msgr2=0x7f408406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 0x7f408406cd20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4090007c20 tx=0x7f4090007b70 comp rx=0 tx=0).stop 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 msgr2=0x7f40a006b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f409800b680 tx=0x7f4098004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 shutdown_connections 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f408406a860 0x7f408406cd20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a00c5d60 0x7f40a006b880 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40a00702f0 0x7f40a0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.749 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 --2- 172.21.15.154:0/4144084745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40a00c4380 0x7f40a00c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 >> 172.21.15.154:0/4144084745 conn(0x7f40a00a9bf0 msgr2=0x7f40a00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:14.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 shutdown_connections 2024-03-14T09:04:14.750 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.748+0000 7f40a7e14700 1 -- 172.21.15.154:0/4144084745 wait complete. 2024-03-14T09:04:14.750 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.1 2024-03-14T09:04:14.760 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:14.760 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:14.782 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:14.783 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:14.845 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.966+0000 7f9ff8dec700 1 Processor -- start 2024-03-14T09:04:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.966+0000 7f9ff8dec700 1 -- start start 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff8dec700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9ff40c6490 0x7f9ff40c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff8dec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff406d610 0x7f9ff415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff8dec700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40c3760 con 0x7f9ff40c7dc0 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff8dec700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40c6210 con 0x7f9ff40c6490 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff8dec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40c3230 con 0x7f9ff406d610 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9ff40c6490 0x7f9ff40c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff1d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9ff40c7dc0 0x7f9ff406cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43486/0 (socket says 172.21.15.154:43486) 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.967+0000 7f9ff1d9b700 1 -- 172.21.15.154:0/1465133852 learned_addr learned my addr 172.21.15.154:0/1465133852 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff259c700 1 -- 172.21.15.154:0/1465133852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff406d610 msgr2=0x7f9ff415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff2d9d700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff406d610 0x7f9ff415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff259c700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff406d610 0x7f9ff415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff259c700 1 -- 172.21.15.154:0/1465133852 >> v1:172.21.15.154:6789/0 conn(0x7f9ff40c7dc0 legacy=0x7f9ff406cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff159a700 1 -- 172.21.15.154:0/1465133852 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156379380 0 0) 0x7f9ff40c3760 con 0x7f9ff40c7dc0 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff259c700 1 -- 172.21.15.154:0/1465133852 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40c3480 con 0x7f9ff40c6490 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff259c700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff40c6490 0x7f9ff40c7880 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d78b8854b541429 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff159a700 1 -- 172.21.15.154:0/1465133852 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011830 con 0x7f9ff40c6490 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff159a700 1 -- 172.21.15.154:0/1465133852 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc011990 con 0x7f9ff40c6490 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.968+0000 7f9ff159a700 1 -- 172.21.15.154:0/1465133852 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc011d20 con 0x7f9ff40c6490 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 -- 172.21.15.154:0/1465133852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff40c6490 msgr2=0x7f9ff40c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff40c6490 0x7f9ff40c7880 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009840 tx=0x7f9fdc009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 -- 172.21.15.154:0/1465133852 shutdown_connections 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff406d610 0x7f9ff415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/1465133852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff40c6490 0x7f9ff40c7880 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 -- 172.21.15.154:0/1465133852 >> 172.21.15.154:0/1465133852 conn(0x7f9ff40a9bf0 msgr2=0x7f9ff40a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 -- 172.21.15.154:0/1465133852 shutdown_connections 2024-03-14T09:04:14.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.969+0000 7f9ff8dec700 1 -- 172.21.15.154:0/1465133852 wait complete. 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.970+0000 7f9ff8dec700 1 Processor -- start 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.970+0000 7f9ff8dec700 1 -- start start 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 0x7f9ff41dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 0x7f9ff41da080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40ac530 con 0x7f9ff406d610 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41ee540 con 0x7f9ff40c7dc0 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41ee820 con 0x7f9ff41dac10 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43620/0 (socket says 172.21.15.154:43620) 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 -- 172.21.15.154:0/4220299923 learned_addr learned my addr 172.21.15.154:0/4220299923 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff2d9d700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 0x7f9ff41da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff1d9b700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 0x7f9ff41dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 -- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 msgr2=0x7f9ff41da080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 0x7f9ff41da080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 -- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 msgr2=0x7f9ff41dbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 0x7f9ff41dbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 -- 172.21.15.154:0/4220299923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0094a0 con 0x7f9ff406d610 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff2d9d700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 0x7f9ff41da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff1d9b700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 0x7f9ff41dbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff259c700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0090b0 tx=0x7f9fdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0038a0 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc01f070 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003c60 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.971+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff40abee0 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.972+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff41dcb30 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.972+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40697d0 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.973+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9fdc01a430 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.973+0000 7f9feaffd700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 0x7f9fd406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.974+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9fdc021070 con 0x7f9ff406d610 2024-03-14T09:04:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.974+0000 7f9ff1d9b700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 0x7f9fd406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:14.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.974+0000 7f9ff1d9b700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 0x7f9fd406ccf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40680e0 tx=0x7f9fe4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:14.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:14.977+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9fdc055e90 con 0x7f9ff406d610 2024-03-14T09:04:15.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.169+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ff40c5f60 con 0x7f9ff406d610 2024-03-14T09:04:15.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.171+0000 7f9feaffd700 1 -- 172.21.15.154:0/4220299923 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f9fdc015090 con 0x7f9ff406d610 2024-03-14T09:04:15.171 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:15.171 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:15.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 msgr2=0x7f9fd406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 0x7f9fd406ccf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40680e0 tx=0x7f9fe4009040 comp rx=0 tx=0).stop 2024-03-14T09:04:15.173 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 msgr2=0x7f9ff40acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0090b0 tx=0x7f9fdc004750 comp rx=0 tx=0).stop 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 shutdown_connections 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ff41dac10 0x7f9ff41da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9fd406a830 0x7f9fd406ccf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ff406d610 0x7f9ff40acef0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 --2- 172.21.15.154:0/4220299923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ff40c7dc0 0x7f9ff41dbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 >> 172.21.15.154:0/4220299923 conn(0x7f9ff40a9bf0 msgr2=0x7f9ff406e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 shutdown_connections 2024-03-14T09:04:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.173+0000 7f9ff8dec700 1 -- 172.21.15.154:0/4220299923 wait complete. 2024-03-14T09:04:15.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:15.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:15.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.310+0000 7f9b93acb700 1 Processor -- start 2024-03-14T09:04:15.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 -- start start 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c3250 0x7f9b8c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9b8c1550b0 0x7f9b8c0c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c13ade0 con 0x7f9b8c0c3250 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c06fb30 con 0x7f9b8c1550b0 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b93acb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c06ec30 con 0x7f9b8c0a8540 2024-03-14T09:04:15.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b91066700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c3250 0x7f9b8c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.311+0000 7f9b91066700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c3250 0x7f9b8c154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43632/0 (socket says 172.21.15.154:43632) 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 -- 172.21.15.154:0/2893017508 learned_addr learned my addr 172.21.15.154:0/2893017508 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b92068700 1 --2- 172.21.15.154:0/2893017508 >> v2:172.21.15.191:3300/0 conn(0x7f9b8c1550b0 0x7f9b8c0c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91867700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 -- 172.21.15.154:0/2893017508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 msgr2=0x7f9b8c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 -- 172.21.15.154:0/2893017508 >> v2:172.21.15.191:3300/0 conn(0x7f9b8c1550b0 msgr2=0x7f9b8c0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 --2- 172.21.15.154:0/2893017508 >> v2:172.21.15.191:3300/0 conn(0x7f9b8c1550b0 0x7f9b8c0c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 -- 172.21.15.154:0/2893017508 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c06ef10 con 0x7f9b8c0c3250 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91867700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b91066700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c154b70 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0c1440 tx=0x7f9b8800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e2706edf0012cb server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b90865700 1 -- 172.21.15.154:0/2893017508 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800dc70 con 0x7f9b8c0c3250 2024-03-14T09:04:15.314 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b90865700 1 -- 172.21.15.154:0/2893017508 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8800ddd0 con 0x7f9b8c0c3250 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.312+0000 7f9b90865700 1 -- 172.21.15.154:0/2893017508 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88010690 con 0x7f9b8c0c3250 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 -- 172.21.15.154:0/2893017508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c3250 msgr2=0x7f9b8c154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c154b70 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0c1440 tx=0x7f9b8800be70 comp rx=0 tx=0).stop 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 -- 172.21.15.154:0/2893017508 shutdown_connections 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 --2- 172.21.15.154:0/2893017508 >> v2:172.21.15.191:3300/0 conn(0x7f9b8c1550b0 0x7f9b8c0c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 --2- 172.21.15.154:0/2893017508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c154b70 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 -- 172.21.15.154:0/2893017508 >> 172.21.15.154:0/2893017508 conn(0x7f9b8c0c3880 msgr2=0x7f9b8c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 -- 172.21.15.154:0/2893017508 shutdown_connections 2024-03-14T09:04:15.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.313+0000 7f9b93acb700 1 -- 172.21.15.154:0/2893017508 wait complete. 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.314+0000 7f9b93acb700 1 Processor -- start 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.314+0000 7f9b93acb700 1 -- start start 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c133a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1e9270 con 0x7f9b8c1550b0 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1e9550 con 0x7f9b8c0c3250 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c1e9830 con 0x7f9b8c0a8540 2024-03-14T09:04:15.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43646/0 (socket says 172.21.15.154:43646) 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 -- 172.21.15.154:0/1438250572 learned_addr learned my addr 172.21.15.154:0/1438250572 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b91867700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b91066700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 -- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 msgr2=0x7f9b8c133a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c133a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 -- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b8c0c3250 msgr2=0x7f9b8c153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 -- 172.21.15.154:0/1438250572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8800b7f0 con 0x7f9b8c1550b0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b92068700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400efc0 tx=0x7f9b8400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400cad0 con 0x7f9b8c1550b0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8400f040 con 0x7f9b8c1550b0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c1e9b10 con 0x7f9b8c1550b0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400ce70 con 0x7f9b8c1550b0 2024-03-14T09:04:15.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.315+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c1542a0 con 0x7f9b8c1550b0 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.316+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c1ea030 con 0x7f9b8c1550b0 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.317+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9b84014480 con 0x7f9b8c1550b0 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.317+0000 7f9b827fc700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 0x7f9b7006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.317+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9b84089730 con 0x7f9b8c1550b0 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.317+0000 7f9b91867700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 0x7f9b7006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.318+0000 7f9b91867700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 0x7f9b7006ccf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c1383d0 tx=0x7f9b7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.322+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9b8c1ea030 con 0x7f9b8c1550b0 2024-03-14T09:04:15.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.514+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b8c1ea030 con 0x7f9b8c1550b0 2024-03-14T09:04:15.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.515+0000 7f9b827fc700 1 -- 172.21.15.154:0/1438250572 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b8c1ea030 con 0x7f9b8c1550b0 2024-03-14T09:04:15.515 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 msgr2=0x7f9b7006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 0x7f9b7006ccf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c1383d0 tx=0x7f9b7c008040 comp rx=0 tx=0).stop 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 msgr2=0x7f9b8c14e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400efc0 tx=0x7f9b8400c0c0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 shutdown_connections 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9b7006a830 0x7f9b7006ccf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c1550b0 0x7f9b8c14e600 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9b8c0a8540 0x7f9b8c133a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 --2- 172.21.15.154:0/1438250572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b8c0c3250 0x7f9b8c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 >> 172.21.15.154:0/1438250572 conn(0x7f9b8c0c3880 msgr2=0x7f9b8c0c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:15.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 shutdown_connections 2024-03-14T09:04:15.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.517+0000 7f9b93acb700 1 -- 172.21.15.154:0/1438250572 wait complete. 2024-03-14T09:04:15.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:15.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.653+0000 7f733e73d700 1 Processor -- start 2024-03-14T09:04:15.653 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.653+0000 7f733e73d700 1 -- start start 2024-03-14T09:04:15.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733e73d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380c3250 0x7f733813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733e73d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73380c1780 0x7f73380c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733e73d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733813ade0 con 0x7f73380a8540 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733e73d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806fb30 con 0x7f73380c1780 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733e73d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733806ec30 con 0x7f73380c3250 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733ccda700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73380c1780 0x7f73380c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733ccda700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f73380c1780 0x7f73380c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43654/0 (socket says 172.21.15.154:43654) 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f733ccda700 1 -- 172.21.15.154:0/1072561635 learned_addr learned my addr 172.21.15.154:0/1072561635 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.654+0000 7f73377fe700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380c3250 0x7f733813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733ccda700 1 -- 172.21.15.154:0/1072561635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380c3250 msgr2=0x7f733813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733ccda700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380c3250 0x7f733813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733ccda700 1 -- 172.21.15.154:0/1072561635 >> v1:172.21.15.191:6789/0 conn(0x7f73380a8540 legacy=0x7f7338154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733ccda700 1 -- 172.21.15.154:0/1072561635 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733806ef10 con 0x7f73380c1780 2024-03-14T09:04:15.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733ccda700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c1780 0x7f73380c1c10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f733000d150 tx=0x7f733000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b5a8c9bde53ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f7336ffd700 1 -- 172.21.15.154:0/1072561635 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330010070 con 0x7f73380c1780 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f7336ffd700 1 -- 172.21.15.154:0/1072561635 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733000dd10 con 0x7f73380c1780 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f7336ffd700 1 -- 172.21.15.154:0/1072561635 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000b310 con 0x7f73380c1780 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733e73d700 1 -- 172.21.15.154:0/1072561635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c1780 msgr2=0x7f73380c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733e73d700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c1780 0x7f73380c1c10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f733000d150 tx=0x7f733000d460 comp rx=0 tx=0).stop 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733e73d700 1 -- 172.21.15.154:0/1072561635 shutdown_connections 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733e73d700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c1780 0x7f73380c1c10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.655+0000 7f733e73d700 1 --2- 172.21.15.154:0/1072561635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380c3250 0x7f733813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.656+0000 7f733e73d700 1 -- 172.21.15.154:0/1072561635 >> 172.21.15.154:0/1072561635 conn(0x7f73380c3880 msgr2=0x7f73380c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.656+0000 7f733e73d700 1 -- 172.21.15.154:0/1072561635 shutdown_connections 2024-03-14T09:04:15.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.656+0000 7f733e73d700 1 -- 172.21.15.154:0/1072561635 wait complete. 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.657+0000 7f733e73d700 1 Processor -- start 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.657+0000 7f733e73d700 1 -- start start 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.657+0000 7f733e73d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380a8540 0x7f73381eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.657+0000 7f733e73d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73380c1780 0x7f73381ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381eac10 con 0x7f73380c3250 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381549e0 con 0x7f73380c1780 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73381dad80 con 0x7f73380a8540 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43656/0 (socket says 172.21.15.154:43656) 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 -- 172.21.15.154:0/546695006 learned_addr learned my addr 172.21.15.154:0/546695006 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f73377fe700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73380c1780 0x7f73381ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f7337fff700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380a8540 0x7f73381eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 -- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380a8540 msgr2=0x7f73381eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380a8540 0x7f73381eddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 -- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73380c1780 msgr2=0x7f73381ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73380c1780 0x7f73381ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 -- 172.21.15.154:0/546695006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733000bd40 con 0x7f73380c3250 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733ccda700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f73380c43c0 tx=0x7f7330004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330009a10 con 0x7f73380c3250 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330009bb0 con 0x7f73380c3250 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000b4c0 con 0x7f73380c3250 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73381ed2e0 con 0x7f73380c3250 2024-03-14T09:04:15.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.658+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381ed8a0 con 0x7f73380c3250 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.659+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73380697d0 con 0x7f73380c3250 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.660+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7330004040 con 0x7f73380c3250 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.660+0000 7f7334ff9700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 0x7f732406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.660+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f733008a740 con 0x7f73380c3250 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.660+0000 7f7337fff700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 0x7f732406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:15.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.661+0000 7f7337fff700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 0x7f732406ccf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f73380c50d0 tx=0x7f732c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:15.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.665+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f73300557d0 con 0x7f73380c3250 2024-03-14T09:04:15.857 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.858+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f733806d420 con 0x7f73380c3250 2024-03-14T09:04:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.858+0000 7f7334ff9700 1 -- 172.21.15.154:0/546695006 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7330058df0 con 0x7f73380c3250 2024-03-14T09:04:15.858 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 msgr2=0x7f732406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 0x7f732406ccf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f73380c50d0 tx=0x7f732c009040 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 msgr2=0x7f73381dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f73380c43c0 tx=0x7f7330004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 shutdown_connections 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f732406a830 0x7f732406ccf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73380c3250 0x7f73381dbd00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73380a8540 0x7f73381eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 --2- 172.21.15.154:0/546695006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73380c1780 0x7f73381ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 >> 172.21.15.154:0/546695006 conn(0x7f73380c3880 msgr2=0x7f733806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.860+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 shutdown_connections 2024-03-14T09:04:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.861+0000 7f733e73d700 1 -- 172.21.15.154:0/546695006 wait complete. 2024-03-14T09:04:15.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:15.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.997+0000 7f861f6ac700 1 Processor -- start 2024-03-14T09:04:15.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.997+0000 7f861f6ac700 1 -- start start 2024-03-14T09:04:16.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861f6ac700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f86180c3250 0x7f861813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861f6ac700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861813ade0 con 0x7f86180a8540 2024-03-14T09:04:16.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861f6ac700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861806fb30 con 0x7f86180c1780 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861f6ac700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861806ec30 con 0x7f86180c3250 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861d448700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f86180a8540 0x7f8618154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43496/0 (socket says 172.21.15.154:43496) 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861d448700 1 -- 172.21.15.154:0/1442372573 learned_addr learned my addr 172.21.15.154:0/1442372573 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337284322 0 0) 0x7f8614001110 con 0x7f86180a8540 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86040035b0 con 0x7f86180a8540 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861cc47700 1 --2- 172.21.15.154:0/1442372573 >> v2:172.21.15.191:3300/0 conn(0x7f86180c3250 0x7f861813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 396294936 0 0) 0x7f86040035b0 con 0x7f86180a8540 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 >> v2:172.21.15.191:3300/0 conn(0x7f86180c3250 msgr2=0x7f861813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 --2- 172.21.15.154:0/1442372573 >> v2:172.21.15.191:3300/0 conn(0x7f86180c3250 0x7f861813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 >> v1:172.21.15.191:6789/0 conn(0x7f86180c1780 legacy=0x7f86180c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861806ef10 con 0x7f86180a8540 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8614001800 con 0x7f86180a8540 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 >> v1:172.21.15.154:6789/0 conn(0x7f86180a8540 legacy=0x7f8618154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 0x7f8604003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8604004850 0x7f8604004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604005330 0x7f860400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861806ec30 con 0x7f8604003a00 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860400c200 con 0x7f8604005330 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860400c3c0 con 0x7f8604004850 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.998+0000 7f861d448700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 0x7f8604003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861cc47700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8604004850 0x7f8604004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861dc49700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604005330 0x7f860400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 -- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8604004850 msgr2=0x7f8604004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8604004850 0x7f8604004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 -- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604005330 msgr2=0x7f860400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604005330 0x7f860400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 -- 172.21.15.154:0/1442372573 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040035b0 con 0x7f8604003a00 2024-03-14T09:04:16.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861cc47700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8604004850 0x7f8604004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861dc49700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604005330 0x7f860400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861dc49700 1 -- 172.21.15.154:0/1442372573 reap_dead start 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861d448700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 0x7f8604003e20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f861400c210 tx=0x7f8604005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce4f43943426bc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614007d80 con 0x7f8604003a00 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861400cd70 con 0x7f8604003a00 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f860ffff700 1 -- 172.21.15.154:0/1442372573 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614005ba0 con 0x7f8604003a00 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861f6ac700 1 -- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 msgr2=0x7f8604003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:15.999+0000 7f861f6ac700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 0x7f8604003e20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f861400c210 tx=0x7f8604005880 comp rx=0 tx=0).stop 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.000+0000 7f861f6ac700 1 -- 172.21.15.154:0/1442372573 shutdown_connections 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.000+0000 7f861f6ac700 1 --2- 172.21.15.154:0/1442372573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8604003a00 0x7f8604003e20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.000+0000 7f861f6ac700 1 -- 172.21.15.154:0/1442372573 >> 172.21.15.154:0/1442372573 conn(0x7f86180c3880 msgr2=0x7f86180c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.000+0000 7f861f6ac700 1 -- 172.21.15.154:0/1442372573 shutdown_connections 2024-03-14T09:04:16.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.000+0000 7f861f6ac700 1 -- 172.21.15.154:0/1442372573 wait complete. 2024-03-14T09:04:16.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 Processor -- start 2024-03-14T09:04:16.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 -- start start 2024-03-14T09:04:16.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 0x7f86180aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86181ea2b0 0x7f86180c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180aafd0 con 0x7f86181edec0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180aa730 con 0x7f8604003a00 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861f6ac700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86180aaa10 con 0x7f86181ea2b0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861dc49700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861dc49700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43684/0 (socket says 172.21.15.154:43684) 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.001+0000 7f861dc49700 1 -- 172.21.15.154:0/3034102886 learned_addr learned my addr 172.21.15.154:0/3034102886 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861d448700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 0x7f86180aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861cc47700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86181ea2b0 0x7f86180c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 -- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86181ea2b0 msgr2=0x7f86180c0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86181ea2b0 0x7f86180c0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 -- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 msgr2=0x7f86180aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 0x7f86180aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 -- 172.21.15.154:0/3034102886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180aacf0 con 0x7f86181edec0 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861d448700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 0x7f86180aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:16.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861dc49700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f861806d8e0 tx=0x7f861000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610003c10 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610003d70 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86181ed9a0 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610005310 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.002+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86180aced0 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.003+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86180697d0 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.003+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8610005470 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.004+0000 7f860dffb700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 0x7f85fc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.004+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f86100890b0 con 0x7f86181edec0 2024-03-14T09:04:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.004+0000 7f861d448700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 0x7f85fc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.005+0000 7f861d448700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 0x7f85fc06ccf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8614001ac0 tx=0x7f861400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.008+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f86100541e0 con 0x7f86181edec0 2024-03-14T09:04:16.201 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.202+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f861806d420 con 0x7f86181edec0 2024-03-14T09:04:16.202 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.202+0000 7f860dffb700 1 -- 172.21.15.154:0/3034102886 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8610057800 con 0x7f86181edec0 2024-03-14T09:04:16.202 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 msgr2=0x7f85fc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 0x7f85fc06ccf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8614001ac0 tx=0x7f861400a000 comp rx=0 tx=0).stop 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 msgr2=0x7f86181ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f861806d8e0 tx=0x7f861000ee00 comp rx=0 tx=0).stop 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 shutdown_connections 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f85fc06a830 0x7f85fc06ccf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86181edec0 0x7f86181ee350 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8604003a00 0x7f86180aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 --2- 172.21.15.154:0/3034102886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86181ea2b0 0x7f86180c0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 >> 172.21.15.154:0/3034102886 conn(0x7f86180c3880 msgr2=0x7f86180c1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.204+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 shutdown_connections 2024-03-14T09:04:16.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.205+0000 7f861f6ac700 1 -- 172.21.15.154:0/3034102886 wait complete. 2024-03-14T09:04:16.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:16.340 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 Processor -- start 2024-03-14T09:04:16.341 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 -- start start 2024-03-14T09:04:16.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f368015e280 0x7f3680160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800ac8e0 con 0x7f368014eaf0 2024-03-14T09:04:16.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368006de30 con 0x7f3680155df0 2024-03-14T09:04:16.343 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.341+0000 7f36863a1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368006b310 con 0x7f368015e280 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f368015e280 0x7f3680160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f368014eaf0 0x7f368006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43504/0 (socket says 172.21.15.154:43504) 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f368015e280 0x7f3680160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43694/0 (socket says 172.21.15.154:43694) 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367ffff700 1 -- 172.21.15.154:0/3595520920 learned_addr learned my addr 172.21.15.154:0/3595520920 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 -- 172.21.15.154:0/3595520920 >> v1:172.21.15.191:6789/0 conn(0x7f3680155df0 legacy=0x7f3680156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 -- 172.21.15.154:0/3595520920 >> v1:172.21.15.154:6789/0 conn(0x7f368014eaf0 legacy=0x7f368006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 -- 172.21.15.154:0/3595520920 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3680156960 con 0x7f368015e280 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367effd700 1 -- 172.21.15.154:0/3595520920 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699892169 0 0) 0x7f36800ac8e0 con 0x7f368014eaf0 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f368493e700 1 --2- 172.21.15.154:0/3595520920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680160670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3678009d90 tx=0x7f367800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc43519b5e72df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.344 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367effd700 1 -- 172.21.15.154:0/3595520920 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800e070 con 0x7f368015e280 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367effd700 1 -- 172.21.15.154:0/3595520920 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800bb90 con 0x7f368015e280 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.342+0000 7f367effd700 1 -- 172.21.15.154:0/3595520920 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678003a20 con 0x7f368015e280 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 -- 172.21.15.154:0/3595520920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 msgr2=0x7f3680160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 --2- 172.21.15.154:0/3595520920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680160670 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3678009d90 tx=0x7f367800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 -- 172.21.15.154:0/3595520920 shutdown_connections 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 --2- 172.21.15.154:0/3595520920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680160670 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 -- 172.21.15.154:0/3595520920 >> 172.21.15.154:0/3595520920 conn(0x7f36800c3880 msgr2=0x7f36800c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 -- 172.21.15.154:0/3595520920 shutdown_connections 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.343+0000 7f36863a1700 1 -- 172.21.15.154:0/3595520920 wait complete. 2024-03-14T09:04:16.345 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.344+0000 7f36863a1700 1 Processor -- start 2024-03-14T09:04:16.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.344+0000 7f36863a1700 1 -- start start 2024-03-14T09:04:16.348 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f368014eaf0 0x7f3680150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3680155df0 0x7f368014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c24c0 con 0x7f368015e280 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3680158760 con 0x7f368014eaf0 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c2760 con 0x7f3680155df0 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43702/0 (socket says 172.21.15.154:43702) 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 -- 172.21.15.154:0/2562293746 learned_addr learned my addr 172.21.15.154:0/2562293746 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:16.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f367ffff700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f368014eaf0 0x7f3680150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f367f7fe700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3680155df0 0x7f368014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 -- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3680155df0 msgr2=0x7f368014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3680155df0 0x7f368014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 -- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f368014eaf0 msgr2=0x7f3680150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f368014eaf0 0x7f3680150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 -- 172.21.15.154:0/2562293746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678009a70 con 0x7f368015e280 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f368493e700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f367800b730 tx=0x7f3678004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678012630 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780127b0 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678012b60 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.345+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680151dd0 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.346+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680158c90 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.346+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3680158ea0 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.347+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3678012cc0 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.347+0000 7f367cff9700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 0x7f366c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.347+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f367808b170 con 0x7f368015e280 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.347+0000 7f367ffff700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 0x7f366c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.348+0000 7f367ffff700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 0x7f366c06ccf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3674007fd0 tx=0x7f3674007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.352+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3680158ea0 con 0x7f368015e280 2024-03-14T09:04:16.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.546+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3680158e70 con 0x7f368015e280 2024-03-14T09:04:16.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.547+0000 7f367cff9700 1 -- 172.21.15.154:0/2562293746 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7f3678059d30 con 0x7f368015e280 2024-03-14T09:04:16.547 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:16.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 msgr2=0x7f366c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 0x7f366c06ccf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3674007fd0 tx=0x7f3674007de0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 msgr2=0x7f3680156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f367800b730 tx=0x7f3678004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 shutdown_connections 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f366c06a830 0x7f366c06ccf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f368015e280 0x7f3680156b20 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f368014eaf0 0x7f3680150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 --2- 172.21.15.154:0/2562293746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3680155df0 0x7f368014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 >> 172.21.15.154:0/2562293746 conn(0x7f36800c3880 msgr2=0x7f3680154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 shutdown_connections 2024-03-14T09:04:16.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.549+0000 7f36863a1700 1 -- 172.21.15.154:0/2562293746 wait complete. 2024-03-14T09:04:16.561 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:16.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.686+0000 7f9d87c16700 1 Processor -- start 2024-03-14T09:04:16.686 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.686+0000 7f9d87c16700 1 -- start start 2024-03-14T09:04:16.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d87c16700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9d800c4380 0x7f9d8014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d87c16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d87c16700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800acd10 con 0x7f9d800702f0 2024-03-14T09:04:16.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d87c16700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80150110 con 0x7f9d800c4380 2024-03-14T09:04:16.687 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d87c16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800ac7b0 con 0x7f9d800c5d60 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d861b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d851b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9d800c4380 0x7f9d8014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d861b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:47184/0 (socket says 172.21.15.154:47184) 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d851b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9d800c4380 0x7f9d8014fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:40668/0 (socket says 172.21.15.154:40668) 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d861b3700 1 -- 172.21.15.154:0/2861892470 learned_addr learned my addr 172.21.15.154:0/2861892470 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d859b2700 1 --1- 172.21.15.154:0/2861892470 >> v1:172.21.15.191:6789/0 conn(0x7f9d800702f0 0x7f9d800c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:46594/0 (socket says 172.21.15.154:46594) 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d851b1700 1 -- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 msgr2=0x7f9d800c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d851b1700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.687+0000 7f9d851b1700 1 -- 172.21.15.154:0/2861892470 >> v1:172.21.15.191:6789/0 conn(0x7f9d800702f0 legacy=0x7f9d800c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d851b1700 1 -- 172.21.15.154:0/2861892470 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800ac9c0 con 0x7f9d800c4380 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d849b0700 1 -- 172.21.15.154:0/2861892470 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352351492 0 0) 0x7f9d800acd10 con 0x7f9d800702f0 2024-03-14T09:04:16.688 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d861b3700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d851b1700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c4380 0x7f9d8014fbd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009840 tx=0x7f9d7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb06c9df2969b87 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d849b0700 1 -- 172.21.15.154:0/2861892470 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c011830 con 0x7f9d800c4380 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d849b0700 1 -- 172.21.15.154:0/2861892470 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c011990 con 0x7f9d800c4380 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d849b0700 1 -- 172.21.15.154:0/2861892470 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c011d20 con 0x7f9d800c4380 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 -- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c4380 msgr2=0x7f9d8014fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c4380 0x7f9d8014fbd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009840 tx=0x7f9d7c009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 -- 172.21.15.154:0/2861892470 shutdown_connections 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800c5d60 0x7f9d800c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.692 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 --2- 172.21.15.154:0/2861892470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c4380 0x7f9d8014fbd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.688+0000 7f9d87c16700 1 -- 172.21.15.154:0/2861892470 >> 172.21.15.154:0/2861892470 conn(0x7f9d800a9bf0 msgr2=0x7f9d800a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.689+0000 7f9d87c16700 1 -- 172.21.15.154:0/2861892470 shutdown_connections 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.689+0000 7f9d87c16700 1 -- 172.21.15.154:0/2861892470 wait complete. 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 Processor -- start 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 -- start start 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 0x7f9d80151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 0x7f9d800c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8006bd90 con 0x7f9d800c4380 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d801520d0 con 0x7f9d800c5d60 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.690+0000 7f9d87c16700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80150d80 con 0x7f9d800702f0 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43708/0 (socket says 172.21.15.154:43708) 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 -- 172.21.15.154:0/1176412240 learned_addr learned my addr 172.21.15.154:0/1176412240 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:16.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d861b3700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 0x7f9d800c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d859b2700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 0x7f9d80151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 -- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 msgr2=0x7f9d80151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 0x7f9d80151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 -- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 msgr2=0x7f9d800c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 0x7f9d800c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 -- 172.21.15.154:0/1176412240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d78009b30 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d861b3700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 0x7f9d800c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d859b2700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 0x7f9d80151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d851b1700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0038a0 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c01f070 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0094a0 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c003c40 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.691+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801506f0 con 0x7f9d800c4380 2024-03-14T09:04:16.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.692+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d800697d0 con 0x7f9d800c4380 2024-03-14T09:04:16.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.693+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9d7c026480 con 0x7f9d800c4380 2024-03-14T09:04:16.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.693+0000 7f9d767fc700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 0x7f9d6406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:16.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.693+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9d7c021070 con 0x7f9d800c4380 2024-03-14T09:04:16.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.693+0000 7f9d859b2700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 0x7f9d6406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:16.695 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.694+0000 7f9d859b2700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 0x7f9d6406ccf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d70007fd0 tx=0x7f9d70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:16.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.698+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9d7c055bb0 con 0x7f9d800c4380 2024-03-14T09:04:16.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.890+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f9d800c7160 con 0x7f9d800c4380 2024-03-14T09:04:16.890 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.890+0000 7f9d767fc700 1 -- 172.21.15.154:0/1176412240 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7f9d7c0150a0 con 0x7f9d800c4380 2024-03-14T09:04:16.891 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:16.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.892+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 msgr2=0x7f9d6406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.892+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 0x7f9d6406ccf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d70007fd0 tx=0x7f9d70009040 comp rx=0 tx=0).stop 2024-03-14T09:04:16.893 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.892+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 msgr2=0x7f9d801edb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.892+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c004750 comp rx=0 tx=0).stop 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.892+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 shutdown_connections 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d800c5d60 0x7f9d800c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d800702f0 0x7f9d80151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9d6406a830 0x7f9d6406ccf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 --2- 172.21.15.154:0/1176412240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d800c4380 0x7f9d801edb60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 >> 172.21.15.154:0/1176412240 conn(0x7f9d800a9bf0 msgr2=0x7f9d800c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 shutdown_connections 2024-03-14T09:04:16.894 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:16.893+0000 7f9d87c16700 1 -- 172.21.15.154:0/1176412240 wait complete. 2024-03-14T09:04:16.905 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:04:17.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.027+0000 7f983c04e700 1 Processor -- start 2024-03-14T09:04:17.028 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 -- start start 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f98340a8540 0x7f983413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98340c3250 0x7f9834154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983413ade0 con 0x7f98341550b0 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983406fb30 con 0x7f98340a8540 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.028+0000 7f983c04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983406ec30 con 0x7f98340c3250 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f983a5eb700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f98341550b0 0x7f98340c1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43506/0 (socket says 172.21.15.154:43506) 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f983a5eb700 1 -- 172.21.15.154:0/3074862046 learned_addr learned my addr 172.21.15.154:0/3074862046 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 --2- 172.21.15.154:0/3074862046 >> v2:172.21.15.191:3300/0 conn(0x7f98340a8540 0x7f983413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.031 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587445719 0 0) 0x7f982c001490 con 0x7f98341550b0 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f98395e9700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98340c3250 0x7f9834154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98200035b0 con 0x7f98341550b0 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165994305 0 0) 0x7f98200035b0 con 0x7f98341550b0 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 -- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98340c3250 msgr2=0x7f9834154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98340c3250 0x7f9834154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 -- 172.21.15.154:0/3074862046 >> v1:172.21.15.154:6789/0 conn(0x7f98341550b0 legacy=0x7f98340c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 -- 172.21.15.154:0/3074862046 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983406ef10 con 0x7f98340a8540 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9839dea700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98340a8540 0x7f983413a8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9828009840 tx=0x7f9828009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90c1abcd72e831ee server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.029+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c001b80 con 0x7f98340a8540 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9828004030 con 0x7f98340a8540 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f9838de8700 1 -- 172.21.15.154:0/3074862046 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280043c0 con 0x7f98340a8540 2024-03-14T09:04:17.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 -- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98340a8540 msgr2=0x7f983413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98340a8540 0x7f983413a8a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9828009840 tx=0x7f9828009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 -- 172.21.15.154:0/3074862046 shutdown_connections 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98340a8540 0x7f983413a8a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 --2- 172.21.15.154:0/3074862046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98340c3250 0x7f9834154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 -- 172.21.15.154:0/3074862046 >> 172.21.15.154:0/3074862046 conn(0x7f98340c3880 msgr2=0x7f98340c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 -- 172.21.15.154:0/3074862046 shutdown_connections 2024-03-14T09:04:17.033 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.030+0000 7f983c04e700 1 -- 172.21.15.154:0/3074862046 wait complete. 2024-03-14T09:04:17.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.031+0000 7f983c04e700 1 Processor -- start 2024-03-14T09:04:17.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.031+0000 7f983c04e700 1 -- start start 2024-03-14T09:04:17.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.036 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98341550b0 0x7f98341ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9834133ee0 0x7f98341ea600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98341eab40 con 0x7f98340c3250 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98341edf20 con 0x7f98341550b0 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983c04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98341ed630 con 0x7f9834133ee0 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43716/0 (socket says 172.21.15.154:43716) 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 -- 172.21.15.154:0/1516752921 learned_addr learned my addr 172.21.15.154:0/1516752921 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f98395e9700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98341550b0 0x7f98341ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f983a5eb700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9834133ee0 0x7f98341ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 -- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9834133ee0 msgr2=0x7f98341ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.037 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9834133ee0 0x7f98341ea600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 -- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98341550b0 msgr2=0x7f98341ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98341550b0 0x7f98341ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 -- 172.21.15.154:0/1516752921 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280094a0 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.032+0000 7f9839dea700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f98280090b0 tx=0x7f9828004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.033+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828003c30 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.033+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982801f070 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.033+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9834154e20 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.033+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828011630 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.033+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98341ed8d0 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.034+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98340697d0 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.034+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f98280117b0 con 0x7f98340c3250 2024-03-14T09:04:17.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.034+0000 7f98267fc700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 0x7f981806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.034+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9828021070 con 0x7f98340c3250 2024-03-14T09:04:17.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.035+0000 7f98395e9700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 0x7f981806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.035+0000 7f98395e9700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 0x7f981806ccf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9830005c50 tx=0x7f983000b750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.039+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9828055a50 con 0x7f98340c3250 2024-03-14T09:04:17.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.247+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f983406d420 con 0x7f98340c3250 2024-03-14T09:04:17.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.247+0000 7f98267fc700 1 -- 172.21.15.154:0/1516752921 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9828059070 con 0x7f98340c3250 2024-03-14T09:04:17.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 msgr2=0x7f981806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 0x7f981806ccf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9830005c50 tx=0x7f983000b750 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 msgr2=0x7f98340c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f98280090b0 tx=0x7f9828004750 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 shutdown_connections 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9834133ee0 0x7f98341ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f981806a830 0x7f981806ccf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98340c3250 0x7f98340c0520 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 --2- 172.21.15.154:0/1516752921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98341550b0 0x7f98341ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 >> 172.21.15.154:0/1516752921 conn(0x7f98340c3880 msgr2=0x7f983406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 shutdown_connections 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.250+0000 7f983c04e700 1 -- 172.21.15.154:0/1516752921 wait complete. 2024-03-14T09:04:17.251 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:04:17.262 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:04:17.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.262 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:17.312 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:04:17.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.313 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:04:17.332 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:17.332 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:17.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:04:17.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:04:17.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.513+0000 7fd9f227a700 1 Processor -- start 2024-03-14T09:04:17.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.513+0000 7fd9f227a700 1 -- start start 2024-03-14T09:04:17.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9f227a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9f227a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9ec0c3250 0x7fd9ec154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9f227a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec13ade0 con 0x7fd9ec1550b0 2024-03-14T09:04:17.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9f227a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec06fb30 con 0x7fd9ec0c3250 2024-03-14T09:04:17.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9f227a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec06ec30 con 0x7fd9ec0a8540 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9ec0c3250 0x7fd9ec154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9ec0c3250 0x7fd9ec154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43722/0 (socket says 172.21.15.154:43722) 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/3056727857 learned_addr learned my addr 172.21.15.154:0/3056727857 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9ebfff700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/3056727857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 msgr2=0x7fd9ec13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/3056727857 >> v1:172.21.15.191:6789/0 conn(0x7fd9ec1550b0 legacy=0x7fd9ec0c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/3056727857 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec06ef10 con 0x7fd9ec0c3250 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eb7fe700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec154b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000bb90 tx=0x7fd9e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614257720fee48f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eaffd700 1 -- 172.21.15.154:0/3056727857 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000dd10 con 0x7fd9ec0c3250 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eaffd700 1 -- 172.21.15.154:0/3056727857 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000de70 con 0x7fd9ec0c3250 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.514+0000 7fd9eaffd700 1 -- 172.21.15.154:0/3056727857 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0004230 con 0x7fd9ec0c3250 2024-03-14T09:04:17.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 -- 172.21.15.154:0/3056727857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 msgr2=0x7fd9ec154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec154b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000bb90 tx=0x7fd9e000bea0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 -- 172.21.15.154:0/3056727857 shutdown_connections 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec13a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 --2- 172.21.15.154:0/3056727857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec154b70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 -- 172.21.15.154:0/3056727857 >> 172.21.15.154:0/3056727857 conn(0x7fd9ec0c3880 msgr2=0x7fd9ec0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 -- 172.21.15.154:0/3056727857 shutdown_connections 2024-03-14T09:04:17.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.515+0000 7fd9f227a700 1 -- 172.21.15.154:0/3056727857 wait complete. 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.516+0000 7fd9f227a700 1 Processor -- start 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.516+0000 7fd9f227a700 1 -- start start 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9ec1550b0 0x7fd9ec1edce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1ed570 con 0x7fd9ec0c3250 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1ed850 con 0x7fd9ec1550b0 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f227a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec1db090 con 0x7fd9ec0a8540 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43732/0 (socket says 172.21.15.154:43732) 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/4116594341 learned_addr learned my addr 172.21.15.154:0/4116594341 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9ebfff700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.522 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9f0817700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9ec1550b0 0x7fd9ec1edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 msgr2=0x7fd9ec150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9ec1550b0 msgr2=0x7fd9ec1edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9ec1550b0 0x7fd9ec1edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.517+0000 7fd9eb7fe700 1 -- 172.21.15.154:0/4116594341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e000b7f0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9eb7fe700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000d5d0 tx=0x7fd9e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0003c30 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e0003d90 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec1dadb0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0010690 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.518+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec1eadc0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.519+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0697d0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.519+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd9e0023440 con 0x7fd9ec0c3250 2024-03-14T09:04:17.523 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.520+0000 7fd9e8ff9700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 0x7fd9d806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:17.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.520+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fd9e008a950 con 0x7fd9ec0c3250 2024-03-14T09:04:17.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.520+0000 7fd9ebfff700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 0x7fd9d806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:17.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.520+0000 7fd9ebfff700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 0x7fd9d806ccf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006b20 tx=0x7fd9dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:17.524 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.524+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd9e0055a80 con 0x7fd9ec0c3250 2024-03-14T09:04:17.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.766+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9ec13ade0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.767+0000 7fd9e8ff9700 1 -- 172.21.15.154:0/4116594341 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+149 (secure 0 0 0) 0x7fd9ec13ade0 con 0x7fd9ec0c3250 2024-03-14T09:04:17.767 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:04:17.767 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 msgr2=0x7fd9d806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 0x7fd9d806ccf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006b20 tx=0x7fd9dc008040 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 msgr2=0x7fd9ec0c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000d5d0 tx=0x7fd9e0004750 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 shutdown_connections 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9ec1550b0 0x7fd9ec1edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9ec0a8540 0x7fd9ec150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd9d806a830 0x7fd9d806ccf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 --2- 172.21.15.154:0/4116594341 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9ec0c3250 0x7fd9ec0c0520 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 >> 172.21.15.154:0/4116594341 conn(0x7fd9ec0c3880 msgr2=0x7fd9ec0c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 shutdown_connections 2024-03-14T09:04:17.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:17.769+0000 7fd9f227a700 1 -- 172.21.15.154:0/4116594341 wait complete. 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:04:17.781 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:04:17.782 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-14T09:04:17.782 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:17.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.782 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:04:17.804 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:17.805 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:17.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.806 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:04:17.875 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:04:17.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.876 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:04:17.938 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:04:17.939 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:04:17.939 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:04:17.939 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:04:17.939 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:04:17.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:17.939 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:04:17.939 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:04:17.939 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:04:17.940 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:04:17.940 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:04:17.940 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:04:18.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.049 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:04:18.067 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:04:18.067 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:18.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.068 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:04:18.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:18.137 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:18.192 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:18.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:18.192 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:18.247 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs2=/ 2024-03-14T09:04:18.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:18.333 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:18.333 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:18.352 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:18.352 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:18.353 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:18.370 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:04:18.371 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:04:18.373 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:04:18.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.373 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:18.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.401 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:04:18.486 INFO:teuthology.orchestra.run.smithi191.stdout:client6175 2024-03-14T09:04:18.487 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:18.487 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:18.504 INFO:tasks.cephfs.caps_helper:creating test file on FS cephfs mounted at /home/ubuntu/cephtest/mnt.0... 2024-03-14T09:04:18.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.504 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/testdir) 2024-03-14T09:04:18.525 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:18.526 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/mnt.0/testdir/testfile 2024-03-14T09:04:18.586 INFO:tasks.cephfs.caps_helper:test file created at {path} with data "{data}. 2024-03-14T09:04:18.586 INFO:tasks.cephfs.caps_helper:creating test file on FS cephfs2 mounted at /home/ubuntu/cephtest/mnt.1... 2024-03-14T09:04:18.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:18.586 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec mkdir /home/ubuntu/cephtest/mnt.1/testdir) 2024-03-14T09:04:18.609 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:18.610 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/mnt.1/testdir/testfile 2024-03-14T09:04:18.669 INFO:tasks.cephfs.caps_helper:test file created at {path} with data "{data}. 2024-03-14T09:04:18.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth add client.testuser mon 'allow r' osd 'allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2' mds 'allow r fsname=cephfs, allow r fsname=cephfs2' 2024-03-14T09:04:18.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.799+0000 7fe4d4523700 1 Processor -- start 2024-03-14T09:04:18.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.799+0000 7fe4d4523700 1 -- start start 2024-03-14T09:04:18.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d4523700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe4cc06c6c0 0x7fe4cc070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d4523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 0x7fe4cc0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d4523700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4cc0c7ce0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d4523700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4cc0c7fc0 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d4523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0c82a0 con 0x7fe4cc0c62f0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d1abe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4cc0c31c0 0x7fe4cc06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43520/0 (socket says 172.21.15.154:43520) 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d1abe700 1 -- 172.21.15.154:0/4204459024 learned_addr learned my addr 172.21.15.154:0/4204459024 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d2ac0700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 0x7fe4cc0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/4204459024 >> v2:172.21.15.191:3300/0 conn(0x7fe4cc06c6c0 0x7fe4cc070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322357506 0 0) 0x7fe4cc0c7ce0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b80035b0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d22bf700 1 -- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 msgr2=0x7fe4cc0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 0x7fe4cc0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d22bf700 1 -- 172.21.15.154:0/4204459024 >> v1:172.21.15.154:6789/0 conn(0x7fe4cc0c31c0 legacy=0x7fe4cc06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667488120 0 0) 0x7fe4b80035b0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d22bf700 1 -- 172.21.15.154:0/4204459024 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc0c8580 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.800+0000 7fe4d2ac0700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 0x7fe4cc0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc06c6c0 0x7fe4cc070960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009840 tx=0x7fe4c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a793ddd11d13f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc001670 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8004030 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d12bd700 1 -- 172.21.15.154:0/4204459024 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80043c0 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 -- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc06c6c0 msgr2=0x7fe4cc070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc06c6c0 0x7fe4cc070960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009840 tx=0x7fe4c8009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 -- 172.21.15.154:0/4204459024 shutdown_connections 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc0c62f0 0x7fe4cc0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 --2- 172.21.15.154:0/4204459024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc06c6c0 0x7fe4cc070960 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 -- 172.21.15.154:0/4204459024 >> 172.21.15.154:0/4204459024 conn(0x7fe4cc0a9bf0 msgr2=0x7fe4cc0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 -- 172.21.15.154:0/4204459024 shutdown_connections 2024-03-14T09:04:18.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.801+0000 7fe4d4523700 1 -- 172.21.15.154:0/4204459024 wait complete. 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.802+0000 7fe4d4523700 1 Processor -- start 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 -- start start 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc06c6c0 0x7fe4cc1dc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc0c62f0 0x7fe4cc1ed620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc1dde90 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc0ac280 con 0x7fe4cc0c62f0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d4523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc09aa90 con 0x7fe4cc06c6c0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d1abe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d1abe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43738/0 (socket says 172.21.15.154:43738) 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d1abe700 1 -- 172.21.15.154:0/2616749579 learned_addr learned my addr 172.21.15.154:0/2616749579 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc06c6c0 0x7fe4cc1dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.803+0000 7fe4d2ac0700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc0c62f0 0x7fe4cc1ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 -- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc06c6c0 msgr2=0x7fe4cc1dc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc06c6c0 0x7fe4cc1dc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 -- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc0c62f0 msgr2=0x7fe4cc1ed620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc0c62f0 0x7fe4cc1ed620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 -- 172.21.15.154:0/2616749579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4009a70 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d1abe700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c780 tx=0x7fe4bc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004c80 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc004de0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c80094a0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0072f0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.804+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc0ac4d0 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.805+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc09ac00 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.805+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe4bc007450 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.806+0000 7fe4c2ffd700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 0x7fe4b006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.806+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fe4bc088500 con 0x7fe4cc0c31c0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.806+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 0x7fe4b006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:18.808 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.806+0000 7fe4d22bf700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 0x7fe4b006ccf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80090b0 tx=0x7fe4c8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:18.810 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:18.810+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe4bc0535a0 con 0x7fe4cc0c31c0 2024-03-14T09:04:19.083 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.083+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow r", "osd", "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2", "mds", "allow r fsname=cephfs, allow r fsname=cephfs2"]} v 0) v1 -- 0x7fe4cc0697d0 con 0x7fe4cc0c31c0 2024-03-14T09:04:19.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.087+0000 7fe4c2ffd700 1 -- 172.21.15.154:0/2616749579 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth add", "entity": "client.testuser", "caps": ["mon", "allow r", "osd", "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2", "mds", "allow r fsname=cephfs, allow r fsname=cephfs2"]}]=0 added key for client.testuser v25) v1 ==== 272+0+0 (secure 0 0 0) 0x7fe4bc007700 con 0x7fe4cc0c31c0 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 msgr2=0x7fe4b006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 0x7fe4b006ccf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80090b0 tx=0x7fe4c8008030 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 msgr2=0x7fe4cc099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c780 tx=0x7fe4bc00ca90 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 shutdown_connections 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4cc0c62f0 0x7fe4cc1ed620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4cc06c6c0 0x7fe4cc1dc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe4b006a830 0x7fe4b006ccf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 --2- 172.21.15.154:0/2616749579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4cc0c31c0 0x7fe4cc099a70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 >> 172.21.15.154:0/2616749579 conn(0x7fe4cc0a9bf0 msgr2=0x7fe4cc06e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:19.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 shutdown_connections 2024-03-14T09:04:19.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.090+0000 7fe4d4523700 1 -- 172.21.15.154:0/2616749579 wait complete. 2024-03-14T09:04:19.091 INFO:teuthology.orchestra.run.smithi154.stderr:added key for client.testuser 2024-03-14T09:04:19.102 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.testuser 2024-03-14T09:04:19.224 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.224+0000 7f2030f24700 1 Processor -- start 2024-03-14T09:04:19.225 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 -- start start 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202c152d80 0x7f202c155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c153210 con 0x7f202c152900 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c14bbb0 con 0x7f202c152d80 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.225+0000 7f2030f24700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c06cff0 con 0x7f202c14b290 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202c152d80 0x7f202c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202c152d80 0x7f202c155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43746/0 (socket says 172.21.15.154:43746) 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 -- 172.21.15.154:0/1409538543 learned_addr learned my addr 172.21.15.154:0/1409538543 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202a59c700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 -- 172.21.15.154:0/1409538543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 msgr2=0x7f202c14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:19.227 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 -- 172.21.15.154:0/1409538543 >> v1:172.21.15.191:6789/0 conn(0x7f202c152900 legacy=0x7f202c154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 -- 172.21.15.154:0/1409538543 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c06d2d0 con 0x7f202c152d80 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202a59c700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202ad9d700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c155350 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f202400d150 tx=0x7f202400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4a5f1d522eb665 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202959a700 1 -- 172.21.15.154:0/1409538543 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024010070 con 0x7f202c152d80 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202959a700 1 -- 172.21.15.154:0/1409538543 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202400ddc0 con 0x7f202c152d80 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.226+0000 7f202959a700 1 -- 172.21.15.154:0/1409538543 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202400b310 con 0x7f202c152d80 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 -- 172.21.15.154:0/1409538543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 msgr2=0x7f202c155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c155350 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f202400d150 tx=0x7f202400d460 comp rx=0 tx=0).stop 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 -- 172.21.15.154:0/1409538543 shutdown_connections 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 --2- 172.21.15.154:0/1409538543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c155350 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 -- 172.21.15.154:0/1409538543 >> 172.21.15.154:0/1409538543 conn(0x7f202c0c5e70 msgr2=0x7f202c0c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:19.228 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 -- 172.21.15.154:0/1409538543 shutdown_connections 2024-03-14T09:04:19.229 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.227+0000 7f2030f24700 1 -- 172.21.15.154:0/1409538543 wait complete. 2024-03-14T09:04:19.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.228+0000 7f2030f24700 1 Processor -- start 2024-03-14T09:04:19.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.228+0000 7f2030f24700 1 -- start start 2024-03-14T09:04:19.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c0aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c152900 0x7f202c1da800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.231 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1edc10 con 0x7f202c152d80 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1ed000 con 0x7f202c152900 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2030f24700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c1ed2e0 con 0x7f202c14b290 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43754/0 (socket says 172.21.15.154:43754) 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 -- 172.21.15.154:0/3046467673 learned_addr learned my addr 172.21.15.154:0/3046467673 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202a59c700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c0aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f2029d9b700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c152900 0x7f202c1da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 -- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 msgr2=0x7f202c0aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c0aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 -- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c152900 msgr2=0x7f202c1da800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c152900 0x7f202c1da800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 -- 172.21.15.154:0/3046467673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202400bd40 con 0x7f202c152d80 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f202ad9d700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f202c0c6e40 tx=0x7f2024004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.232 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.229+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024009a10 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.230+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024009bb0 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.230+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202400bbc0 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.230+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c1ed5c0 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.230+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c0aad40 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.230+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c1edd50 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.231+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2024004040 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.231+0000 7f201affd700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 0x7f200c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.231+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f202408a700 con 0x7f202c152d80 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.232+0000 7f202a59c700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 0x7f200c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.233 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.232+0000 7f202a59c700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 0x7f200c06ccf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2020006f10 tx=0x7f2020008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.235 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.236+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f202c1edd50 con 0x7f202c152d80 2024-03-14T09:04:19.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.507+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.testuser"} v 0) v1 -- 0x7f202c1edd50 con 0x7f202c152d80 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.507+0000 7f201affd700 1 -- 172.21.15.154:0/3046467673 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.testuser"}]=0 v25) v1 ==== 85+0+226 (secure 0 0 0) 0x7f202c1edd50 con 0x7f202c152d80 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stdout:[client.testuser] 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg== 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2024-03-14T09:04:19.507 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.509+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 msgr2=0x7f200c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.509+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 0x7f200c06ccf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2020006f10 tx=0x7f2020008040 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.509+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 msgr2=0x7f202c144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.509+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f202c0c6e40 tx=0x7f2024004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 shutdown_connections 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f200c06a830 0x7f200c06ccf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202c152d80 0x7f202c144830 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f202c14b290 0x7f202c0aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 --2- 172.21.15.154:0/3046467673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c152900 0x7f202c1da800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 >> 172.21.15.154:0/3046467673 conn(0x7f202c0c5e70 msgr2=0x7f202c0c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 shutdown_connections 2024-03-14T09:04:19.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.510+0000 7f2030f24700 1 -- 172.21.15.154:0/3046467673 wait complete. 2024-03-14T09:04:19.521 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:04:19.538 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.ajJGAroN3h 2024-03-14T09:04:19.538 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:19.538 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/tmp/tmp.ajJGAroN3h 2024-03-14T09:04:19.595 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-03-14T09:04:19.616 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.ILqEPLP1Vm 2024-03-14T09:04:19.616 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:19.616 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/tmp/tmp.ILqEPLP1Vm 2024-03-14T09:04:19.675 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:19.676 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:19.693 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:19.693 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:19.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:19.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.913+0000 7f5f57f25700 1 Processor -- start 2024-03-14T09:04:19.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.913+0000 7f5f57f25700 1 -- start start 2024-03-14T09:04:19.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f57f25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f5015df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f57f25700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500acd10 con 0x7f5f500c4380 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f57f25700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500ac9c0 con 0x7f5f500702f0 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f57f25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500ac7b0 con 0x7f5f5014fbd0 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f554c0700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5f500c4380 0x7f5f500c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43114/0 (socket says 172.21.15.154:43114) 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f554c0700 1 -- 172.21.15.154:0/2040564200 learned_addr learned my addr 172.21.15.154:0/2040564200 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f5015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.914+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335819777 0 0) 0x7f5f500acd10 con 0x7f5f500c4380 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f3c0035b0 con 0x7f5f500c4380 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575380445 0 0) 0x7f5f3c0035b0 con 0x7f5f500c4380 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 msgr2=0x7f5f5015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f5015df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 >> v1:172.21.15.191:6789/0 conn(0x7f5f500702f0 legacy=0x7f5f500c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50150020 con 0x7f5f500c4380 2024-03-14T09:04:19.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f5f4c001670 con 0x7f5f500c4380 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 >> v1:172.21.15.154:6789/0 conn(0x7f5f500c4380 legacy=0x7f5f500c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f3c003a30 0x7f5f3c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500ac7b0 con 0x7f5f3c004d90 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c00bac0 con 0x7f5f3c003a30 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c00bc80 con 0x7f5f5014fbd0 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f554c0700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f55cc1700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f3c003a30 0x7f5f3c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 -- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 msgr2=0x7f5f3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f5014fbd0 0x7f5f3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 -- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f3c003a30 msgr2=0x7f5f3c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:19.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f3c003a30 0x7f5f3c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 -- 172.21.15.154:0/2040564200 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0035b0 con 0x7f5f3c004d90 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.915+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009f90 tx=0x7f5f3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2692562d08669adf server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4800ec30 con 0x7f5f3c004d90 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4800f040 con 0x7f5f3c004d90 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f54cbf700 1 -- 172.21.15.154:0/2040564200 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48014480 con 0x7f5f3c004d90 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 -- 172.21.15.154:0/2040564200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c004d90 msgr2=0x7f5f3c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2040564200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009f90 tx=0x7f5f3c005880 comp rx=0 tx=0).stop 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f564c2700 1 -- 172.21.15.154:0/2040564200 reap_dead start 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 -- 172.21.15.154:0/2040564200 shutdown_connections 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 -- 172.21.15.154:0/2040564200 >> 172.21.15.154:0/2040564200 conn(0x7f5f500a9bf0 msgr2=0x7f5f500a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 -- 172.21.15.154:0/2040564200 shutdown_connections 2024-03-14T09:04:19.919 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.916+0000 7f5f57f25700 1 -- 172.21.15.154:0/2040564200 wait complete. 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.917+0000 7f5f57f25700 1 Processor -- start 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.917+0000 7f5f57f25700 1 -- start start 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f3c004d90 0x7f5f50142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 0x7f5f501eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f50143230 con 0x7f5f501ebc80 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f501ec130 con 0x7f5f50145040 2024-03-14T09:04:19.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f57f25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5014a9e0 con 0x7f5f3c004d90 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33512/0 (socket says 172.21.15.154:33512) 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 -- 172.21.15.154:0/2769050415 learned_addr learned my addr 172.21.15.154:0/2769050415 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f55cc1700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f3c004d90 0x7f5f50142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f554c0700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 0x7f5f501eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 -- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f3c004d90 msgr2=0x7f5f50142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f3c004d90 0x7f5f50142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 -- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 msgr2=0x7f5f501eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 0x7f5f501eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f564c2700 1 -- 172.21.15.154:0/2769050415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5014acc0 con 0x7f5f501ebc80 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.918+0000 7f5f554c0700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 0x7f5f501eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f564c2700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0047b0 tx=0x7f5f48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48015ab0 con 0x7f5f501ebc80 2024-03-14T09:04:19.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f48015c30 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5014afa0 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4801e430 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.919+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f501459f0 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.920+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f50074110 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.920+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5f48024020 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.921+0000 7f5f467fc700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 0x7f5f3406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.921+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f5f4808a970 con 0x7f5f501ebc80 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.921+0000 7f5f55cc1700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 0x7f5f3406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:19.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.921+0000 7f5f55cc1700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 0x7f5f3406ccf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f501dfcb0 tx=0x7f5f40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:19.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:19.925+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5f48055bc0 con 0x7f5f501ebc80 2024-03-14T09:04:20.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.121+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f500697d0 con 0x7f5f501ebc80 2024-03-14T09:04:20.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.122+0000 7f5f467fc700 1 -- 172.21.15.154:0/2769050415 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f5f480591e0 con 0x7f5f501ebc80 2024-03-14T09:04:20.123 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:20.123 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 msgr2=0x7f5f3406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 0x7f5f3406ccf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f501dfcb0 tx=0x7f5f40009040 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 msgr2=0x7f5f50141cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0047b0 tx=0x7f5f48009750 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 shutdown_connections 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f3406a830 0x7f5f3406ccf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f501ebc80 0x7f5f50141cd0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f3c004d90 0x7f5f50142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 --2- 172.21.15.154:0/2769050415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f50145040 0x7f5f501eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 >> 172.21.15.154:0/2769050415 conn(0x7f5f500a9bf0 msgr2=0x7f5f500c2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 shutdown_connections 2024-03-14T09:04:20.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.124+0000 7f5f57f25700 1 -- 172.21.15.154:0/2769050415 wait complete. 2024-03-14T09:04:20.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:20.137 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:20.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.262+0000 7f5e730b1700 1 Processor -- start 2024-03-14T09:04:20.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.262+0000 7f5e730b1700 1 -- start start 2024-03-14T09:04:20.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e730b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c14eaf0 0x7f5e6c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e730b1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e6c156500 0x7f5e6c15f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e730b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c0ac8e0 con 0x7f5e6c06acb0 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e730b1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6c06b310 con 0x7f5e6c156500 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e730b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c06de30 con 0x7f5e6c14eaf0 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e7164e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e6c156500 0x7f5e6c15f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e7164e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5e6c156500 0x7f5e6c15f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33520/0 (socket says 172.21.15.154:33520) 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e7164e700 1 -- 172.21.15.154:0/1014663646 learned_addr learned my addr 172.21.15.154:0/1014663646 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.263+0000 7f5e70e4d700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c14eaf0 0x7f5e6c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e7164e700 1 -- 172.21.15.154:0/1014663646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c14eaf0 msgr2=0x7f5e6c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e7164e700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c14eaf0 0x7f5e6c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e7164e700 1 -- 172.21.15.154:0/1014663646 >> v1:172.21.15.191:6789/0 conn(0x7f5e6c06acb0 legacy=0x7f5e6c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e7164e700 1 -- 172.21.15.154:0/1014663646 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c156950 con 0x7f5e6c156500 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e7164e700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c156500 0x7f5e6c15f280 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e50 tx=0x7f5e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77b41df1c59f773 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e6b7fe700 1 -- 172.21.15.154:0/1014663646 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400e070 con 0x7f5e6c156500 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e6b7fe700 1 -- 172.21.15.154:0/1014663646 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6400b910 con 0x7f5e6c156500 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e6b7fe700 1 -- 172.21.15.154:0/1014663646 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6400bcc0 con 0x7f5e6c156500 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e730b1700 1 -- 172.21.15.154:0/1014663646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c156500 msgr2=0x7f5e6c15f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.264+0000 7f5e730b1700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c156500 0x7f5e6c15f280 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e50 tx=0x7f5e6400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 -- 172.21.15.154:0/1014663646 shutdown_connections 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c14eaf0 0x7f5e6c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 --2- 172.21.15.154:0/1014663646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c156500 0x7f5e6c15f280 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 -- 172.21.15.154:0/1014663646 >> 172.21.15.154:0/1014663646 conn(0x7f5e6c0c3880 msgr2=0x7f5e6c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 -- 172.21.15.154:0/1014663646 shutdown_connections 2024-03-14T09:04:20.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.265+0000 7f5e730b1700 1 -- 172.21.15.154:0/1014663646 wait complete. 2024-03-14T09:04:20.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 Processor -- start 2024-03-14T09:04:20.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 -- start start 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e6c14eaf0 0x7f5e6c145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c156500 0x7f5e6c1ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c153650 con 0x7f5e6c06acb0 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c1498b0 con 0x7f5e6c14eaf0 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.266+0000 7f5e730b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c148e30 con 0x7f5e6c156500 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33526/0 (socket says 172.21.15.154:33526) 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 -- 172.21.15.154:0/2909774283 learned_addr learned my addr 172.21.15.154:0/2909774283 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e7164e700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c156500 0x7f5e6c1ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e6bfff700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e6c14eaf0 0x7f5e6c145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 -- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c156500 msgr2=0x7f5e6c1ed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c156500 0x7f5e6c1ed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 -- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e6c14eaf0 msgr2=0x7f5e6c145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e6c14eaf0 0x7f5e6c145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 -- 172.21.15.154:0/2909774283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64009b30 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e70e4d700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9b0 tx=0x7f5e6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60009710 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6000c850 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c149b50 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000cbc0 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.267+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c149110 con 0x7f5e6c06acb0 2024-03-14T09:04:20.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.268+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c152f60 con 0x7f5e6c06acb0 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.269+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5e6000cd20 con 0x7f5e6c06acb0 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.269+0000 7f5e697fa700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 0x7f5e5006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.269+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f5e60089150 con 0x7f5e6c06acb0 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.269+0000 7f5e6bfff700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 0x7f5e5006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.270+0000 7f5e6bfff700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 0x7f5e5006ccf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c154df0 tx=0x7f5e58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.273+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5e60054280 con 0x7f5e6c06acb0 2024-03-14T09:04:20.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.470+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e6c067dc0 con 0x7f5e6c06acb0 2024-03-14T09:04:20.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.471+0000 7f5e697fa700 1 -- 172.21.15.154:0/2909774283 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e600578a0 con 0x7f5e6c06acb0 2024-03-14T09:04:20.470 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:20.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 msgr2=0x7f5e5006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 0x7f5e5006ccf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c154df0 tx=0x7f5e58009040 comp rx=0 tx=0).stop 2024-03-14T09:04:20.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 msgr2=0x7f5e6c153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9b0 tx=0x7f5e6000ed70 comp rx=0 tx=0).stop 2024-03-14T09:04:20.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 shutdown_connections 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e6c156500 0x7f5e6c1ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5e5006a830 0x7f5e5006ccf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e6c06acb0 0x7f5e6c153110 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 --2- 172.21.15.154:0/2909774283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e6c14eaf0 0x7f5e6c145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 >> 172.21.15.154:0/2909774283 conn(0x7f5e6c0c3880 msgr2=0x7f5e6c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 shutdown_connections 2024-03-14T09:04:20.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.473+0000 7f5e730b1700 1 -- 172.21.15.154:0/2909774283 wait complete. 2024-03-14T09:04:20.485 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:20.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.609+0000 7f132fe2e700 1 Processor -- start 2024-03-14T09:04:20.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.609+0000 7f132fe2e700 1 -- start start 2024-03-14T09:04:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132fe2e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1328155df0 0x7f1328156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132fe2e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f132815d270 0x7f132815f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132fe2e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280ac8e0 con 0x7f132814eaf0 2024-03-14T09:04:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132fe2e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132806b310 con 0x7f132815d270 2024-03-14T09:04:20.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132fe2e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132806de30 con 0x7f1328155df0 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f132815d270 0x7f132815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132dbca700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f132814eaf0 0x7f132806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43120/0 (socket says 172.21.15.154:43120) 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132dbca700 1 -- 172.21.15.154:0/2788028973 learned_addr learned my addr 172.21.15.154:0/2788028973 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 -- 172.21.15.154:0/2788028973 >> v2:172.21.15.191:3300/0 conn(0x7f1328155df0 msgr2=0x7f1328156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 --2- 172.21.15.154:0/2788028973 >> v2:172.21.15.191:3300/0 conn(0x7f1328155df0 0x7f1328156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 -- 172.21.15.154:0/2788028973 >> v1:172.21.15.154:6789/0 conn(0x7f132814eaf0 legacy=0x7f132806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 -- 172.21.15.154:0/2788028973 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280acb30 con 0x7f132815d270 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132cbc8700 1 -- 172.21.15.154:0/2788028973 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169892272 0 0) 0x7f13280ac8e0 con 0x7f132814eaf0 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132e3cb700 1 --2- 172.21.15.154:0/2788028973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815d270 0x7f132815f660 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f132814f1c0 tx=0x7f132000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cf2dea7e62fdba server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132cbc8700 1 -- 172.21.15.154:0/2788028973 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f132815d270 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.610+0000 7f132cbc8700 1 -- 172.21.15.154:0/2788028973 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132000bbd0 con 0x7f132815d270 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132cbc8700 1 -- 172.21.15.154:0/2788028973 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320012430 con 0x7f132815d270 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 -- 172.21.15.154:0/2788028973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815d270 msgr2=0x7f132815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 --2- 172.21.15.154:0/2788028973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815d270 0x7f132815f660 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f132814f1c0 tx=0x7f132000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 -- 172.21.15.154:0/2788028973 shutdown_connections 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 --2- 172.21.15.154:0/2788028973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815d270 0x7f132815f660 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 --2- 172.21.15.154:0/2788028973 >> v2:172.21.15.191:3300/0 conn(0x7f1328155df0 0x7f1328156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 -- 172.21.15.154:0/2788028973 >> 172.21.15.154:0/2788028973 conn(0x7f13280c3880 msgr2=0x7f13280c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 -- 172.21.15.154:0/2788028973 shutdown_connections 2024-03-14T09:04:20.613 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.611+0000 7f132fe2e700 1 -- 172.21.15.154:0/2788028973 wait complete. 2024-03-14T09:04:20.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.612+0000 7f132fe2e700 1 Processor -- start 2024-03-14T09:04:20.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.612+0000 7f132fe2e700 1 -- start start 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1328155df0 0x7f1328153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f132815d270 0x7f13281ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132814cc70 con 0x7f132814eaf0 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328154380 con 0x7f1328155df0 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132fe2e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1328152a20 con 0x7f132815d270 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33544/0 (socket says 172.21.15.154:33544) 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 -- 172.21.15.154:0/4078330570 learned_addr learned my addr 172.21.15.154:0/4078330570 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 -- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f132815d270 msgr2=0x7f13281ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132e3cb700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f132815d270 0x7f13281ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132d3c9700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1328155df0 0x7f1328153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f132815d270 0x7f13281ed5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 -- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1328155df0 msgr2=0x7f1328153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1328155df0 0x7f1328153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 -- 172.21.15.154:0/4078330570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1320009a70 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.613+0000 7f132dbca700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f13240011e0 tx=0x7f132400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.614+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324004c80 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.614+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324004e00 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.614+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324007310 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.614+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1328152d00 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.614+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132813ee00 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.615+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1328153220 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.615+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1324007470 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.615+0000 7f131e7fc700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 0x7f130c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.616+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f1324088de0 con 0x7f132814eaf0 2024-03-14T09:04:20.617 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.616+0000 7f132d3c9700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 0x7f130c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.616+0000 7f132d3c9700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 0x7f130c06ccf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.620+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1328153220 con 0x7f132814eaf0 2024-03-14T09:04:20.815 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.815+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1328153220 con 0x7f132814eaf0 2024-03-14T09:04:20.816 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.816+0000 7f131e7fc700 1 -- 172.21.15.154:0/4078330570 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1328153220 con 0x7f132814eaf0 2024-03-14T09:04:20.816 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:20.818 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 msgr2=0x7f130c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.818 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 0x7f130c06ccf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1318006fd0 tx=0x7f1318008040 comp rx=0 tx=0).stop 2024-03-14T09:04:20.818 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 msgr2=0x7f132813e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f13240011e0 tx=0x7f132400cae0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 shutdown_connections 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f132815d270 0x7f13281ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f130c06a830 0x7f130c06ccf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132814eaf0 0x7f132813e8c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 --2- 172.21.15.154:0/4078330570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1328155df0 0x7f1328153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 >> 172.21.15.154:0/4078330570 conn(0x7f13280c3880 msgr2=0x7f1328154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 shutdown_connections 2024-03-14T09:04:20.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.818+0000 7f132fe2e700 1 -- 172.21.15.154:0/4078330570 wait complete. 2024-03-14T09:04:20.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:20.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 Processor -- start 2024-03-14T09:04:20.955 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 -- start start 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f59fc155df0 0x7f59fc156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc0ac8e0 con 0x7f59fc15d270 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc06de30 con 0x7f59fc14eaf0 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.955+0000 7f5a01ed6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc06b310 con 0x7f59fc155df0 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fbfff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f59fc15d270 0x7f59fc15f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43122/0 (socket says 172.21.15.154:43122) 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fbfff700 1 -- 172.21.15.154:0/2273357068 learned_addr learned my addr 172.21.15.154:0/2273357068 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59faffd700 1 --2- 172.21.15.154:0/2273357068 >> v2:172.21.15.191:3300/0 conn(0x7f59fc155df0 0x7f59fc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053161565 0 0) 0x7f59fc0ac8e0 con 0x7f59fc15d270 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e40035b0 con 0x7f59fc15d270 2024-03-14T09:04:20.962 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077800873 0 0) 0x7f59e40035b0 con 0x7f59fc15d270 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 >> v2:172.21.15.191:3300/0 conn(0x7f59fc155df0 msgr2=0x7f59fc156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 --2- 172.21.15.154:0/2273357068 >> v2:172.21.15.191:3300/0 conn(0x7f59fc155df0 0x7f59fc156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 >> v1:172.21.15.191:6789/0 conn(0x7f59fc14eaf0 legacy=0x7f59fc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0acb30 con 0x7f59fc15d270 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f59f40019f0 con 0x7f59fc15d270 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 >> v1:172.21.15.154:6789/0 conn(0x7f59fc15d270 legacy=0x7f59fc15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4003a00 0x7f59e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 0x7f59e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59e4005330 0x7f59e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.963 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc06b310 con 0x7f59e4004850 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e400c200 con 0x7f59e4005330 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e400c3c0 con 0x7f59e4003a00 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59faffd700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 0x7f59e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fbfff700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59e4005330 0x7f59e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.956+0000 7f59fb7fe700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4003a00 0x7f59e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 -- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4003a00 msgr2=0x7f59e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4003a00 0x7f59e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 -- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59e4005330 msgr2=0x7f59e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59e4005330 0x7f59e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59fbfff700 1 -- 172.21.15.154:0/2273357068 reap_dead start 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 -- 172.21.15.154:0/2273357068 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40035b0 con 0x7f59e4004850 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59faffd700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 0x7f59e4004cd0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009b30 tx=0x7f59e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efd12657592a9df server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000e6c0 con 0x7f59e4004850 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f000e820 con 0x7f59e4004850 2024-03-14T09:04:20.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f59fa7fc700 1 -- 172.21.15.154:0/2273357068 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000eb90 con 0x7f59e4004850 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 msgr2=0x7f59e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 0x7f59e4004cd0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009b30 tx=0x7f59e4005880 comp rx=0 tx=0).stop 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2273357068 shutdown_connections 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2273357068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59e4004850 0x7f59e4004cd0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2273357068 >> 172.21.15.154:0/2273357068 conn(0x7f59fc0c3880 msgr2=0x7f59fc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2273357068 shutdown_connections 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.957+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2273357068 wait complete. 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 Processor -- start 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 -- start start 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4004850 0x7f59fc1eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59fc13eec0 0x7f59fc14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc14ada0 con 0x7f59fc14dca0 2024-03-14T09:04:20.965 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1571d0 con 0x7f59fc13eec0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.959+0000 7f5a01ed6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc148d00 con 0x7f59e4004850 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33570/0 (socket says 172.21.15.154:33570) 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 -- 172.21.15.154:0/2515749639 learned_addr learned my addr 172.21.15.154:0/2515749639 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59fb7fe700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4004850 0x7f59fc1eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59fbfff700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59fc13eec0 0x7f59fc14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 -- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4004850 msgr2=0x7f59fc1eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4004850 0x7f59fc1eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 -- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59fc13eec0 msgr2=0x7f59fc14caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59fc13eec0 0x7f59fc14caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 -- 172.21.15.154:0/2515749639 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc148fe0 con 0x7f59fc14dca0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59faffd700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e4b0 tx=0x7f59f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000e6c0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0003ad0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc14c2a0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0003e80 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.960+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc1492c0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.961+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc1556c0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.962+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f59f0013470 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.962+0000 7f59e3fff700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 0x7f59e806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.962+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f59f008a7d0 con 0x7f59fc14dca0 2024-03-14T09:04:20.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.962+0000 7f59fb7fe700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 0x7f59e806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:20.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.963+0000 7f59fb7fe700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 0x7f59e806ccf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f59ec007fd0 tx=0x7f59ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:20.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:20.966+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f59f0055900 con 0x7f59fc14dca0 2024-03-14T09:04:21.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.159+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59fc0697d0 con 0x7f59fc14dca0 2024-03-14T09:04:21.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.159+0000 7f59e3fff700 1 -- 172.21.15.154:0/2515749639 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59f0058f20 con 0x7f59fc14dca0 2024-03-14T09:04:21.159 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:21.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 msgr2=0x7f59e806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 0x7f59e806ccf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f59ec007fd0 tx=0x7f59ec009040 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 msgr2=0x7f59fc14a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e4b0 tx=0x7f59f00047c0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 shutdown_connections 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59fc13eec0 0x7f59fc14caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59e4004850 0x7f59fc1eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f59e806a830 0x7f59e806ccf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 --2- 172.21.15.154:0/2515749639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59fc14dca0 0x7f59fc14a860 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 >> 172.21.15.154:0/2515749639 conn(0x7f59fc0c3880 msgr2=0x7f59fc13f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 shutdown_connections 2024-03-14T09:04:21.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.161+0000 7f5a01ed6700 1 -- 172.21.15.154:0/2515749639 wait complete. 2024-03-14T09:04:21.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:21.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.298+0000 7f99acef2700 1 Processor -- start 2024-03-14T09:04:21.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.298+0000 7f99acef2700 1 -- start start 2024-03-14T09:04:21.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99acef2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99a814eaf0 0x7f99a806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99acef2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 0x7f99a815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99acef2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80ac8e0 con 0x7f99a806acb0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99acef2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a806b310 con 0x7f99a814eaf0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99acef2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806de30 con 0x7f99a8156500 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99a814eaf0 0x7f99a806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f99a806acb0 0x7f99a8155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43136/0 (socket says 172.21.15.154:43136) 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a5d9b700 1 -- 172.21.15.154:0/647428239 learned_addr learned my addr 172.21.15.154:0/647428239 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a6d9d700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 0x7f99a815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 -- 172.21.15.154:0/647428239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 msgr2=0x7f99a815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 0x7f99a815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 -- 172.21.15.154:0/647428239 >> v1:172.21.15.154:6789/0 conn(0x7f99a806acb0 legacy=0x7f99a8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 -- 172.21.15.154:0/647428239 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8156930 con 0x7f99a814eaf0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a559a700 1 -- 172.21.15.154:0/647428239 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471248127 0 0) 0x7f99a80ac8e0 con 0x7f99a806acb0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a6d9d700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 0x7f99a815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a659c700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a814eaf0 0x7f99a806a770 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9998009840 tx=0x7f9998009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d710dee2ccf203b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a559a700 1 -- 172.21.15.154:0/647428239 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998011830 con 0x7f99a814eaf0 2024-03-14T09:04:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a559a700 1 -- 172.21.15.154:0/647428239 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998011990 con 0x7f99a814eaf0 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.299+0000 7f99a559a700 1 -- 172.21.15.154:0/647428239 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998011d20 con 0x7f99a814eaf0 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 -- 172.21.15.154:0/647428239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a814eaf0 msgr2=0x7f99a806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a814eaf0 0x7f99a806a770 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9998009840 tx=0x7f9998009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 -- 172.21.15.154:0/647428239 shutdown_connections 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a8156500 0x7f99a815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 --2- 172.21.15.154:0/647428239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a814eaf0 0x7f99a806a770 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 -- 172.21.15.154:0/647428239 >> 172.21.15.154:0/647428239 conn(0x7f99a80c3880 msgr2=0x7f99a80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 -- 172.21.15.154:0/647428239 shutdown_connections 2024-03-14T09:04:21.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.300+0000 7f99acef2700 1 -- 172.21.15.154:0/647428239 wait complete. 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.301+0000 7f99acef2700 1 Processor -- start 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.301+0000 7f99acef2700 1 -- start start 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 0x7f99a8143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 0x7f99a8143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81447b0 con 0x7f99a806acb0 2024-03-14T09:04:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8150ea0 con 0x7f99a8156500 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99acef2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a814b390 con 0x7f99a814eaf0 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99a659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99a659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33580/0 (socket says 172.21.15.154:33580) 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99a659c700 1 -- 172.21.15.154:0/3423359624 learned_addr learned my addr 172.21.15.154:0/3423359624 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99a5d9b700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 0x7f99a8143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.302+0000 7f99a6d9d700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 0x7f99a8143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 -- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 msgr2=0x7f99a8143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 0x7f99a8143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 -- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 msgr2=0x7f99a8143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 0x7f99a8143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 -- 172.21.15.154:0/3423359624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980094a0 con 0x7f99a806acb0 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a5d9b700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 0x7f99a8143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a6d9d700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 0x7f99a8143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99a659c700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f99980090b0 tx=0x7f9998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980038a0 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999801f070 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a814af00 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998003c20 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.303+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a81441e0 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.304+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a814b1e0 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.304+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f999801a430 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.305+0000 7f9996ffd700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 0x7f998806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.305+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9998021070 con 0x7f99a806acb0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.305+0000 7f99a5d9b700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 0x7f998806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.306+0000 7f99a5d9b700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 0x7f998806ccf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f99a814c5f0 tx=0x7f999c007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.309+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9998055bc0 con 0x7f99a806acb0 2024-03-14T09:04:21.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.502+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99a80697d0 con 0x7f99a806acb0 2024-03-14T09:04:21.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.502+0000 7f9996ffd700 1 -- 172.21.15.154:0/3423359624 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7f9998015020 con 0x7f99a806acb0 2024-03-14T09:04:21.502 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:21.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 msgr2=0x7f998806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 0x7f998806ccf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f99a814c5f0 tx=0x7f999c007ca0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.504 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 msgr2=0x7f99a8153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f99980090b0 tx=0x7f9998004750 comp rx=0 tx=0).stop 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 shutdown_connections 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99a8156500 0x7f99a8143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f998806a830 0x7f998806ccf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99a806acb0 0x7f99a8153e70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 --2- 172.21.15.154:0/3423359624 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99a814eaf0 0x7f99a8143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 >> 172.21.15.154:0/3423359624 conn(0x7f99a80c3880 msgr2=0x7f99a814c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 shutdown_connections 2024-03-14T09:04:21.505 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.504+0000 7f99acef2700 1 -- 172.21.15.154:0/3423359624 wait complete. 2024-03-14T09:04:21.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:21.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.640+0000 7f3191825700 1 Processor -- start 2024-03-14T09:04:21.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.640+0000 7f3191825700 1 -- start start 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f3191825700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f318c06c6e0 0x7f318c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f3191825700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f318c070a10 0x7f318c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f3191825700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318c0c5f10 con 0x7f318c06d600 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f3191825700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c0c7d50 con 0x7f318c070a10 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f3191825700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c0c8030 con 0x7f318c06c6e0 2024-03-14T09:04:21.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f318a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f318c070a10 0x7f318c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f318a7fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f318c070a10 0x7f318c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33592/0 (socket says 172.21.15.154:33592) 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.641+0000 7f318a7fc700 1 -- 172.21.15.154:0/2052620877 learned_addr learned my addr 172.21.15.154:0/2052620877 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318affd700 1 --2- 172.21.15.154:0/2052620877 >> v2:172.21.15.191:3300/0 conn(0x7f318c06c6e0 0x7f318c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318a7fc700 1 -- 172.21.15.154:0/2052620877 >> v2:172.21.15.191:3300/0 conn(0x7f318c06c6e0 msgr2=0x7f318c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318a7fc700 1 --2- 172.21.15.154:0/2052620877 >> v2:172.21.15.191:3300/0 conn(0x7f318c06c6e0 0x7f318c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318a7fc700 1 -- 172.21.15.154:0/2052620877 >> v1:172.21.15.191:6789/0 conn(0x7f318c06d600 legacy=0x7f318c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318a7fc700 1 -- 172.21.15.154:0/2052620877 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c0c8310 con 0x7f318c070a10 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f318a7fc700 1 --2- 172.21.15.154:0/2052620877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c070a10 0x7f318c0c63a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f318c073f80 tx=0x7f317c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25d1ad813a8fe2c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f3189ffb700 1 -- 172.21.15.154:0/2052620877 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c011830 con 0x7f318c070a10 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f3189ffb700 1 -- 172.21.15.154:0/2052620877 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c011990 con 0x7f318c070a10 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.642+0000 7f3189ffb700 1 -- 172.21.15.154:0/2052620877 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c011d20 con 0x7f318c070a10 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 -- 172.21.15.154:0/2052620877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c070a10 msgr2=0x7f318c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 --2- 172.21.15.154:0/2052620877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c070a10 0x7f318c0c63a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f318c073f80 tx=0x7f317c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.644 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 -- 172.21.15.154:0/2052620877 shutdown_connections 2024-03-14T09:04:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 --2- 172.21.15.154:0/2052620877 >> v2:172.21.15.191:3300/0 conn(0x7f318c06c6e0 0x7f318c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 --2- 172.21.15.154:0/2052620877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c070a10 0x7f318c0c63a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 -- 172.21.15.154:0/2052620877 >> 172.21.15.154:0/2052620877 conn(0x7f318c0a9bf0 msgr2=0x7f318c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 -- 172.21.15.154:0/2052620877 shutdown_connections 2024-03-14T09:04:21.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.643+0000 7f3191825700 1 -- 172.21.15.154:0/2052620877 wait complete. 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.644+0000 7f3191825700 1 Processor -- start 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.644+0000 7f3191825700 1 -- start start 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c06d600 0x7f318c1edb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f318c070a10 0x7f318c1ee040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1ed670 con 0x7f318c06c6e0 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1db010 con 0x7f318c06d600 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3191825700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0ac710 con 0x7f318c070a10 2024-03-14T09:04:21.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33594/0 (socket says 172.21.15.154:33594) 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 -- 172.21.15.154:0/2208256413 learned_addr learned my addr 172.21.15.154:0/2208256413 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318a7fc700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c06d600 0x7f318c1edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318b7fe700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f318c070a10 0x7f318c1ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 -- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f318c070a10 msgr2=0x7f318c1ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f318c070a10 0x7f318c1ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 -- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c06d600 msgr2=0x7f318c1edb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c06d600 0x7f318c1edb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 -- 172.21.15.154:0/2208256413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009550 con 0x7f318c06c6e0 2024-03-14T09:04:21.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f318affd700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9b0 tx=0x7f318000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180009710 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.645+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000c850 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.646+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c0a6cc0 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.646+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000cbc0 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.646+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c0c5300 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.646+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c1ed950 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.647+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f318000cd20 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.647+0000 7f3173fff700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 0x7f317806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.647+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f31800891e0 con 0x7f318c06c6e0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.647+0000 7f318a7fc700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 0x7f317806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.650 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.648+0000 7f318a7fc700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 0x7f317806ccf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f318c070ea0 tx=0x7f317c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.652+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3180054290 con 0x7f318c06c6e0 2024-03-14T09:04:21.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.848+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f318c0c7630 con 0x7f318c06c6e0 2024-03-14T09:04:21.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.848+0000 7f3173fff700 1 -- 172.21.15.154:0/2208256413 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7f31800578b0 con 0x7f318c06c6e0 2024-03-14T09:04:21.848 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:21.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 msgr2=0x7f317806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 0x7f317806ccf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f318c070ea0 tx=0x7f317c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:21.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 msgr2=0x7f318c0c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9b0 tx=0x7f318000ed70 comp rx=0 tx=0).stop 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 shutdown_connections 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f318c070a10 0x7f318c1ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f317806a830 0x7f317806ccf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f318c06c6e0 0x7f318c0c4dc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 --2- 172.21.15.154:0/2208256413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c06d600 0x7f318c1edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 >> 172.21.15.154:0/2208256413 conn(0x7f318c0a9bf0 msgr2=0x7f318c0a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 shutdown_connections 2024-03-14T09:04:21.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.850+0000 7f3191825700 1 -- 172.21.15.154:0/2208256413 wait complete. 2024-03-14T09:04:21.863 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:04:21.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.989+0000 7fa16a320700 1 Processor -- start 2024-03-14T09:04:21.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.989+0000 7fa16a320700 1 -- start start 2024-03-14T09:04:21.991 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa16a320700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa1640c4380 0x7fa16414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa16a320700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1640c5d60 0x7fa1640c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa16a320700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1640acd10 con 0x7fa1640702f0 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa16a320700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa164150110 con 0x7fa1640c5d60 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa16a320700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640c66b0 con 0x7fa1640c4380 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1640c5d60 0x7fa1640c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1640c5d60 0x7fa1640c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33600/0 (socket says 172.21.15.154:33600) 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 -- 172.21.15.154:0/1543989547 learned_addr learned my addr 172.21.15.154:0/1543989547 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1637fe700 1 --2- 172.21.15.154:0/1543989547 >> v2:172.21.15.191:3300/0 conn(0x7fa1640c4380 0x7fa16414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 -- 172.21.15.154:0/1543989547 >> v2:172.21.15.191:3300/0 conn(0x7fa1640c4380 msgr2=0x7fa16414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 --2- 172.21.15.154:0/1543989547 >> v2:172.21.15.191:3300/0 conn(0x7fa1640c4380 0x7fa16414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 -- 172.21.15.154:0/1543989547 >> v1:172.21.15.191:6789/0 conn(0x7fa1640702f0 legacy=0x7fa1640c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 -- 172.21.15.154:0/1543989547 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640ac9c0 con 0x7fa1640c5d60 2024-03-14T09:04:21.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa1688bd700 1 --2- 172.21.15.154:0/1543989547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c5d60 0x7fa1640c6170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa1640c5980 tx=0x7fa15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724de157e4d4e8ee server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa162ffd700 1 -- 172.21.15.154:0/1543989547 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e070 con 0x7fa1640c5d60 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa162ffd700 1 -- 172.21.15.154:0/1543989547 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c003a20 con 0x7fa1640c5d60 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.990+0000 7fa162ffd700 1 -- 172.21.15.154:0/1543989547 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003dd0 con 0x7fa1640c5d60 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 -- 172.21.15.154:0/1543989547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c5d60 msgr2=0x7fa1640c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 --2- 172.21.15.154:0/1543989547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c5d60 0x7fa1640c6170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa1640c5980 tx=0x7fa15c00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 -- 172.21.15.154:0/1543989547 shutdown_connections 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 --2- 172.21.15.154:0/1543989547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640c5d60 0x7fa1640c6170 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 --2- 172.21.15.154:0/1543989547 >> v2:172.21.15.191:3300/0 conn(0x7fa1640c4380 0x7fa16414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 -- 172.21.15.154:0/1543989547 >> 172.21.15.154:0/1543989547 conn(0x7fa1640a9bf0 msgr2=0x7fa1640a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 -- 172.21.15.154:0/1543989547 shutdown_connections 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.991+0000 7fa16a320700 1 -- 172.21.15.154:0/1543989547 wait complete. 2024-03-14T09:04:21.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.992+0000 7fa16a320700 1 Processor -- start 2024-03-14T09:04:21.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.992+0000 7fa16a320700 1 -- start start 2024-03-14T09:04:21.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640c4380 0x7fa1641e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1640c5d60 0x7fa1641e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641ebff0 con 0x7fa1640702f0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164145a70 con 0x7fa1640c5d60 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa16a320700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641ec4b0 con 0x7fa1640c4380 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33612/0 (socket says 172.21.15.154:33612) 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 -- 172.21.15.154:0/3324785490 learned_addr learned my addr 172.21.15.154:0/3324785490 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa1688bd700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1640c5d60 0x7fa1641e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa1637fe700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640c4380 0x7fa1641e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 -- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640c4380 msgr2=0x7fa1641e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640c4380 0x7fa1641e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 -- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1640c5d60 msgr2=0x7fa1641e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1640c5d60 0x7fa1641e87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.993+0000 7fa163fff700 1 -- 172.21.15.154:0/3324785490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009b30 con 0x7fa1640702f0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa163fff700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c4d0 tx=0x7fa15800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011790 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1580118f0 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1641ec1d0 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158011ca0 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.994+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16406ff20 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.995+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1641e80e0 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.995+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa15801c460 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.995+0000 7fa160ff9700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 0x7fa15006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.996+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fa158088d90 con 0x7fa1640702f0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.996+0000 7fa1637fe700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 0x7fa15006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:21.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:21.996+0000 7fa1637fe700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 0x7fa15006ccf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1641498b0 tx=0x7fa154008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:21.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.000+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa158053ec0 con 0x7fa1640702f0 2024-03-14T09:04:22.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.205+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa16406f060 con 0x7fa1640702f0 2024-03-14T09:04:22.205 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.206+0000 7fa160ff9700 1 -- 172.21.15.154:0/3324785490 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1580574e0 con 0x7fa1640702f0 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 msgr2=0x7fa15006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 0x7fa15006ccf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1641498b0 tx=0x7fa154008040 comp rx=0 tx=0).stop 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 msgr2=0x7fa164146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c4d0 tx=0x7fa15800c7e0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 shutdown_connections 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1640c5d60 0x7fa1641e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa15006a830 0x7fa15006ccf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1640702f0 0x7fa164146d30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 --2- 172.21.15.154:0/3324785490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1640c4380 0x7fa1641e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 >> 172.21.15.154:0/3324785490 conn(0x7fa1640a9bf0 msgr2=0x7fa164148d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:22.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 shutdown_connections 2024-03-14T09:04:22.209 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.208+0000 7fa16a320700 1 -- 172.21.15.154:0/3324785490 wait complete. 2024-03-14T09:04:22.209 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:04:22.220 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-14T09:04:22.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:22.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:22.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:04:22.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:22.325 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T09:04:22.357 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:22.357 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:22.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:04:22.423 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:04:22.546 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.546+0000 7f4673dfa700 1 Processor -- start 2024-03-14T09:04:22.547 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.547+0000 7f4673dfa700 1 -- start start 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4673dfa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f466c155df0 0x7f466c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4673dfa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4673dfa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0ac8e0 con 0x7f466c14eaf0 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4673dfa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c06b310 con 0x7f466c155df0 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4673dfa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c06de30 con 0x7f466c15d270 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f466c155df0 0x7f466c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f466c155df0 0x7f466c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33624/0 (socket says 172.21.15.154:33624) 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 -- 172.21.15.154:0/658234397 learned_addr learned my addr 172.21.15.154:0/658234397 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 -- 172.21.15.154:0/658234397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 msgr2=0x7f466c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:22.549 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 --2- 172.21.15.154:0/658234397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 -- 172.21.15.154:0/658234397 >> v1:172.21.15.191:6789/0 conn(0x7f466c14eaf0 legacy=0x7f466c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 -- 172.21.15.154:0/658234397 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0acb30 con 0x7f466c155df0 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4671395700 1 --2- 172.21.15.154:0/658234397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c156250 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f46600098f0 tx=0x7f4660009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50ec088a2ccebfd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4670b94700 1 -- 172.21.15.154:0/658234397 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660004230 con 0x7f466c155df0 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4670b94700 1 -- 172.21.15.154:0/658234397 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660004390 con 0x7f466c155df0 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.548+0000 7f4670b94700 1 -- 172.21.15.154:0/658234397 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46600036a0 con 0x7f466c155df0 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 -- 172.21.15.154:0/658234397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 msgr2=0x7f466c156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 --2- 172.21.15.154:0/658234397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c156250 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f46600098f0 tx=0x7f4660009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 -- 172.21.15.154:0/658234397 shutdown_connections 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 --2- 172.21.15.154:0/658234397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 --2- 172.21.15.154:0/658234397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c156250 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 -- 172.21.15.154:0/658234397 >> 172.21.15.154:0/658234397 conn(0x7f466c0c3880 msgr2=0x7f466c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:22.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 -- 172.21.15.154:0/658234397 shutdown_connections 2024-03-14T09:04:22.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.549+0000 7f4673dfa700 1 -- 172.21.15.154:0/658234397 wait complete. 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.550+0000 7f4673dfa700 1 Processor -- start 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.550+0000 7f4673dfa700 1 -- start start 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f466c14eaf0 0x7f466c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1eb0e0 con 0x7f466c155df0 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c1eb610 con 0x7f466c14eaf0 2024-03-14T09:04:22.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4673dfa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c13f230 con 0x7f466c15d270 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4671395700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4671395700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33632/0 (socket says 172.21.15.154:33632) 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.551+0000 7f4671395700 1 -- 172.21.15.154:0/244443285 learned_addr learned my addr 172.21.15.154:0/244443285 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671b96700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f466c14eaf0 0x7f466c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4672397700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 -- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 msgr2=0x7f466c0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c0c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 -- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f466c14eaf0 msgr2=0x7f466c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f466c14eaf0 0x7f466c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 -- 172.21.15.154:0/244443285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660009550 con 0x7f466c155df0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4672397700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4671395700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4660009130 tx=0x7f4660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000b660 con 0x7f466c155df0 2024-03-14T09:04:22.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000b7c0 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c142b40 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000bb70 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.552+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c154150 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.553+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c13f3a0 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.553+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4660021070 con 0x7f466c155df0 2024-03-14T09:04:22.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.554+0000 7f465e7fc700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 0x7f465006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.554+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f466008a840 con 0x7f466c155df0 2024-03-14T09:04:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.554+0000 7f4671b96700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 0x7f465006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:22.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.554+0000 7f4671b96700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 0x7f465006ccf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f466c15d240 tx=0x7f4668009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:22.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.558+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4660055970 con 0x7f466c155df0 2024-03-14T09:04:22.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.797+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f466c06e050 con 0x7f466c155df0 2024-03-14T09:04:22.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.798+0000 7f465e7fc700 1 -- 172.21.15.154:0/244443285 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+149 (secure 0 0 0) 0x7f46600290b0 con 0x7f466c155df0 2024-03-14T09:04:22.798 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:04:22.798 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 msgr2=0x7f465006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 0x7f465006ccf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f466c15d240 tx=0x7f4668009040 comp rx=0 tx=0).stop 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 msgr2=0x7f466c153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4660009130 tx=0x7f4660004750 comp rx=0 tx=0).stop 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 shutdown_connections 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f466c15d270 0x7f466c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f466c14eaf0 0x7f466c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f465006a830 0x7f465006ccf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 --2- 172.21.15.154:0/244443285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f466c155df0 0x7f466c153c10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:22.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 >> 172.21.15.154:0/244443285 conn(0x7f466c0c3880 msgr2=0x7f466c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:22.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 shutdown_connections 2024-03-14T09:04:22.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:22.800+0000 7f4673dfa700 1 -- 172.21.15.154:0/244443285 wait complete. 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:04:22.812 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.ajJGAroN3h 2024-03-14T09:04:22.812 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:04:22.829 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.TxxQ3Q8XqY 2024-03-14T09:04:22.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /tmp/tmp.ajJGAroN3h /tmp/tmp.TxxQ3Q8XqY 2024-03-14T09:04:22.897 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.TxxQ3Q8XqY 2024-03-14T09:04:23.023 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.TxxQ3Q8XqY is 226B 2024-03-14T09:04:23.045 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.TxxQ3Q8XqY 2024-03-14T09:04:23.062 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg== caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" caps mon = "allow r" caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-03-14T09:04:23.062 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:04:23.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.062 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2024-03-14T09:04:23.118 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:23.119 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42196sec preferred_lft 42196sec 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::cc9:6ff1:d3a7:c8f3/64 scope link noprefixroute 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::e8b6:15ff:fe52:d11f/64 scope link 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 56:08:9c:52:15:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::5408:9cff:fe52:1506/64 scope link 2024-03-14T09:04:23.120 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:23.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.121 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:04:23.184 INFO:teuthology.orchestra.run.smithi154.stdout:1 2024-03-14T09:04:23.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.186 DEBUG:teuthology.orchestra.run.smithi154:> route 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:04:23.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.246 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2024-03-14T09:04:23.246 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2024-03-14T09:04:23.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:04:23.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:04:23.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:04:23.247 DEBUG:teuthology.orchestra.run.smithi154:> ') 2024-03-14T09:04:23.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.351 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:04:23.367 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:04:23.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:04:23.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.368 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2024-03-14T09:04:23.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:23.435 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:23.491 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-14T09:04:23.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:23.492 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:23.547 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:23.547 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/tmp/tmp.ajJGAroN3h of=/dev/stdout 2024-03-14T09:04:23.604 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs=/ 2024-03-14T09:04:23.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.604 DEBUG:teuthology.orchestra.run.smithi154:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:23.691 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:04:23.711 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:23.712 INFO:teuthology.orchestra.run.smithi154.stdout:keyring.get_secret failed 2024-03-14T09:04:23.730 INFO:teuthology.orchestra.run.smithi154.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs=/ 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:04:23.731 INFO:teuthology.orchestra.run.smithi154.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,key=testuser,mds_namespace=cephfs,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:04:23.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:04:23.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.733 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:23.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:23.760 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:04:23.832 INFO:teuthology.orchestra.run.smithi154.stdout:client5753 2024-03-14T09:04:23.832 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:23.832 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:23.890 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:23.890 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:23.912 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:23.912 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:23.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:24.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.093+0000 7f30c72d6700 1 Processor -- start 2024-03-14T09:04:24.093 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.093+0000 7f30c72d6700 1 -- start start 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f30c00a8540 0x7f30c013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30c00c3250 0x7f30c0154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c01550b0 0x7f30c00c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c013ade0 con 0x7f30c00c3250 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c006fb30 con 0x7f30c00a8540 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c72d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c006ec30 con 0x7f30c01550b0 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30c00c3250 0x7f30c0154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30c00c3250 0x7f30c0154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33636/0 (socket says 172.21.15.154:33636) 2024-03-14T09:04:24.096 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 -- 172.21.15.154:0/3876148844 learned_addr learned my addr 172.21.15.154:0/3876148844 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c5873700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c01550b0 0x7f30c00c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c5072700 1 --2- 172.21.15.154:0/3876148844 >> v2:172.21.15.191:3300/0 conn(0x7f30c00a8540 0x7f30c013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 -- 172.21.15.154:0/3876148844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c01550b0 msgr2=0x7f30c00c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c01550b0 0x7f30c00c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 -- 172.21.15.154:0/3876148844 >> v2:172.21.15.191:3300/0 conn(0x7f30c00a8540 msgr2=0x7f30c013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 --2- 172.21.15.154:0/3876148844 >> v2:172.21.15.191:3300/0 conn(0x7f30c00a8540 0x7f30c013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 -- 172.21.15.154:0/3876148844 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c006ef10 con 0x7f30c00c3250 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30c4871700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c0154b70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f30c0070da0 tx=0x7f30b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faa9df0562617ee server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30b7fff700 1 -- 172.21.15.154:0/3876148844 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000b870 con 0x7f30c00c3250 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30b7fff700 1 -- 172.21.15.154:0/3876148844 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b000b9d0 con 0x7f30c00c3250 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.094+0000 7f30b7fff700 1 -- 172.21.15.154:0/3876148844 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000bd60 con 0x7f30c00c3250 2024-03-14T09:04:24.097 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 -- 172.21.15.154:0/3876148844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 msgr2=0x7f30c0154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c0154b70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f30c0070da0 tx=0x7f30b0009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 -- 172.21.15.154:0/3876148844 shutdown_connections 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c01550b0 0x7f30c00c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 --2- 172.21.15.154:0/3876148844 >> v2:172.21.15.191:3300/0 conn(0x7f30c00a8540 0x7f30c013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 --2- 172.21.15.154:0/3876148844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c0154b70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 -- 172.21.15.154:0/3876148844 >> 172.21.15.154:0/3876148844 conn(0x7f30c00c3880 msgr2=0x7f30c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 -- 172.21.15.154:0/3876148844 shutdown_connections 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.095+0000 7f30c72d6700 1 -- 172.21.15.154:0/3876148844 wait complete. 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.096+0000 7f30c72d6700 1 Processor -- start 2024-03-14T09:04:24.098 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 -- start start 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 0x7f30c0138540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 0x7f30c0151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c00aa4c0 con 0x7f30c00c3250 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01dd100 con 0x7f30c01550b0 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.097+0000 7f30c72d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01ec2f0 con 0x7f30c00a8540 2024-03-14T09:04:24.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33642/0 (socket says 172.21.15.154:33642) 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 -- 172.21.15.154:0/1841012674 learned_addr learned my addr 172.21.15.154:0/1841012674 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c5873700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 0x7f30c0151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c5072700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 0x7f30c0138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 -- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 msgr2=0x7f30c0138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 0x7f30c0138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 -- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 msgr2=0x7f30c0151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 0x7f30c0151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 -- 172.21.15.154:0/1841012674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0009550 con 0x7f30c00c3250 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c5873700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 0x7f30c0151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c5072700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 0x7f30c0138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c4871700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f30b0000c00 tx=0x7f30b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0004540 con 0x7f30c00c3250 2024-03-14T09:04:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b00036a0 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c00a9e70 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0003a50 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.098+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c0138a80 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.099+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c0067970 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.100+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f30b0021070 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.100+0000 7f30b5ffb700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 0x7f30a406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.100+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f30b008a6b0 con 0x7f30c00c3250 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.100+0000 7f30c5072700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 0x7f30a406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.101+0000 7f30c5072700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 0x7f30a406ccf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f30c0150500 tx=0x7f30bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.104 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.104+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f30b0055900 con 0x7f30c00c3250 2024-03-14T09:04:24.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.299+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30c00c2080 con 0x7f30c00c3250 2024-03-14T09:04:24.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.300+0000 7f30b5ffb700 1 -- 172.21.15.154:0/1841012674 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f30b0058f20 con 0x7f30c00c3250 2024-03-14T09:04:24.300 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:24.300 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 msgr2=0x7f30a406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 0x7f30a406ccf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f30c0150500 tx=0x7f30bc008040 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 msgr2=0x7f30c01335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f30b0000c00 tx=0x7f30b0004750 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 shutdown_connections 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30c01550b0 0x7f30c0151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30c00a8540 0x7f30c0138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f30a406a830 0x7f30a406ccf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 --2- 172.21.15.154:0/1841012674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30c00c3250 0x7f30c01335f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 >> 172.21.15.154:0/1841012674 conn(0x7f30c00c3880 msgr2=0x7f30c00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 shutdown_connections 2024-03-14T09:04:24.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.303+0000 7f30c72d6700 1 -- 172.21.15.154:0/1841012674 wait complete. 2024-03-14T09:04:24.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:24.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.443+0000 7fd16c3ba700 1 Processor -- start 2024-03-14T09:04:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.443+0000 7fd16c3ba700 1 -- start start 2024-03-14T09:04:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16c3ba700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1640c1e90 0x7fd16413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16c3ba700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16413ade0 con 0x7fd1640a8540 2024-03-14T09:04:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16c3ba700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16406fb30 con 0x7fd1640c3250 2024-03-14T09:04:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16c3ba700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16406ec30 con 0x7fd1640c1e90 2024-03-14T09:04:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1640c1e90 0x7fd16413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1640c1e90 0x7fd16413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33656/0 (socket says 172.21.15.154:33656) 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 -- 172.21.15.154:0/2366335894 learned_addr learned my addr 172.21.15.154:0/2366335894 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a156700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1640a8540 0x7fd164154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43144/0 (socket says 172.21.15.154:43144) 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 -- 172.21.15.154:0/2366335894 >> v1:172.21.15.191:6789/0 conn(0x7fd1640c3250 legacy=0x7fd1640c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 -- 172.21.15.154:0/2366335894 >> v1:172.21.15.154:6789/0 conn(0x7fd1640a8540 legacy=0x7fd164154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd169154700 1 -- 172.21.15.154:0/2366335894 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069504530 0 0) 0x7fd160001110 con 0x7fd1640a8540 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 -- 172.21.15.154:0/2366335894 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16406ef10 con 0x7fd1640c1e90 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd16a957700 1 --2- 172.21.15.154:0/2366335894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c1e90 0x7fd16413a8a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009e50 tx=0x7fd15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb828495ddc1291 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd169154700 1 -- 172.21.15.154:0/2366335894 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00e070 con 0x7fd1640c1e90 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.444+0000 7fd169154700 1 -- 172.21.15.154:0/2366335894 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c00b9c0 con 0x7fd1640c1e90 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd169154700 1 -- 172.21.15.154:0/2366335894 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00bd70 con 0x7fd1640c1e90 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 -- 172.21.15.154:0/2366335894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c1e90 msgr2=0x7fd16413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/2366335894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c1e90 0x7fd16413a8a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009e50 tx=0x7fd15c00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 -- 172.21.15.154:0/2366335894 shutdown_connections 2024-03-14T09:04:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/2366335894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c1e90 0x7fd16413a8a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 -- 172.21.15.154:0/2366335894 >> 172.21.15.154:0/2366335894 conn(0x7fd1640c3880 msgr2=0x7fd1640c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 -- 172.21.15.154:0/2366335894 shutdown_connections 2024-03-14T09:04:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.445+0000 7fd16c3ba700 1 -- 172.21.15.154:0/2366335894 wait complete. 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.446+0000 7fd16c3ba700 1 Processor -- start 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.446+0000 7fd16c3ba700 1 -- start start 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1640a8540 0x7fd16414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1640c1e90 0x7fd164151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641550d0 con 0x7fd1640c3250 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16414fae0 con 0x7fd1640c1e90 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16c3ba700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641eb390 con 0x7fd1640a8540 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33668/0 (socket says 172.21.15.154:33668) 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 -- 172.21.15.154:0/376230747 learned_addr learned my addr 172.21.15.154:0/376230747 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a156700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1640a8540 0x7fd16414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd169955700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1640c1e90 0x7fd164151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 -- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1640a8540 msgr2=0x7fd16414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1640a8540 0x7fd16414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 -- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1640c1e90 msgr2=0x7fd164151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1640c1e90 0x7fd164151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.447+0000 7fd16a957700 1 -- 172.21.15.154:0/376230747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c009b30 con 0x7fd1640c3250 2024-03-14T09:04:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd16a957700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b730 tx=0x7fd15c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01c640 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c01c7c0 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1641eb670 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c01cb70 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.448+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16414f770 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.449+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16406d6d0 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.449+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd15c01ccd0 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.449+0000 7fd15affd700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 0x7fd14806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.449+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fd15c08a9e0 con 0x7fd1640c3250 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.450+0000 7fd16a156700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 0x7fd14806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.450+0000 7fd16a156700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 0x7fd14806ccf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd160007c40 tx=0x7fd160007b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.454 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.454+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fd15c055c30 con 0x7fd1640c3250 2024-03-14T09:04:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.645+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1640697d0 con 0x7fd1640c3250 2024-03-14T09:04:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.646+0000 7fd15affd700 1 -- 172.21.15.154:0/376230747 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd15c059250 con 0x7fd1640c3250 2024-03-14T09:04:24.646 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 msgr2=0x7fd14806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 0x7fd14806ccf0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd160007c40 tx=0x7fd160007b90 comp rx=0 tx=0).stop 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 msgr2=0x7fd1640ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b730 tx=0x7fd15c004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 shutdown_connections 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd14806a830 0x7fd14806ccf0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.648 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1640c3250 0x7fd1640ab3b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd1640a8540 0x7fd16414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 --2- 172.21.15.154:0/376230747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1640c1e90 0x7fd164151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 >> 172.21.15.154:0/376230747 conn(0x7fd1640c3880 msgr2=0x7fd1640c2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 shutdown_connections 2024-03-14T09:04:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.648+0000 7fd16c3ba700 1 -- 172.21.15.154:0/376230747 wait complete. 2024-03-14T09:04:24.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:24.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.783+0000 7f2fbf579700 1 Processor -- start 2024-03-14T09:04:24.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.784+0000 7f2fbf579700 1 -- start start 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.784+0000 7f2fbf579700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fb815e280 0x7f2fb8160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.784+0000 7f2fbf579700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80ac8e0 con 0x7f2fb8155df0 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.784+0000 7f2fbf579700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb806de30 con 0x7f2fb814eaf0 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.784+0000 7f2fbf579700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb806b310 con 0x7f2fb815e280 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbcb14700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2fb8155df0 0x7f2fb8156250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43148/0 (socket says 172.21.15.154:43148) 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbcb14700 1 -- 172.21.15.154:0/1814253432 learned_addr learned my addr 172.21.15.154:0/1814253432 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:24.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbdb16700 1 --2- 172.21.15.154:0/1814253432 >> v2:172.21.15.191:3300/0 conn(0x7f2fb815e280 0x7f2fb8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790394422 0 0) 0x7f2fb80ac8e0 con 0x7f2fb8155df0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa40035b0 con 0x7f2fb8155df0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3934136843 0 0) 0x7f2fa40035b0 con 0x7f2fb8155df0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 >> v2:172.21.15.191:3300/0 conn(0x7f2fb815e280 msgr2=0x7f2fb8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 --2- 172.21.15.154:0/1814253432 >> v2:172.21.15.191:3300/0 conn(0x7f2fb815e280 0x7f2fb8160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 >> v1:172.21.15.191:6789/0 conn(0x7f2fb814eaf0 legacy=0x7f2fb806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8156960 con 0x7f2fb8155df0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f2fb4001670 con 0x7f2fb8155df0 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbdb16700 1 --2- 172.21.15.154:0/1814253432 >> v2:172.21.15.191:3300/0 conn(0x7f2fb815e280 0x7f2fb8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 >> v1:172.21.15.154:6789/0 conn(0x7f2fb8155df0 legacy=0x7f2fb8156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fa4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa4004850 0x7f2fa4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa4005330 0x7f2fa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb806b310 con 0x7f2fa4003a00 2024-03-14T09:04:24.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa400c200 con 0x7f2fa4005330 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa400c3c0 con 0x7f2fa4004850 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbd315700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fa4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbdb16700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa4005330 0x7f2fa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.785+0000 7f2fbcb14700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa4004850 0x7f2fa4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 -- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa4004850 msgr2=0x7f2fa4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa4004850 0x7f2fa4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 -- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa4005330 msgr2=0x7f2fa400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa4005330 0x7f2fa400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 -- 172.21.15.154:0/1814253432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40035b0 con 0x7f2fa4003a00 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbdb16700 1 -- 172.21.15.154:0/1814253432 reap_dead start 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbd315700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fa4003e20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007fd0 tx=0x7f2fa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62daf3fb9e25e6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0052d0 con 0x7f2fa4003a00 2024-03-14T09:04:24.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac005450 con 0x7f2fa4003a00 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fabfff700 1 -- 172.21.15.154:0/1814253432 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0038c0 con 0x7f2fa4003a00 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbf579700 1 -- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 msgr2=0x7f2fa4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbf579700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fa4003e20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007fd0 tx=0x7f2fa4005880 comp rx=0 tx=0).stop 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbf579700 1 -- 172.21.15.154:0/1814253432 shutdown_connections 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbf579700 1 --2- 172.21.15.154:0/1814253432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fa4003e20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.786+0000 7f2fbf579700 1 -- 172.21.15.154:0/1814253432 >> 172.21.15.154:0/1814253432 conn(0x7f2fb80c3880 msgr2=0x7f2fb80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.787+0000 7f2fbf579700 1 -- 172.21.15.154:0/1814253432 shutdown_connections 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.787+0000 7f2fbf579700 1 -- 172.21.15.154:0/1814253432 wait complete. 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.788+0000 7f2fbf579700 1 Processor -- start 2024-03-14T09:04:24.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.788+0000 7f2fbf579700 1 -- start start 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fb8149610 0x7f2fb81ea450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb81ea990 0x7f2fb81445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81ee2f0 con 0x7f2fa4003a00 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81eae20 con 0x7f2fb81ea990 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbf579700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8149af0 con 0x7f2fb8149610 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33678/0 (socket says 172.21.15.154:33678) 2024-03-14T09:04:24.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 -- 172.21.15.154:0/2640023868 learned_addr learned my addr 172.21.15.154:0/2640023868 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbcb14700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fb8149610 0x7f2fb81ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbdb16700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb81ea990 0x7f2fb81445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 -- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fb8149610 msgr2=0x7f2fb81ea450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fb8149610 0x7f2fb81ea450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 -- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb81ea990 msgr2=0x7f2fb81445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb81ea990 0x7f2fb81445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 -- 172.21.15.154:0/2640023868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb81ed9c0 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.789+0000 7f2fbd315700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005040 tx=0x7f2fac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.790+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac001290 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.790+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac007040 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.790+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb81edca0 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.790+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0073f0 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.790+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb8147360 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.791+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb80697d0 con 0x7f2fa4003a00 2024-03-14T09:04:24.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.791+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2fac007550 con 0x7f2fa4003a00 2024-03-14T09:04:24.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.792+0000 7f2fa9ffb700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 0x7f2f9c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:24.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.792+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f2fac089c60 con 0x7f2fa4003a00 2024-03-14T09:04:24.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.792+0000 7f2fbcb14700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 0x7f2f9c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:24.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.792+0000 7f2fbcb14700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 0x7f2f9c06ccf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb4000fe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:24.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.796+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2fac054d90 con 0x7f2fa4003a00 2024-03-14T09:04:24.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.990+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fb806b470 con 0x7f2fa4003a00 2024-03-14T09:04:24.990 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.990+0000 7f2fa9ffb700 1 -- 172.21.15.154:0/2640023868 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fac0210a0 con 0x7f2fa4003a00 2024-03-14T09:04:24.990 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:24.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 msgr2=0x7f2f9c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 0x7f2f9c06ccf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4000c00 tx=0x7f2fb4000fe0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 msgr2=0x7f2fb8146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:24.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005040 tx=0x7f2fac001120 comp rx=0 tx=0).stop 2024-03-14T09:04:24.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 shutdown_connections 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fb81ea990 0x7f2fb81445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2f9c06a830 0x7f2f9c06ccf0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa4003a00 0x7f2fb8146e20 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 --2- 172.21.15.154:0/2640023868 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fb8149610 0x7f2fb81ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 >> 172.21.15.154:0/2640023868 conn(0x7f2fb80c3880 msgr2=0x7f2fb815d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 shutdown_connections 2024-03-14T09:04:24.993 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:24.992+0000 7f2fbf579700 1 -- 172.21.15.154:0/2640023868 wait complete. 2024-03-14T09:04:25.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:25.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.127+0000 7f6a3be82700 1 Processor -- start 2024-03-14T09:04:25.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.127+0000 7f6a3be82700 1 -- start start 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3be82700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 0x7f6a3406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3be82700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a3406acb0 0x7f6a34155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3be82700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a340ac8e0 con 0x7f6a34156330 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3be82700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3406b310 con 0x7f6a3406acb0 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3be82700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3406de30 con 0x7f6a3414eaf0 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3941d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a3406acb0 0x7f6a34155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3941d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a3406acb0 0x7f6a34155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33692/0 (socket says 172.21.15.154:33692) 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.128+0000 7f6a3941d700 1 -- 172.21.15.154:0/1816561384 learned_addr learned my addr 172.21.15.154:0/1816561384 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/1816561384 >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 0x7f6a3406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3941d700 1 -- 172.21.15.154:0/1816561384 >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 msgr2=0x7f6a3406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3941d700 1 --2- 172.21.15.154:0/1816561384 >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 0x7f6a3406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3941d700 1 -- 172.21.15.154:0/1816561384 >> v1:172.21.15.191:6789/0 conn(0x7f6a34156330 legacy=0x7f6a34157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3941d700 1 -- 172.21.15.154:0/1816561384 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340acb30 con 0x7f6a3406acb0 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/1816561384 >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 0x7f6a3406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3941d700 1 --2- 172.21.15.154:0/1816561384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a3406acb0 0x7f6a34155df0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6a34070da0 tx=0x7f6a30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633400da49797b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a38c1c700 1 -- 172.21.15.154:0/1816561384 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004230 con 0x7f6a3406acb0 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a38c1c700 1 -- 172.21.15.154:0/1816561384 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30004390 con 0x7f6a3406acb0 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a38c1c700 1 -- 172.21.15.154:0/1816561384 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a300036a0 con 0x7f6a3406acb0 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3be82700 1 -- 172.21.15.154:0/1816561384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a3406acb0 msgr2=0x7f6a34155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.129+0000 7f6a3be82700 1 --2- 172.21.15.154:0/1816561384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a3406acb0 0x7f6a34155df0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6a34070da0 tx=0x7f6a30009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 -- 172.21.15.154:0/1816561384 shutdown_connections 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 --2- 172.21.15.154:0/1816561384 >> v2:172.21.15.191:3300/0 conn(0x7f6a3414eaf0 0x7f6a3406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 --2- 172.21.15.154:0/1816561384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a3406acb0 0x7f6a34155df0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 -- 172.21.15.154:0/1816561384 >> 172.21.15.154:0/1816561384 conn(0x7f6a340c3880 msgr2=0x7f6a340c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 -- 172.21.15.154:0/1816561384 shutdown_connections 2024-03-14T09:04:25.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.130+0000 7f6a3be82700 1 -- 172.21.15.154:0/1816561384 wait complete. 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.131+0000 7f6a3be82700 1 Processor -- start 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.131+0000 7f6a3be82700 1 -- start start 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 0x7f6a3414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 0x7f6a341ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341def60 con 0x7f6a34156330 2024-03-14T09:04:25.134 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341de9a0 con 0x7f6a3406acb0 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3be82700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341dec80 con 0x7f6a3414eaf0 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33700/0 (socket says 172.21.15.154:33700) 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 -- 172.21.15.154:0/2718501635 learned_addr learned my addr 172.21.15.154:0/2718501635 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3941d700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 0x7f6a341ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 0x7f6a3414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 -- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 msgr2=0x7f6a341ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 0x7f6a341ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 -- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 msgr2=0x7f6a3414a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 0x7f6a3414a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3a41f700 1 -- 172.21.15.154:0/2718501635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009550 con 0x7f6a34156330 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 0x7f6a3414a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.132+0000 7f6a3941d700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 0x7f6a341ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:25.135 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a3a41f700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00cc10 tx=0x7f6a2c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00abb0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00ad10 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c012660 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a341edda0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.133+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a34154ac0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.134+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a341edfe0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.134+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6a2c0127c0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.135+0000 7f6a2a7fc700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 0x7f6a1806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.135+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f6a2c0898c0 con 0x7f6a34156330 2024-03-14T09:04:25.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.135+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 0x7f6a1806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.137 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.135+0000 7f6a39c1e700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 0x7f6a1806ccf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6a24006fd0 tx=0x7f6a24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.139+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6a341edfe0 con 0x7f6a34156330 2024-03-14T09:04:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.331+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a341edfe0 con 0x7f6a34156330 2024-03-14T09:04:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.332+0000 7f6a2a7fc700 1 -- 172.21.15.154:0/2718501635 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a341edfe0 con 0x7f6a34156330 2024-03-14T09:04:25.332 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 msgr2=0x7f6a1806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 0x7f6a1806ccf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6a24006fd0 tx=0x7f6a24008040 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 msgr2=0x7f6a341ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00cc10 tx=0x7f6a2c00a060 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 shutdown_connections 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6a1806a830 0x7f6a1806ccf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a34156330 0x7f6a341ee800 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a3406acb0 0x7f6a3414a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 --2- 172.21.15.154:0/2718501635 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a3414eaf0 0x7f6a341ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 >> 172.21.15.154:0/2718501635 conn(0x7f6a340c3880 msgr2=0x7f6a3414d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 shutdown_connections 2024-03-14T09:04:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.334+0000 7f6a3be82700 1 -- 172.21.15.154:0/2718501635 wait complete. 2024-03-14T09:04:25.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:25.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 Processor -- start 2024-03-14T09:04:25.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 -- start start 2024-03-14T09:04:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f5415e280 0x7f7f54160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540ac8e0 con 0x7f7f54155df0 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5406de30 con 0x7f7f5414eaf0 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.473+0000 7f7f5914c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5406b310 con 0x7f7f5415e280 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f5415e280 0x7f7f54160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f5415e280 0x7f7f54160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33710/0 (socket says 172.21.15.154:33710) 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 -- 172.21.15.154:0/2016480915 learned_addr learned my addr 172.21.15.154:0/2016480915 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7f54155df0 0x7f7f54156250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43156/0 (socket says 172.21.15.154:43156) 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 -- 172.21.15.154:0/2016480915 >> v1:172.21.15.191:6789/0 conn(0x7f7f5414eaf0 legacy=0x7f7f5406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 -- 172.21.15.154:0/2016480915 >> v1:172.21.15.154:6789/0 conn(0x7f7f54155df0 legacy=0x7f7f54156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 -- 172.21.15.154:0/2016480915 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54156960 con 0x7f7f5415e280 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f51d9b700 1 -- 172.21.15.154:0/2016480915 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172074837 0 0) 0x7f7f540ac8e0 con 0x7f7f54155df0 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f5359e700 1 --2- 172.21.15.154:0/2016480915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5415e280 0x7f7f54160670 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009e50 tx=0x7f7f4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ded572e9a38a0f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f51d9b700 1 -- 172.21.15.154:0/2016480915 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00e070 con 0x7f7f5415e280 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f51d9b700 1 -- 172.21.15.154:0/2016480915 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c003a20 con 0x7f7f5415e280 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.474+0000 7f7f51d9b700 1 -- 172.21.15.154:0/2016480915 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c003dd0 con 0x7f7f5415e280 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 -- 172.21.15.154:0/2016480915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5415e280 msgr2=0x7f7f54160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 --2- 172.21.15.154:0/2016480915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5415e280 0x7f7f54160670 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009e50 tx=0x7f7f4c00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 -- 172.21.15.154:0/2016480915 shutdown_connections 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 --2- 172.21.15.154:0/2016480915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5415e280 0x7f7f54160670 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 -- 172.21.15.154:0/2016480915 >> 172.21.15.154:0/2016480915 conn(0x7f7f540c3880 msgr2=0x7f7f540c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 -- 172.21.15.154:0/2016480915 shutdown_connections 2024-03-14T09:04:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.475+0000 7f7f5914c700 1 -- 172.21.15.154:0/2016480915 wait complete. 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.476+0000 7f7f5914c700 1 Processor -- start 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.476+0000 7f7f5914c700 1 -- start start 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f54155df0 0x7f7f54157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 0x7f7f5415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54157610 con 0x7f7f5414eaf0 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54144540 con 0x7f7f54155df0 2024-03-14T09:04:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5914c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54144820 con 0x7f7f5415e280 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33712/0 (socket says 172.21.15.154:33712) 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 -- 172.21.15.154:0/4225173584 learned_addr learned my addr 172.21.15.154:0/4225173584 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5259c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f54155df0 0x7f7f54157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5359e700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 0x7f7f5415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 -- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 msgr2=0x7f7f5415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 0x7f7f5415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 -- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f54155df0 msgr2=0x7f7f54157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f54155df0 0x7f7f54157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 -- 172.21.15.154:0/4225173584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c009b30 con 0x7f7f5414eaf0 2024-03-14T09:04:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f5359e700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 0x7f7f5415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f52d9d700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c4d0 tx=0x7f7f4400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.477+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44011790 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.478+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f440118f0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.478+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44011ca0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.478+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f54158720 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.478+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f54158ce0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.478+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f540697d0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.479+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7f4401c460 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.479+0000 7f7f3b7fe700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 0x7f7f4006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.479+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f7f44088ec0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.480+0000 7f7f5259c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 0x7f7f4006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.480+0000 7f7f5259c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 0x7f7f4006ccf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f54154860 tx=0x7f7f48007be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.484+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7f44053ff0 con 0x7f7f5414eaf0 2024-03-14T09:04:25.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.677+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f5406b470 con 0x7f7f5414eaf0 2024-03-14T09:04:25.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.678+0000 7f7f3b7fe700 1 -- 172.21.15.154:0/4225173584 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7f7f44057610 con 0x7f7f5414eaf0 2024-03-14T09:04:25.678 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 msgr2=0x7f7f4006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 0x7f7f4006ccf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7f54154860 tx=0x7f7f48007be0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 msgr2=0x7f7f540c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400c4d0 tx=0x7f7f4400c890 comp rx=0 tx=0).stop 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 shutdown_connections 2024-03-14T09:04:25.680 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f5415e280 0x7f7f5415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7f4006a830 0x7f7f4006ccf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f5414eaf0 0x7f7f540c2730 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 --2- 172.21.15.154:0/4225173584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f54155df0 0x7f7f54157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 >> 172.21.15.154:0/4225173584 conn(0x7f7f540c3880 msgr2=0x7f7f5406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 shutdown_connections 2024-03-14T09:04:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.680+0000 7f7f5914c700 1 -- 172.21.15.154:0/4225173584 wait complete. 2024-03-14T09:04:25.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:25.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.819+0000 7f58ac491700 1 Processor -- start 2024-03-14T09:04:25.819 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.819+0000 7f58ac491700 1 -- start start 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58ac491700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 0x7f58a4158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58ac491700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f58a40c1d50 0x7f58a40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58ac491700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a4154f40 con 0x7f58a40bf9c0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58ac491700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a4158d80 con 0x7f58a40c1d50 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58ac491700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a4154930 con 0x7f58a41553d0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58aaa2e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f58a40bf9c0 0x7f58a40c4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43166/0 (socket says 172.21.15.154:43166) 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58aaa2e700 1 -- 172.21.15.154:0/3774952196 learned_addr learned my addr 172.21.15.154:0/3774952196 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58aa22d700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 0x7f58a4158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/3774952196 >> v2:172.21.15.191:3300/0 conn(0x7f58a40c1d50 0x7f58a40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.820+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818382981 0 0) 0x7f58a4154f40 con 0x7f58a40bf9c0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58900035b0 con 0x7f58a40bf9c0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569747049 0 0) 0x7f58900035b0 con 0x7f58a40bf9c0 2024-03-14T09:04:25.822 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a9a2c700 1 -- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 msgr2=0x7f58a4158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 0x7f58a4158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a9a2c700 1 -- 172.21.15.154:0/3774952196 >> v1:172.21.15.154:6789/0 conn(0x7f58a40bf9c0 legacy=0x7f58a40c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a9a2c700 1 -- 172.21.15.154:0/3774952196 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40c57e0 con 0x7f58a40c1d50 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58aa22d700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 0x7f58a4158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a40c1d50 0x7f58a40c0f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009840 tx=0x7f58a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c6dede0d8ac07c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0019f0 con 0x7f58a40c1d50 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a0004030 con 0x7f58a40c1d50 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.821+0000 7f58a922b700 1 -- 172.21.15.154:0/3774952196 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00043c0 con 0x7f58a40c1d50 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 -- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a40c1d50 msgr2=0x7f58a40c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a40c1d50 0x7f58a40c0f90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009840 tx=0x7f58a0009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 -- 172.21.15.154:0/3774952196 shutdown_connections 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a41553d0 0x7f58a4158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 --2- 172.21.15.154:0/3774952196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a40c1d50 0x7f58a40c0f90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.823 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 -- 172.21.15.154:0/3774952196 >> 172.21.15.154:0/3774952196 conn(0x7f58a406a770 msgr2=0x7f58a406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:25.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 -- 172.21.15.154:0/3774952196 shutdown_connections 2024-03-14T09:04:25.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.822+0000 7f58ac491700 1 -- 172.21.15.154:0/3774952196 wait complete. 2024-03-14T09:04:25.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.823+0000 7f58ac491700 1 Processor -- start 2024-03-14T09:04:25.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.823+0000 7f58ac491700 1 -- start start 2024-03-14T09:04:25.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a40bf9c0 0x7f58a41df870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.826 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a41553d0 0x7f58a414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41e16e0 con 0x7f58a40c1d50 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40abd00 con 0x7f58a41553d0 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40abfe0 con 0x7f58a40bf9c0 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33726/0 (socket says 172.21.15.154:33726) 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 -- 172.21.15.154:0/2758626673 learned_addr learned my addr 172.21.15.154:0/2758626673 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58aa22d700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a40bf9c0 0x7f58a41df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58aaa2e700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a41553d0 0x7f58a414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 -- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a40bf9c0 msgr2=0x7f58a41df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a40bf9c0 0x7f58a41df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 -- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a41553d0 msgr2=0x7f58a414e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a41553d0 0x7f58a414e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:25.827 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 -- 172.21.15.154:0/2758626673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940094a0 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58a9a2c700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f58a000a010 tx=0x7f58a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003cf0 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a001f070 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00094a0 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0011870 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.824+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40720e0 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.825+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a41e0110 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.826+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f58a00119d0 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.826+0000 7f589affd700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 0x7f588806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.826+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f58a0021070 con 0x7f58a40c1d50 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.826+0000 7f58aa22d700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 0x7f588806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:25.828 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.827+0000 7f58aa22d700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 0x7f588806ccf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ab690 tx=0x7f5894009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:25.830 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:25.831+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f58a0055980 con 0x7f58a40c1d50 2024-03-14T09:04:26.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.023+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f58a40697d0 con 0x7f58a40c1d50 2024-03-14T09:04:26.023 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.024+0000 7f589affd700 1 -- 172.21.15.154:0/2758626673 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7f58a0058fa0 con 0x7f58a40c1d50 2024-03-14T09:04:26.024 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 msgr2=0x7f588806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 0x7f588806ccf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ab690 tx=0x7f5894009210 comp rx=0 tx=0).stop 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 msgr2=0x7f58a4071ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f58a000a010 tx=0x7f58a0004750 comp rx=0 tx=0).stop 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 shutdown_connections 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a41553d0 0x7f58a414e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f58a40bf9c0 0x7f58a41df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.026 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f588806a830 0x7f588806ccf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 --2- 172.21.15.154:0/2758626673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a40c1d50 0x7f58a4071ba0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 >> 172.21.15.154:0/2758626673 conn(0x7f58a406a770 msgr2=0x7f58a40c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:26.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 shutdown_connections 2024-03-14T09:04:26.027 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.026+0000 7f58ac491700 1 -- 172.21.15.154:0/2758626673 wait complete. 2024-03-14T09:04:26.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:04:26.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.160+0000 7f9f0d81e700 1 Processor -- start 2024-03-14T09:04:26.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.160+0000 7f9f0d81e700 1 -- start start 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f0d81e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9f0815dd60 0x7f9f08160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f0d81e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080c3760 con 0x7f9f0806cf00 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f0d81e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080c3230 con 0x7f9f080c6490 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f0d81e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080c6210 con 0x7f9f0815dd60 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f067fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9f0806cf00 0x7f9f0806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43182/0 (socket says 172.21.15.154:43182) 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f067fc700 1 -- 172.21.15.154:0/758544788 learned_addr learned my addr 172.21.15.154:0/758544788 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917355144 0 0) 0x7f9f080c3760 con 0x7f9f0806cf00 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f077fe700 1 --2- 172.21.15.154:0/758544788 >> v2:172.21.15.191:3300/0 conn(0x7f9f0815dd60 0x7f9f08160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.161+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef00035b0 con 0x7f9f0806cf00 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4004594649 0 0) 0x7f9ef00035b0 con 0x7f9f0806cf00 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 >> v2:172.21.15.191:3300/0 conn(0x7f9f0815dd60 msgr2=0x7f9f08160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 --2- 172.21.15.154:0/758544788 >> v2:172.21.15.191:3300/0 conn(0x7f9f0815dd60 0x7f9f08160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 >> v1:172.21.15.191:6789/0 conn(0x7f9f080c6490 legacy=0x7f9f080c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.164 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080c3480 con 0x7f9f0806cf00 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9ef8001670 con 0x7f9f0806cf00 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 >> v1:172.21.15.154:6789/0 conn(0x7f9f0806cf00 legacy=0x7f9f0806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9ef0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ef0004850 0x7f9ef0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ef0005210 0x7f9ef000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080c6210 con 0x7f9ef0003a00 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef000c0b0 con 0x7f9ef0004850 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef000c270 con 0x7f9ef0005210 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9ef0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f067fc700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ef0004850 0x7f9ef0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f077fe700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ef0005210 0x7f9ef000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 -- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ef0005210 msgr2=0x7f9ef000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ef0005210 0x7f9ef000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 -- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ef0004850 msgr2=0x7f9ef0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:26.165 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ef0004850 0x7f9ef0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 -- 172.21.15.154:0/758544788 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00035b0 con 0x7f9ef0003a00 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f077fe700 1 -- 172.21.15.154:0/758544788 reap_dead start 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9ef0003e20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0017c0 tx=0x7f9ef0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc50fa905fa43438 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0052d0 con 0x7f9ef0003a00 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc005450 con 0x7f9ef0003a00 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.162+0000 7f9f05ffb700 1 -- 172.21.15.154:0/758544788 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0038c0 con 0x7f9ef0003a00 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 -- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 msgr2=0x7f9ef0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9ef0003e20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0017c0 tx=0x7f9ef0005880 comp rx=0 tx=0).stop 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 -- 172.21.15.154:0/758544788 shutdown_connections 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/758544788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9ef0003e20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 -- 172.21.15.154:0/758544788 >> 172.21.15.154:0/758544788 conn(0x7f9f080a9bf0 msgr2=0x7f9f080a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 -- 172.21.15.154:0/758544788 shutdown_connections 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.163+0000 7f9f0d81e700 1 -- 172.21.15.154:0/758544788 wait complete. 2024-03-14T09:04:26.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.164+0000 7f9f0d81e700 1 Processor -- start 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.164+0000 7f9f0d81e700 1 -- start start 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 0x7f9f081edc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 0x7f9f080aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0815a060 con 0x7f9ef0003a00 2024-03-14T09:04:26.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081533a0 con 0x7f9f0815b150 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f0d81e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f080abcf0 con 0x7f9f080a80a0 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33752/0 (socket says 172.21.15.154:33752) 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 -- 172.21.15.154:0/3059933409 learned_addr learned my addr 172.21.15.154:0/3059933409 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f067fc700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 0x7f9f081edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f077fe700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 0x7f9f080aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 -- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 msgr2=0x7f9f080aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 0x7f9f080aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 -- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 msgr2=0x7f9f081edc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 0x7f9f081edc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f06ffd700 1 -- 172.21.15.154:0/3059933409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080abfd0 con 0x7f9ef0003a00 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f077fe700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 0x7f9f080aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:26.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.165+0000 7f9f067fc700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 0x7f9f081edc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9f06ffd700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000c00 tx=0x7f9efc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc001290 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc007040 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f080ac2b0 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0073f0 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.166+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081ed740 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.167+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f080697d0 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.167+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9efc007550 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.168+0000 7f9eeffff700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 0x7f9ef406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.168+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9efc089d10 con 0x7f9ef0003a00 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.168+0000 7f9f067fc700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 0x7f9ef406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.171 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.168+0000 7f9f067fc700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 0x7f9ef406ccf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8001eb0 tx=0x7f9ef8001110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.172+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9efc054f60 con 0x7f9ef0003a00 2024-03-14T09:04:26.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:04:26.314 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:04:26.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.388+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9f080c8cc0 con 0x7f9ef0003a00 2024-03-14T09:04:26.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.388+0000 7f9eeffff700 1 -- 172.21.15.154:0/3059933409 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9efc02b080 con 0x7f9ef0003a00 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 msgr2=0x7f9ef406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 0x7f9ef406ccf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8001eb0 tx=0x7f9ef8001110 comp rx=0 tx=0).stop 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 msgr2=0x7f9f08152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000c00 tx=0x7f9efc001120 comp rx=0 tx=0).stop 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 shutdown_connections 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f080a80a0 0x7f9f080aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9ef406a830 0x7f9ef406ccf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ef0003a00 0x7f9f08152e60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.391 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 --2- 172.21.15.154:0/3059933409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f0815b150 0x7f9f081edc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 >> 172.21.15.154:0/3059933409 conn(0x7f9f080a9bf0 msgr2=0x7f9f080a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:26.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 shutdown_connections 2024-03-14T09:04:26.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.391+0000 7f9f0d81e700 1 -- 172.21.15.154:0/3059933409 wait complete. 2024-03-14T09:04:26.392 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:04:26.402 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-14T09:04:26.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:26.403 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:26.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:04:26.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:26.464 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:04:26.494 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:26.494 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:26.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:04:26.552 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-14T09:04:26.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.674+0000 7f8256830700 1 Processor -- start 2024-03-14T09:04:26.674 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.674+0000 7f8256830700 1 -- start start 2024-03-14T09:04:26.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8256830700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f825015e280 0x7f8250160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8256830700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82500ac8e0 con 0x7f825014eaf0 2024-03-14T09:04:26.676 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8256830700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825006de30 con 0x7f8250155df0 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8256830700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825006b310 con 0x7f825015e280 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8254dcd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f825015e280 0x7f8250160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f824ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f825014eaf0 0x7f825006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43192/0 (socket says 172.21.15.154:43192) 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f824ffff700 1 -- 172.21.15.154:0/2691959410 learned_addr learned my addr 172.21.15.154:0/2691959410 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8254dcd700 1 -- 172.21.15.154:0/2691959410 >> v1:172.21.15.191:6789/0 conn(0x7f8250155df0 legacy=0x7f8250156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8254dcd700 1 -- 172.21.15.154:0/2691959410 >> v1:172.21.15.154:6789/0 conn(0x7f825014eaf0 legacy=0x7f825006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f824effd700 1 -- 172.21.15.154:0/2691959410 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341624379 0 0) 0x7f82500ac8e0 con 0x7f825014eaf0 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.675+0000 7f8254dcd700 1 -- 172.21.15.154:0/2691959410 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250156960 con 0x7f825015e280 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8254dcd700 1 --2- 172.21.15.154:0/2691959410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825015e280 0x7f8250160670 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8248009d90 tx=0x7f824800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf3e707cdba91b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f824effd700 1 -- 172.21.15.154:0/2691959410 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800e070 con 0x7f825015e280 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f824effd700 1 -- 172.21.15.154:0/2691959410 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824800bb90 con 0x7f825015e280 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f824effd700 1 -- 172.21.15.154:0/2691959410 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248003a20 con 0x7f825015e280 2024-03-14T09:04:26.677 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 -- 172.21.15.154:0/2691959410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825015e280 msgr2=0x7f8250160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 --2- 172.21.15.154:0/2691959410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825015e280 0x7f8250160670 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8248009d90 tx=0x7f824800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 -- 172.21.15.154:0/2691959410 shutdown_connections 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 --2- 172.21.15.154:0/2691959410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825015e280 0x7f8250160670 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 -- 172.21.15.154:0/2691959410 >> 172.21.15.154:0/2691959410 conn(0x7f82500c3880 msgr2=0x7f82500c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 -- 172.21.15.154:0/2691959410 shutdown_connections 2024-03-14T09:04:26.678 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.676+0000 7f8256830700 1 -- 172.21.15.154:0/2691959410 wait complete. 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.677+0000 7f8256830700 1 Processor -- start 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 -- start start 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8250155df0 0x7f8250151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f825015e280 0x7f8250152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250150690 con 0x7f825014eaf0 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501436f0 con 0x7f8250155df0 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f8256830700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82501439a0 con 0x7f825015e280 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f824ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.681 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f824ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33776/0 (socket says 172.21.15.154:33776) 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.678+0000 7f824ffff700 1 -- 172.21.15.154:0/1756192604 learned_addr learned my addr 172.21.15.154:0/1756192604 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824f7fe700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8250155df0 0x7f8250151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f8254dcd700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f825015e280 0x7f8250152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 -- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f825015e280 msgr2=0x7f8250152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f825015e280 0x7f8250152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 -- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8250155df0 msgr2=0x7f8250151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8250155df0 0x7f8250151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 -- 172.21.15.154:0/1756192604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248009a70 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824ffff700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f82440011e0 tx=0x7f824400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244004c80 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244004e00 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82500c3110 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244007310 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.679+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82500c1e80 con 0x7f825014eaf0 2024-03-14T09:04:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.680+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825006e060 con 0x7f825014eaf0 2024-03-14T09:04:26.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.680+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8244007470 con 0x7f825014eaf0 2024-03-14T09:04:26.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.681+0000 7f824cff9700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 0x7f823c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:26.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.681+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f8244088c40 con 0x7f825014eaf0 2024-03-14T09:04:26.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.681+0000 7f824f7fe700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 0x7f823c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:26.683 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.681+0000 7f824f7fe700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 0x7f823c06cd20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8250066b30 tx=0x7f8240007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:26.685 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.685+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f825006e060 con 0x7f825014eaf0 2024-03-14T09:04:26.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.927+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f825006e060 con 0x7f825014eaf0 2024-03-14T09:04:26.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.928+0000 7f824cff9700 1 -- 172.21.15.154:0/1756192604 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+149 (secure 0 0 0) 0x7f825006e060 con 0x7f825014eaf0 2024-03-14T09:04:26.930 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-14T09:04:26.930 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs2, metadata pool: cephfs2_metadata, data pools: [cephfs2_data ] 2024-03-14T09:04:26.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.929+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 msgr2=0x7f823c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.929+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 0x7f823c06cd20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8250066b30 tx=0x7f8240007ec0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.929+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 msgr2=0x7f8250149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:26.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.929+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f82440011e0 tx=0x7f824400ca30 comp rx=0 tx=0).stop 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 shutdown_connections 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f825015e280 0x7f8250152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f823c06a860 0x7f823c06cd20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825014eaf0 0x7f8250149f50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 --2- 172.21.15.154:0/1756192604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8250155df0 0x7f8250151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 >> 172.21.15.154:0/1756192604 conn(0x7f82500c3880 msgr2=0x7f8250154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 shutdown_connections 2024-03-14T09:04:26.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:26.930+0000 7f8256830700 1 -- 172.21.15.154:0/1756192604 wait complete. 2024-03-14T09:04:26.941 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs2; just confirmed its presence on cluster 2024-03-14T09:04:26.941 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-14T09:04:26.941 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-03-14T09:04:26.942 INFO:tasks.cephfs.mount:self.client.name = client.testuser 2024-03-14T09:04:26.942 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:26.942 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs2 2024-03-14T09:04:26.942 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-14T09:04:26.942 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.ILqEPLP1Vm 2024-03-14T09:04:26.942 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-03-14T09:04:26.961 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.BUgs2OcZAA 2024-03-14T09:04:26.961 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /tmp/tmp.ILqEPLP1Vm /tmp/tmp.BUgs2OcZAA 2024-03-14T09:04:27.032 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.BUgs2OcZAA 2024-03-14T09:04:27.164 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.BUgs2OcZAA is 226B 2024-03-14T09:04:27.213 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.BUgs2OcZAA 2024-03-14T09:04:27.231 INFO:tasks.cephfs.mount:keyring content - [client.testuser] key = AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg== caps mds = "allow r fsname=cephfs, allow r fsname=cephfs2" caps mon = "allow r" caps osd = "allow r tag cephfs data=cephfs, allow r tag cephfs data=cephfs2" 2024-03-14T09:04:27.231 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:27.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.231 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-03-14T09:04:27.287 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42197sec preferred_lft 42197sec 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::233d:e1f2:14ba:2432/64 scope link noprefixroute 2024-03-14T09:04:27.288 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::504f:cfff:fef8:8c63/64 scope link 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:1e:25:52:17:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1e:25ff:fe52:1721/64 scope link 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-03-14T09:04:27.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.290 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-14T09:04:27.353 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-03-14T09:04:27.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.355 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-03-14T09:04:27.414 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-03-14T09:04:27.414 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-14T09:04:27.414 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-14T09:04:27.414 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-14T09:04:27.414 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-14T09:04:27.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-14T09:04:27.415 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-03-14T09:04:27.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.519 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:04:27.536 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:04:27.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:27.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.536 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-03-14T09:04:27.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:27.603 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:27.660 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-14T09:04:27.661 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-14T09:04:27.661 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:27.718 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:27.718 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/tmp/tmp.ILqEPLP1Vm of=/dev/stdout 2024-03-14T09:04:27.775 INFO:tasks.cephfs.kernel_mount:mounting using device: testuser@.cephfs2=/ 2024-03-14T09:04:27.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.775 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph testuser@.cephfs2=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:27.861 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:27.862 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:27.862 INFO:teuthology.orchestra.run.smithi191.stdout:keyring.get_secret failed 2024-03-14T09:04:27.882 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:27.882 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:27.882 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:27.883 INFO:teuthology.orchestra.run.smithi191.stdout:keyring.get_secret failed 2024-03-14T09:04:27.900 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,secret=AQCTvfJldJgWBRAAQSWZtkqFyz46bF9ydWOxjg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:3300,172.21.15.191:3300,172.21.15.191:3301" 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: testuser@661f6023-01e4-4d6e-b4b3-5b5701ce7cc5.cephfs2=/ 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,ms_mode=prefer-crc,key=testuser,mon_addr=172.21.15.154:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790" 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.154:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2024-03-14T09:04:27.901 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=testuser,key=testuser,mds_namespace=cephfs2,fsid=661f6023-01e4-4d6e-b4b3-5b5701ce7cc5" will pass to kernel 2024-03-14T09:04:27.903 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-14T09:04:27.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:27.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:27.930 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-14T09:04:28.002 INFO:teuthology.orchestra.run.smithi191.stdout:client5780 2024-03-14T09:04:28.003 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:28.003 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:28.060 INFO:tasks.cephfs.caps_helper:test read perm: read file /home/ubuntu/cephtest/mnt.0/testdir/testfile and expect data "filepath = /home/ubuntu/cephtest/mnt.0/testdir/testfile cephfs_name = cephfs cephfs_mntpt = / hostfs_mntpt = /home/ubuntu/cephtest/mnt.0" 2024-03-14T09:04:28.061 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:28.061 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:28.081 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:28.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:28.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.307+0000 7f05e225e700 1 Processor -- start 2024-03-14T09:04:28.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.307+0000 7f05e225e700 1 -- start start 2024-03-14T09:04:28.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05e225e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f05dc0c8350 0x7f05dc1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05e225e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 0x7f05dc1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05e225e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc157e80 con 0x7f05dc0c7f70 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05e225e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc158ac0 con 0x7f05dc0c8350 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05e225e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc158120 con 0x7f05dc154d30 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05db7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f05dc0c7f70 0x7f05dc0c49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43200/0 (socket says 172.21.15.154:43200) 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05db7fe700 1 -- 172.21.15.154:0/2908491227 learned_addr learned my addr 172.21.15.154:0/2908491227 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05daffd700 1 --2- 172.21.15.154:0/2908491227 >> v2:172.21.15.191:3300/0 conn(0x7f05dc0c8350 0x7f05dc1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 --2- 172.21.15.154:0/2908491227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 0x7f05dc1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05da7fc700 1 -- 172.21.15.154:0/2908491227 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601312709 0 0) 0x7f05dc157e80 con 0x7f05dc0c7f70 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05da7fc700 1 -- 172.21.15.154:0/2908491227 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c40035b0 con 0x7f05dc0c7f70 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 -- 172.21.15.154:0/2908491227 >> v2:172.21.15.191:3300/0 conn(0x7f05dc0c8350 msgr2=0x7f05dc1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 --2- 172.21.15.154:0/2908491227 >> v2:172.21.15.191:3300/0 conn(0x7f05dc0c8350 0x7f05dc1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 -- 172.21.15.154:0/2908491227 >> v1:172.21.15.154:6789/0 conn(0x7f05dc0c7f70 legacy=0x7f05dc0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 -- 172.21.15.154:0/2908491227 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc158400 con 0x7f05dc154d30 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.308+0000 7f05dbfff700 1 --2- 172.21.15.154:0/2908491227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 0x7f05dc1551c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009d90 tx=0x7f05d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5038fa44588abe30 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05da7fc700 1 -- 172.21.15.154:0/2908491227 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400e070 con 0x7f05dc154d30 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05da7fc700 1 -- 172.21.15.154:0/2908491227 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400bb20 con 0x7f05dc154d30 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05da7fc700 1 -- 172.21.15.154:0/2908491227 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4012430 con 0x7f05dc154d30 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 -- 172.21.15.154:0/2908491227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 msgr2=0x7f05dc1551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 --2- 172.21.15.154:0/2908491227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 0x7f05dc1551c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009d90 tx=0x7f05d400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 -- 172.21.15.154:0/2908491227 shutdown_connections 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 --2- 172.21.15.154:0/2908491227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc154d30 0x7f05dc1551c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 --2- 172.21.15.154:0/2908491227 >> v2:172.21.15.191:3300/0 conn(0x7f05dc0c8350 0x7f05dc1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 -- 172.21.15.154:0/2908491227 >> 172.21.15.154:0/2908491227 conn(0x7f05dc072080 msgr2=0x7f05dc072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 -- 172.21.15.154:0/2908491227 shutdown_connections 2024-03-14T09:04:28.313 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.309+0000 7f05e225e700 1 -- 172.21.15.154:0/2908491227 wait complete. 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.310+0000 7f05e225e700 1 Processor -- start 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.310+0000 7f05e225e700 1 -- start start 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 0x7f05dc0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc0c8350 0x7f05dc1eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0ae5c0 con 0x7f05dc154d30 2024-03-14T09:04:28.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc14f550 con 0x7f05dc0c7f70 2024-03-14T09:04:28.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05e225e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc0adbf0 con 0x7f05dc0c8350 2024-03-14T09:04:28.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05dbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05dbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33784/0 (socket says 172.21.15.154:33784) 2024-03-14T09:04:28.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05dbfff700 1 -- 172.21.15.154:0/1409482080 learned_addr learned my addr 172.21.15.154:0/1409482080 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05daffd700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc0c8350 0x7f05dc1eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.311+0000 7f05db7fe700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 0x7f05dc0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 -- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc0c8350 msgr2=0x7f05dc1eb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc0c8350 0x7f05dc1eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 -- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 msgr2=0x7f05dc0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 0x7f05dc0ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 -- 172.21.15.154:0/1409482080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d4009a70 con 0x7f05dc154d30 2024-03-14T09:04:28.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05db7fe700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 0x7f05dc0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:28.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05dbfff700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cb0b0 tx=0x7f05d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4012a70 con 0x7f05dc154d30 2024-03-14T09:04:28.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4020070 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc0af520 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4012e30 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.312+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc0ad880 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.313+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc0ad6e0 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.313+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f05d4004040 con 0x7f05dc154d30 2024-03-14T09:04:28.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.314+0000 7f05c3fff700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 0x7f05c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.314+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f05d4022070 con 0x7f05dc154d30 2024-03-14T09:04:28.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.314+0000 7f05db7fe700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 0x7f05c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.314+0000 7f05db7fe700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 0x7f05c806ccf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0011e0 tx=0x7f05cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.325 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.318+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f05dc0ad6e0 con 0x7f05dc154d30 2024-03-14T09:04:28.513 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.514+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05dc0ad6e0 con 0x7f05dc154d30 2024-03-14T09:04:28.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.515+0000 7f05c3fff700 1 -- 172.21.15.154:0/1409482080 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f05dc0ad6e0 con 0x7f05dc154d30 2024-03-14T09:04:28.516 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:28.516 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 msgr2=0x7f05c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 0x7f05c806ccf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0011e0 tx=0x7f05cc009040 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 msgr2=0x7f05dc148360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0cb0b0 tx=0x7f05d4004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 shutdown_connections 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f05c806a830 0x7f05c806ccf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05dc154d30 0x7f05dc148360 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05dc0c7f70 0x7f05dc0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 --2- 172.21.15.154:0/1409482080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05dc0c8350 0x7f05dc1eb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 >> 172.21.15.154:0/1409482080 conn(0x7f05dc072080 msgr2=0x7f05dc0ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:28.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 shutdown_connections 2024-03-14T09:04:28.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.518+0000 7f05e225e700 1 -- 172.21.15.154:0/1409482080 wait complete. 2024-03-14T09:04:28.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:28.530 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:28.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.655+0000 7fa42499b700 1 Processor -- start 2024-03-14T09:04:28.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.655+0000 7fa42499b700 1 -- start start 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa42499b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4201553d0 0x7fa420158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa42499b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa420154f40 con 0x7fa4200c1d50 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa42499b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa420158d80 con 0x7fa4200bf9c0 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa42499b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4200c16a0 con 0x7fa4201553d0 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41e59c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4201553d0 0x7fa420158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41dd9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa4200c1d50 0x7fa4200c0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43206/0 (socket says 172.21.15.154:43206) 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41dd9b700 1 -- 172.21.15.154:0/975695939 learned_addr learned my addr 172.21.15.154:0/975695939 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41e59c700 1 -- 172.21.15.154:0/975695939 >> v1:172.21.15.191:6789/0 conn(0x7fa4200bf9c0 legacy=0x7fa42015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41e59c700 1 -- 172.21.15.154:0/975695939 >> v1:172.21.15.154:6789/0 conn(0x7fa4200c1d50 legacy=0x7fa4200c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41e59c700 1 -- 172.21.15.154:0/975695939 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420154930 con 0x7fa4201553d0 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41d59a700 1 -- 172.21.15.154:0/975695939 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361947866 0 0) 0x7fa420154f40 con 0x7fa4200c1d50 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41e59c700 1 --2- 172.21.15.154:0/975695939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4201553d0 0x7fa420158840 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa414009840 tx=0x7fa414009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582b2db814e1c039 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41d59a700 1 -- 172.21.15.154:0/975695939 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414004230 con 0x7fa4201553d0 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41d59a700 1 -- 172.21.15.154:0/975695939 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa414004390 con 0x7fa4201553d0 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.656+0000 7fa41d59a700 1 -- 172.21.15.154:0/975695939 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4140036a0 con 0x7fa4201553d0 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 -- 172.21.15.154:0/975695939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4201553d0 msgr2=0x7fa420158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 --2- 172.21.15.154:0/975695939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4201553d0 0x7fa420158840 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa414009840 tx=0x7fa414009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 -- 172.21.15.154:0/975695939 shutdown_connections 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 --2- 172.21.15.154:0/975695939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4201553d0 0x7fa420158840 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 -- 172.21.15.154:0/975695939 >> 172.21.15.154:0/975695939 conn(0x7fa42006a770 msgr2=0x7fa42006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 -- 172.21.15.154:0/975695939 shutdown_connections 2024-03-14T09:04:28.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.657+0000 7fa42499b700 1 -- 172.21.15.154:0/975695939 wait complete. 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.658+0000 7fa42499b700 1 Processor -- start 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.658+0000 7fa42499b700 1 -- start start 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 0x7fa420157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 0x7fa420099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201ee280 con 0x7fa4200c1d50 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42009a820 con 0x7fa4200bf9c0 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa42499b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200a94b0 con 0x7fa4201553d0 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33792/0 (socket says 172.21.15.154:33792) 2024-03-14T09:04:28.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 -- 172.21.15.154:0/1281779656 learned_addr learned my addr 172.21.15.154:0/1281779656 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41e59c700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 0x7fa420157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41ed9d700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 0x7fa420099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 -- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 msgr2=0x7fa420099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 0x7fa420099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 -- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 msgr2=0x7fa420157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 0x7fa420157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 -- 172.21.15.154:0/1281779656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140094a0 con 0x7fa4200c1d50 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41e59c700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 0x7fa420157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41ed9d700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 0x7fa420099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.659+0000 7fa41dd9b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa4100011e0 tx=0x7fa41000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.660+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410004c80 con 0x7fa4200c1d50 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.660+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa410004e00 con 0x7fa4200c1d50 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.660+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201ed640 con 0x7fa4200c1d50 2024-03-14T09:04:28.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.660+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410007310 con 0x7fa4200c1d50 2024-03-14T09:04:28.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.660+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201edbc0 con 0x7fa4200c1d50 2024-03-14T09:04:28.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.661+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4201ee3c0 con 0x7fa4200c1d50 2024-03-14T09:04:28.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.661+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa410007470 con 0x7fa4200c1d50 2024-03-14T09:04:28.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.662+0000 7fa40effd700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 0x7fa40006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:28.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.662+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fa410088de0 con 0x7fa4200c1d50 2024-03-14T09:04:28.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.662+0000 7fa41e59c700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 0x7fa40006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:28.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.662+0000 7fa41e59c700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 0x7fa40006ccf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa414008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:28.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.666+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa4201ee3c0 con 0x7fa4200c1d50 2024-03-14T09:04:28.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.861+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4201ee3c0 con 0x7fa4200c1d50 2024-03-14T09:04:28.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.861+0000 7fa40effd700 1 -- 172.21.15.154:0/1281779656 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa4201ee3c0 con 0x7fa4200c1d50 2024-03-14T09:04:28.862 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.863+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 msgr2=0x7fa40006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.863+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 0x7fa40006ccf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa414008030 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.863+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 msgr2=0x7fa420099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.863+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa4100011e0 tx=0x7fa41000cae0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 shutdown_connections 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4201553d0 0x7fa420099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4200bf9c0 0x7fa420157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa40006a830 0x7fa40006ccf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 --2- 172.21.15.154:0/1281779656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4200c1d50 0x7fa420099a70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 >> 172.21.15.154:0/1281779656 conn(0x7fa42006a770 msgr2=0x7fa420153630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 shutdown_connections 2024-03-14T09:04:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:28.864+0000 7fa42499b700 1 -- 172.21.15.154:0/1281779656 wait complete. 2024-03-14T09:04:28.875 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:29.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.000+0000 7fda594b5700 1 Processor -- start 2024-03-14T09:04:29.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.000+0000 7fda594b5700 1 -- start start 2024-03-14T09:04:29.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda594b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda540aa6e0 0x7fda5406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda594b5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 0x7fda540bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda594b5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5406df70 con 0x7fda540ab730 2024-03-14T09:04:29.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda594b5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5406b6c0 con 0x7fda540aa6e0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda594b5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540aab60 con 0x7fda540bf9c0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda540aa6e0 0x7fda5406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda52ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fda540ab730 0x7fda54155260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43208/0 (socket says 172.21.15.154:43208) 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda52ffd700 1 -- 172.21.15.154:0/2511467553 learned_addr learned my addr 172.21.15.154:0/2511467553 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda537fe700 1 --2- 172.21.15.154:0/2511467553 >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 0x7fda540bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 -- 172.21.15.154:0/2511467553 >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 msgr2=0x7fda540bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 --2- 172.21.15.154:0/2511467553 >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 0x7fda540bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 -- 172.21.15.154:0/2511467553 >> v1:172.21.15.154:6789/0 conn(0x7fda540ab730 legacy=0x7fda54155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 -- 172.21.15.154:0/2511467553 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5406dc40 con 0x7fda540aa6e0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda51ffb700 1 -- 172.21.15.154:0/2511467553 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534591163 0 0) 0x7fda5406df70 con 0x7fda540ab730 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda537fe700 1 --2- 172.21.15.154:0/2511467553 >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 0x7fda540bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda527fc700 1 --2- 172.21.15.154:0/2511467553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda540aa6e0 0x7fda5406e400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fda54067550 tx=0x7fda48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0f31c7acdf21de server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda51ffb700 1 -- 172.21.15.154:0/2511467553 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48011830 con 0x7fda540aa6e0 2024-03-14T09:04:29.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda51ffb700 1 -- 172.21.15.154:0/2511467553 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48011990 con 0x7fda540aa6e0 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.001+0000 7fda51ffb700 1 -- 172.21.15.154:0/2511467553 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48011d20 con 0x7fda540aa6e0 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 -- 172.21.15.154:0/2511467553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda540aa6e0 msgr2=0x7fda5406e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 --2- 172.21.15.154:0/2511467553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda540aa6e0 0x7fda5406e400 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fda54067550 tx=0x7fda48009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 -- 172.21.15.154:0/2511467553 shutdown_connections 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 --2- 172.21.15.154:0/2511467553 >> v2:172.21.15.191:3300/0 conn(0x7fda540bf9c0 0x7fda540bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 --2- 172.21.15.154:0/2511467553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda540aa6e0 0x7fda5406e400 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 -- 172.21.15.154:0/2511467553 >> 172.21.15.154:0/2511467553 conn(0x7fda540c3090 msgr2=0x7fda540c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 -- 172.21.15.154:0/2511467553 shutdown_connections 2024-03-14T09:04:29.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.002+0000 7fda594b5700 1 -- 172.21.15.154:0/2511467553 wait complete. 2024-03-14T09:04:29.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.003+0000 7fda594b5700 1 Processor -- start 2024-03-14T09:04:29.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.003+0000 7fda594b5700 1 -- start start 2024-03-14T09:04:29.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda540aa6e0 0x7fda54154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda540bf9c0 0x7fda541ed790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540a8170 con 0x7fda541edcd0 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541eaed0 con 0x7fda540aa6e0 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda594b5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540a7af0 con 0x7fda540bf9c0 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33804/0 (socket says 172.21.15.154:33804) 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 -- 172.21.15.154:0/996022185 learned_addr learned my addr 172.21.15.154:0/996022185 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda52ffd700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda540aa6e0 0x7fda54154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda527fc700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda540bf9c0 0x7fda541ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 -- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda540bf9c0 msgr2=0x7fda541ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda540bf9c0 0x7fda541ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 -- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda540aa6e0 msgr2=0x7fda54154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda540aa6e0 0x7fda54154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.004+0000 7fda537fe700 1 -- 172.21.15.154:0/996022185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda480094a0 con 0x7fda541edcd0 2024-03-14T09:04:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda537fe700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009fd0 tx=0x7fda4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00dd80 con 0x7fda541edcd0 2024-03-14T09:04:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00f040 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda541544f0 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c003c60 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.005+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54155480 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.006+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54074120 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.006+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fda4c004030 con 0x7fda541edcd0 2024-03-14T09:04:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.007+0000 7fda3b7fe700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 0x7fda4006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.007+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fda4c0890d0 con 0x7fda541edcd0 2024-03-14T09:04:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.007+0000 7fda52ffd700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 0x7fda4006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.007+0000 7fda52ffd700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 0x7fda4006ccf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda44000f60 tx=0x7fda44007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.011+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fda4c054200 con 0x7fda541edcd0 2024-03-14T09:04:29.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.207+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda541eb080 con 0x7fda541edcd0 2024-03-14T09:04:29.208 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.208+0000 7fda3b7fe700 1 -- 172.21.15.154:0/996022185 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda541eb080 con 0x7fda541edcd0 2024-03-14T09:04:29.208 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:29.210 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 msgr2=0x7fda4006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 0x7fda4006ccf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fda44000f60 tx=0x7fda44007e70 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 msgr2=0x7fda541ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009fd0 tx=0x7fda4c00d3b0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 shutdown_connections 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fda4006a830 0x7fda4006ccf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda541edcd0 0x7fda541ee180 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda540aa6e0 0x7fda54154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 --2- 172.21.15.154:0/996022185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fda540bf9c0 0x7fda541ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 >> 172.21.15.154:0/996022185 conn(0x7fda540c3090 msgr2=0x7fda5414a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 shutdown_connections 2024-03-14T09:04:29.211 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.210+0000 7fda594b5700 1 -- 172.21.15.154:0/996022185 wait complete. 2024-03-14T09:04:29.222 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.346+0000 7ff149b4a700 1 Processor -- start 2024-03-14T09:04:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.346+0000 7ff149b4a700 1 -- start start 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff149b4a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff144160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff149b4a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440ac8e0 con 0x7ff144155df0 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff149b4a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14406de30 con 0x7ff14414eaf0 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff149b4a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406b310 con 0x7ff14415e280 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff142ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff144155df0 0x7ff144156250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57564/0 (socket says 172.21.15.154:57564) 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff142ffd700 1 -- 172.21.15.154:0/1306509397 learned_addr learned my addr 172.21.15.154:0/1306509397 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff143fff700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff144160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.350 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130679920 0 0) 0x7ff1440ac8e0 con 0x7ff144155df0 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff12c0035b0 con 0x7ff144155df0 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182878971 0 0) 0x7ff12c0035b0 con 0x7ff144155df0 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 msgr2=0x7ff144160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff144160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 >> v1:172.21.15.191:6789/0 conn(0x7ff14414eaf0 legacy=0x7ff14406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144156960 con 0x7ff144155df0 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7ff138001670 con 0x7ff144155df0 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 >> v1:172.21.15.154:6789/0 conn(0x7ff144155df0 legacy=0x7ff144156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff12c003a30 0x7ff12c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff12c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12c004d90 0x7ff12c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14406b310 con 0x7ff12c004d90 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c00bba0 con 0x7ff12c003a30 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.347+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c00bd30 con 0x7ff14415e280 2024-03-14T09:04:29.351 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12c004d90 0x7ff12c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff1437fe700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff12c003a30 0x7ff12c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff142ffd700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff12c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 -- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 msgr2=0x7ff12c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff14415e280 0x7ff12c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 -- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff12c003a30 msgr2=0x7ff12c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff12c003a30 0x7ff12c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 -- 172.21.15.154:0/1306509397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c0035b0 con 0x7ff12c004d90 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff143fff700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12c004d90 0x7ff12c005210 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e170 tx=0x7ff12c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5802172702750725 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00ec90 con 0x7ff12c004d90 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c00f040 con 0x7ff12c004d90 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.348+0000 7ff1427fc700 1 -- 172.21.15.154:0/1306509397 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c014480 con 0x7ff12c004d90 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 -- 172.21.15.154:0/1306509397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12c004d90 msgr2=0x7ff12c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 --2- 172.21.15.154:0/1306509397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12c004d90 0x7ff12c005210 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e170 tx=0x7ff12c005880 comp rx=0 tx=0).stop 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff143fff700 1 -- 172.21.15.154:0/1306509397 reap_dead start 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 -- 172.21.15.154:0/1306509397 shutdown_connections 2024-03-14T09:04:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 -- 172.21.15.154:0/1306509397 >> 172.21.15.154:0/1306509397 conn(0x7ff1440c3880 msgr2=0x7ff1440c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 -- 172.21.15.154:0/1306509397 shutdown_connections 2024-03-14T09:04:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.349+0000 7ff149b4a700 1 -- 172.21.15.154:0/1306509397 wait complete. 2024-03-14T09:04:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.350+0000 7ff149b4a700 1 Processor -- start 2024-03-14T09:04:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.350+0000 7ff149b4a700 1 -- start start 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff144152fa0 0x7ff14415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1441582a0 0x7ff144154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14414ff90 con 0x7ff14414f1f0 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144152770 con 0x7ff1441582a0 2024-03-14T09:04:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff149b4a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1441edcf0 con 0x7ff144152fa0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43394/0 (socket says 172.21.15.154:43394) 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 -- 172.21.15.154:0/724543045 learned_addr learned my addr 172.21.15.154:0/724543045 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff1437fe700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff144152fa0 0x7ff14415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff143fff700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1441582a0 0x7ff144154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 -- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff144152fa0 msgr2=0x7ff14415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff144152fa0 0x7ff14415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 -- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1441582a0 msgr2=0x7ff144154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1441582a0 0x7ff144154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 -- 172.21.15.154:0/724543045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1441ee220 con 0x7ff14414f1f0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.351+0000 7ff142ffd700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff1380072c0 tx=0x7ff1380075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.352+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138012560 con 0x7ff14414f1f0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.352+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1380126c0 con 0x7ff14414f1f0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.352+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff144153c30 con 0x7ff14414f1f0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.352+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138012a70 con 0x7ff14414f1f0 2024-03-14T09:04:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.352+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1441edf40 con 0x7ff14414f1f0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.353+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1440697d0 con 0x7ff14414f1f0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.353+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff138012bd0 con 0x7ff14414f1f0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.353+0000 7ff12bfff700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 0x7ff13006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.353+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff138088f90 con 0x7ff14414f1f0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.353+0000 7ff1437fe700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 0x7ff13006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.354+0000 7ff1437fe700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 0x7ff13006ccf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff1440c31e0 tx=0x7ff134009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.358+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff1380bc4f0 con 0x7ff14414f1f0 2024-03-14T09:04:29.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.550+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff144153dc0 con 0x7ff14414f1f0 2024-03-14T09:04:29.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.551+0000 7ff12bfff700 1 -- 172.21.15.154:0/724543045 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff138057b50 con 0x7ff14414f1f0 2024-03-14T09:04:29.551 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 msgr2=0x7ff13006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 0x7ff13006ccf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff1440c31e0 tx=0x7ff134009040 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 msgr2=0x7ff144157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff1380072c0 tx=0x7ff1380075d0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 shutdown_connections 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1441582a0 0x7ff144154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff144152fa0 0x7ff14415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff13006a830 0x7ff13006ccf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 --2- 172.21.15.154:0/724543045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff14414f1f0 0x7ff144157d60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.553 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 >> 172.21.15.154:0/724543045 conn(0x7ff1440c3880 msgr2=0x7ff14406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 shutdown_connections 2024-03-14T09:04:29.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.553+0000 7ff149b4a700 1 -- 172.21.15.154:0/724543045 wait complete. 2024-03-14T09:04:29.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.689+0000 7fbd503c6700 1 Processor -- start 2024-03-14T09:04:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.689+0000 7fbd503c6700 1 -- start start 2024-03-14T09:04:29.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.690+0000 7fbd503c6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd480702f0 0x7fbd480c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.690 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.690+0000 7fbd503c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.690+0000 7fbd503c6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480acd10 con 0x7fbd480c78c0 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.690+0000 7fbd503c6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd480ac7b0 con 0x7fbd480702f0 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.690+0000 7fbd503c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48150050 con 0x7fbd4814fbd0 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4e162700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd480702f0 0x7fbd480c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4e963700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbd480c78c0 0x7fbd480c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:52006/0 (socket says 172.21.15.154:52006) 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4e963700 1 -- 172.21.15.154:0/385557627 learned_addr learned my addr 172.21.15.154:0/385557627 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 --2- 172.21.15.154:0/385557627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:40920/0 (socket says 172.21.15.154:40920) 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 -- 172.21.15.154:0/385557627 >> v2:172.21.15.191:3300/0 conn(0x7fbd480702f0 msgr2=0x7fbd480c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 --2- 172.21.15.154:0/385557627 >> v2:172.21.15.191:3300/0 conn(0x7fbd480702f0 0x7fbd480c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 -- 172.21.15.154:0/385557627 >> v1:172.21.15.191:6789/0 conn(0x7fbd480c78c0 legacy=0x7fbd480c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.691 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 -- 172.21.15.154:0/385557627 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480ac9c0 con 0x7fbd4814fbd0 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d961700 1 --2- 172.21.15.154:0/385557627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009840 tx=0x7fbd44009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e64491d5a94ded server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d160700 1 -- 172.21.15.154:0/385557627 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44011830 con 0x7fbd4814fbd0 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d160700 1 -- 172.21.15.154:0/385557627 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44011990 con 0x7fbd4814fbd0 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.691+0000 7fbd4d160700 1 -- 172.21.15.154:0/385557627 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44011d20 con 0x7fbd4814fbd0 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 -- 172.21.15.154:0/385557627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 msgr2=0x7fbd480c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 --2- 172.21.15.154:0/385557627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009840 tx=0x7fbd44009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 -- 172.21.15.154:0/385557627 shutdown_connections 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 --2- 172.21.15.154:0/385557627 >> v2:172.21.15.191:3300/0 conn(0x7fbd480702f0 0x7fbd480c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 --2- 172.21.15.154:0/385557627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd4814fbd0 0x7fbd480c7380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 -- 172.21.15.154:0/385557627 >> 172.21.15.154:0/385557627 conn(0x7fbd480a9bf0 msgr2=0x7fbd480a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 -- 172.21.15.154:0/385557627 shutdown_connections 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.692+0000 7fbd503c6700 1 -- 172.21.15.154:0/385557627 wait complete. 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.693+0000 7fbd503c6700 1 Processor -- start 2024-03-14T09:04:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.693+0000 7fbd503c6700 1 -- start start 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd480c78c0 0x7fbd481eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd4814fbd0 0x7fbd48151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48156b90 con 0x7fbd480702f0 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4814c520 con 0x7fbd4814fbd0 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd503c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4814c7c0 con 0x7fbd480c78c0 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.697 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43396/0 (socket says 172.21.15.154:43396) 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 -- 172.21.15.154:0/3045205306 learned_addr learned my addr 172.21.15.154:0/3045205306 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e963700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd4814fbd0 0x7fbd48151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4d961700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd480c78c0 0x7fbd481eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 -- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd480c78c0 msgr2=0x7fbd481eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd480c78c0 0x7fbd481eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 -- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd4814fbd0 msgr2=0x7fbd48151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd4814fbd0 0x7fbd48151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.694+0000 7fbd4e162700 1 -- 172.21.15.154:0/3045205306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440094a0 con 0x7fbd480702f0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd4e162700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e870 tx=0x7fbd3800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38009590 con 0x7fbd480702f0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38013680 con 0x7fbd480702f0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd481ebd60 con 0x7fbd480702f0 2024-03-14T09:04:29.698 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380139f0 con 0x7fbd480702f0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.695+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd48148210 con 0x7fbd480702f0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.696+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd481ebef0 con 0x7fbd480702f0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.696+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbd38013b50 con 0x7fbd480702f0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.697+0000 7fbd3effd700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 0x7fbd2c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.697+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fbd38088ed0 con 0x7fbd480702f0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.697+0000 7fbd4d961700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 0x7fbd2c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.697+0000 7fbd4d961700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 0x7fbd2c06ccf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbd44000c00 tx=0x7fbd44008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:29.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.701+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbd38054000 con 0x7fbd480702f0 2024-03-14T09:04:29.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.895+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd480697d0 con 0x7fbd480702f0 2024-03-14T09:04:29.895 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.895+0000 7fbd3effd700 1 -- 172.21.15.154:0/3045205306 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7fbd380170b0 con 0x7fbd480702f0 2024-03-14T09:04:29.895 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 msgr2=0x7fbd2c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 0x7fbd2c06ccf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbd44000c00 tx=0x7fbd44008030 comp rx=0 tx=0).stop 2024-03-14T09:04:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 msgr2=0x7fbd48150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e870 tx=0x7fbd3800eb80 comp rx=0 tx=0).stop 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 shutdown_connections 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd4814fbd0 0x7fbd48151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbd2c06a830 0x7fbd2c06ccf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbd480702f0 0x7fbd48150610 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 --2- 172.21.15.154:0/3045205306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd480c78c0 0x7fbd481eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 >> 172.21.15.154:0/3045205306 conn(0x7fbd480a9bf0 msgr2=0x7fbd4814e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 shutdown_connections 2024-03-14T09:04:29.898 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:29.897+0000 7fbd503c6700 1 -- 172.21.15.154:0/3045205306 wait complete. 2024-03-14T09:04:29.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:30.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.032+0000 7f20ea063700 1 Processor -- start 2024-03-14T09:04:30.032 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.032+0000 7f20ea063700 1 -- start start 2024-03-14T09:04:30.038 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20ea063700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20e406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20ea063700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e406f5e0 con 0x7f20e406acb0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20ea063700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e4154c50 con 0x7f20e40c1080 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20ea063700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e406dc90 con 0x7f20e41550e0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20e3fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f20e406acb0 0x7f20e415f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57572/0 (socket says 172.21.15.154:57572) 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20e3fff700 1 -- 172.21.15.154:0/2076187749 learned_addr learned my addr 172.21.15.154:0/2076187749 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20e2ffd700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20e406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.033+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028237802 0 0) 0x7f20e406f5e0 con 0x7f20e406acb0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc0035b0 con 0x7f20e406acb0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016981868 0 0) 0x7f20cc0035b0 con 0x7f20e406acb0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 msgr2=0x7f20e406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20e406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 >> v1:172.21.15.191:6789/0 conn(0x7f20e40c1080 legacy=0x7f20e40c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e406fc50 con 0x7f20e406acb0 2024-03-14T09:04:30.039 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f20dc0019f0 con 0x7f20e406acb0 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 >> v1:172.21.15.154:6789/0 conn(0x7f20e406acb0 legacy=0x7f20e415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20cc004d90 0x7f20cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc00bb10 con 0x7f20cc003a30 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc00bc80 con 0x7f20cc004d90 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc00be40 con 0x7f20e41550e0 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e2ffd700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e3fff700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20cc004d90 0x7f20cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 -- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 msgr2=0x7f20cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e41550e0 0x7f20cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.040 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 -- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20cc004d90 msgr2=0x7f20cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20cc004d90 0x7f20cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 -- 172.21.15.154:0/2076187749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0035b0 con 0x7f20cc003a30 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.034+0000 7f20e37fe700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20cc003e50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f20d80017c0 tx=0x7f20cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d96c0c1f271b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80052d0 con 0x7f20cc003a30 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8005430 con 0x7f20cc003a30 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20e27fc700 1 -- 172.21.15.154:0/2076187749 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80038c0 con 0x7f20cc003a30 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 -- 172.21.15.154:0/2076187749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 msgr2=0x7f20cc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 --2- 172.21.15.154:0/2076187749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20cc003e50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f20d80017c0 tx=0x7f20cc005880 comp rx=0 tx=0).stop 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20e3fff700 1 -- 172.21.15.154:0/2076187749 reap_dead start 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 -- 172.21.15.154:0/2076187749 shutdown_connections 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 -- 172.21.15.154:0/2076187749 >> 172.21.15.154:0/2076187749 conn(0x7f20e40c39f0 msgr2=0x7f20e40c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 -- 172.21.15.154:0/2076187749 shutdown_connections 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.035+0000 7f20ea063700 1 -- 172.21.15.154:0/2076187749 wait complete. 2024-03-14T09:04:30.041 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.036+0000 7f20ea063700 1 Processor -- start 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 -- start start 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e4151200 0x7f20e4149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e414a510 0x7f20e41554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4158c90 con 0x7f20cc003a30 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41de690 con 0x7f20e4151200 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20ea063700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4158800 con 0x7f20e414a510 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20e37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20e37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43410/0 (socket says 172.21.15.154:43410) 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.037+0000 7f20e37fe700 1 -- 172.21.15.154:0/1522280787 learned_addr learned my addr 172.21.15.154:0/1522280787 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e3fff700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e414a510 0x7f20e41554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 -- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e414a510 msgr2=0x7f20e41554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e414a510 0x7f20e41554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 -- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e4151200 msgr2=0x7f20e4149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e2ffd700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e4151200 0x7f20e4149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e4151200 0x7f20e4149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 -- 172.21.15.154:0/1522280787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4151cd0 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20e37fe700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006b10 tx=0x7f20d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8007450 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d80075b0 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e414b230 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80038a0 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.038+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e41581b0 con 0x7f20cc003a30 2024-03-14T09:04:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.039+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e404ea80 con 0x7f20cc003a30 2024-03-14T09:04:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.039+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f20d8003a00 con 0x7f20cc003a30 2024-03-14T09:04:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.040+0000 7f20cbfff700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 0x7f20d006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.040+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f20d8089f10 con 0x7f20cc003a30 2024-03-14T09:04:30.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.040+0000 7f20e2ffd700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 0x7f20d006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.041+0000 7f20e2ffd700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 0x7f20d006cd20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ad470 tx=0x7f20d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.044+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f20d8055160 con 0x7f20cc003a30 2024-03-14T09:04:30.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.236+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f20e4067bd0 con 0x7f20cc003a30 2024-03-14T09:04:30.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.236+0000 7f20cbfff700 1 -- 172.21.15.154:0/1522280787 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7f20d8003cb0 con 0x7f20cc003a30 2024-03-14T09:04:30.236 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 msgr2=0x7f20d006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 0x7f20d006cd20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ad470 tx=0x7f20d4008040 comp rx=0 tx=0).stop 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 msgr2=0x7f20e414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006b10 tx=0x7f20d8001120 comp rx=0 tx=0).stop 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 shutdown_connections 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f20e414a510 0x7f20e41554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f20d006a860 0x7f20d006cd20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20cc003a30 0x7f20e414acf0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 --2- 172.21.15.154:0/1522280787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e4151200 0x7f20e4149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.238+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 >> 172.21.15.154:0/1522280787 conn(0x7f20e40c39f0 msgr2=0x7f20e4073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.239+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 shutdown_connections 2024-03-14T09:04:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.239+0000 7f20ea063700 1 -- 172.21.15.154:0/1522280787 wait complete. 2024-03-14T09:04:30.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:04:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 Processor -- start 2024-03-14T09:04:30.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 -- start start 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95640c7f70 0x7f9564155230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f956415d120 0x7f956415f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9564158240 con 0x7f95641586d0 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956414ed10 con 0x7f95640c7f70 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95698a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640cad00 con 0x7f956415d120 2024-03-14T09:04:30.375 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f9562ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95640c7f70 0x7f9564155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.373+0000 7f95627fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f95641586d0 0x7f95640c49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57588/0 (socket says 172.21.15.154:57588) 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f95640c7f70 0x7f9564155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43424/0 (socket says 172.21.15.154:43424) 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f95627fc700 1 -- 172.21.15.154:0/371216675 learned_addr learned my addr 172.21.15.154:0/371216675 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f95637fe700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f956415d120 0x7f956415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 -- 172.21.15.154:0/371216675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f956415d120 msgr2=0x7f956415f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f956415d120 0x7f956415f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 -- 172.21.15.154:0/371216675 >> v1:172.21.15.154:6789/0 conn(0x7f95641586d0 legacy=0x7f95640c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9561ffb700 1 -- 172.21.15.154:0/371216675 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67049511 0 0) 0x7f9564158240 con 0x7f95641586d0 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 -- 172.21.15.154:0/371216675 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564155770 con 0x7f95640c7f70 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9562ffd700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95640c7f70 0x7f9564155230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9554009840 tx=0x7f9554009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9a05954962d769 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9561ffb700 1 -- 172.21.15.154:0/371216675 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011830 con 0x7f95640c7f70 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9561ffb700 1 -- 172.21.15.154:0/371216675 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554011990 con 0x7f95640c7f70 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.374+0000 7f9561ffb700 1 -- 172.21.15.154:0/371216675 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011d20 con 0x7f95640c7f70 2024-03-14T09:04:30.376 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 -- 172.21.15.154:0/371216675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95640c7f70 msgr2=0x7f9564155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95640c7f70 0x7f9564155230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9554009840 tx=0x7f9554009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 -- 172.21.15.154:0/371216675 shutdown_connections 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f956415d120 0x7f956415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 --2- 172.21.15.154:0/371216675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95640c7f70 0x7f9564155230 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 -- 172.21.15.154:0/371216675 >> 172.21.15.154:0/371216675 conn(0x7f9564072080 msgr2=0x7f9564072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 -- 172.21.15.154:0/371216675 shutdown_connections 2024-03-14T09:04:30.377 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.375+0000 7f95698a8700 1 -- 172.21.15.154:0/371216675 wait complete. 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.376+0000 7f95698a8700 1 Processor -- start 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.376+0000 7f95698a8700 1 -- start start 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95640c7f70 0x7f9564156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95641586d0 0x7f956414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564155d00 con 0x7f956415d120 2024-03-14T09:04:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640af030 con 0x7f95640c7f70 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95698a8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640af2d0 con 0x7f95641586d0 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95637fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95637fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43430/0 (socket says 172.21.15.154:43430) 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.377+0000 7f95637fe700 1 -- 172.21.15.154:0/3257938932 learned_addr learned my addr 172.21.15.154:0/3257938932 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95627fc700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95641586d0 0x7f956414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f9562ffd700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95640c7f70 0x7f9564156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 -- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95641586d0 msgr2=0x7f956414f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95641586d0 0x7f956414f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 -- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95640c7f70 msgr2=0x7f9564156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95640c7f70 0x7f9564156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 -- 172.21.15.154:0/3257938932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540094a0 con 0x7f956415d120 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95637fe700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f955c007de0 tx=0x7f955c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00ed30 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c014420 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95640af5b0 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0147d0 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.378+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95641572b0 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.379+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95640afbc0 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.379+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f955c014930 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.380+0000 7f955bfff700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 0x7f954406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.380+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f955c0897a0 con 0x7f956415d120 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.380+0000 7f9562ffd700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 0x7f954406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.380+0000 7f9562ffd700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 0x7f954406ccf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f95540090b0 tx=0x7f955401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.384+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f955c0548d0 con 0x7f956415d120 2024-03-14T09:04:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.593+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95640697d0 con 0x7f956415d120 2024-03-14T09:04:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.594+0000 7f955bfff700 1 -- 172.21.15.154:0/3257938932 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f955c057ef0 con 0x7f956415d120 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 msgr2=0x7f954406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 0x7f954406ccf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f95540090b0 tx=0x7f955401c040 comp rx=0 tx=0).stop 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 msgr2=0x7f956414f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f955c007de0 tx=0x7f955c00e3c0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 shutdown_connections 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f954406a830 0x7f954406ccf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f956415d120 0x7f956414f5d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95640c7f70 0x7f9564156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 --2- 172.21.15.154:0/3257938932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95641586d0 0x7f956414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 >> 172.21.15.154:0/3257938932 conn(0x7f9564072080 msgr2=0x7f9564072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 shutdown_connections 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.596+0000 7f95698a8700 1 -- 172.21.15.154:0/3257938932 wait complete. 2024-03-14T09:04:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:04:30.608 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.testuser... 2024-03-14T09:04:30.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:30.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-14T09:04:30.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:04:30.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:30.646 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-14T09:04:30.703 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:30.703 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:30.725 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:30.725 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:30.786 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.907+0000 7f3e119bb700 1 Processor -- start 2024-03-14T09:04:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.907+0000 7f3e119bb700 1 -- start start 2024-03-14T09:04:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.908+0000 7f3e119bb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3e0c14e550 0x7f3e0c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.908+0000 7f3e119bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c151400 0x7f3e0c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.908+0000 7f3e119bb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c06d940 con 0x7f3e0c06c740 2024-03-14T09:04:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.908+0000 7f3e119bb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c14e9d0 con 0x7f3e0c14e550 2024-03-14T09:04:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.908+0000 7f3e119bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0aa800 con 0x7f3e0c151400 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3e0c14e550 0x7f3e0c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c151400 0x7f3e0c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3e0c14e550 0x7f3e0c150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:37298/0 (socket says 172.21.15.154:37298) 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 -- 172.21.15.154:0/2390846676 learned_addr learned my addr 172.21.15.154:0/2390846676 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 -- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c151400 msgr2=0x7f3e0c15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 --2- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c151400 0x7f3e0c15b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 -- 172.21.15.154:0/2390846676 >> v1:172.21.15.191:6789/0 conn(0x7f3e0c06c740 legacy=0x7f3e0c0c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 -- 172.21.15.154:0/2390846676 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c06cd30 con 0x7f3e0c14e550 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e0a7fc700 1 --2- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c14e550 0x7f3e0c150ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0098f0 tx=0x7f3dfc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6aa1234549794b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e09ffb700 1 -- 172.21.15.154:0/2390846676 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc011830 con 0x7f3e0c14e550 2024-03-14T09:04:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e09ffb700 1 -- 172.21.15.154:0/2390846676 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc011990 con 0x7f3e0c14e550 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.909+0000 7f3e09ffb700 1 -- 172.21.15.154:0/2390846676 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc011d20 con 0x7f3e0c14e550 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 -- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c14e550 msgr2=0x7f3e0c150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 --2- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c14e550 0x7f3e0c150ec0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0098f0 tx=0x7f3dfc009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 -- 172.21.15.154:0/2390846676 shutdown_connections 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 --2- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c151400 0x7f3e0c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 --2- 172.21.15.154:0/2390846676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c14e550 0x7f3e0c150ec0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 -- 172.21.15.154:0/2390846676 >> 172.21.15.154:0/2390846676 conn(0x7f3e0c0c5e70 msgr2=0x7f3e0c0c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 -- 172.21.15.154:0/2390846676 shutdown_connections 2024-03-14T09:04:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.910+0000 7f3e119bb700 1 -- 172.21.15.154:0/2390846676 wait complete. 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.911+0000 7f3e119bb700 1 Processor -- start 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.911+0000 7f3e119bb700 1 -- start start 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c06c740 0x7f3e0c0a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c14e550 0x7f3e0c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0be3a0 con 0x7f3e0c151400 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c145430 con 0x7f3e0c06c740 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e119bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0bbfa0 con 0x7f3e0c14e550 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43432/0 (socket says 172.21.15.154:43432) 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 -- 172.21.15.154:0/771869415 learned_addr learned my addr 172.21.15.154:0/771869415 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0affd700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c06c740 0x7f3e0c0a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0a7fc700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c14e550 0x7f3e0c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 -- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c14e550 msgr2=0x7f3e0c1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c14e550 0x7f3e0c1449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 -- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c06c740 msgr2=0x7f3e0c0a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c06c740 0x7f3e0c0a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.912+0000 7f3e0b7fe700 1 -- 172.21.15.154:0/771869415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc009550 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3e0b7fe700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009de0 tx=0x7f3e040074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04003c20 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04010040 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c0bb420 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04007970 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.913+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c0bb9e0 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.914+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c1ed180 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.914+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3e04007ad0 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.914+0000 7f3df3fff700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 0x7f3df806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.914+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f3e04088d00 con 0x7f3e0c151400 2024-03-14T09:04:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.915+0000 7f3e0affd700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 0x7f3df806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.915+0000 7f3e0affd700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 0x7f3df806ccf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3e00007fd0 tx=0x7f3e00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:30.919+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3e0c1ed180 con 0x7f3e0c151400 2024-03-14T09:04:31.114 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.115+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e0c1ed180 con 0x7f3e0c151400 2024-03-14T09:04:31.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.116+0000 7f3df3fff700 1 -- 172.21.15.154:0/771869415 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f3e0c1ed180 con 0x7f3e0c151400 2024-03-14T09:04:31.116 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:31.116 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 msgr2=0x7f3df806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 0x7f3df806ccf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3e00007fd0 tx=0x7f3e00009040 comp rx=0 tx=0).stop 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 msgr2=0x7f3e0c144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009de0 tx=0x7f3e040074f0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 shutdown_connections 2024-03-14T09:04:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3df806a830 0x7f3df806ccf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c151400 0x7f3e0c144ef0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e0c06c740 0x7f3e0c0a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 --2- 172.21.15.154:0/771869415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e0c14e550 0x7f3e0c1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 >> 172.21.15.154:0/771869415 conn(0x7f3e0c0c5e70 msgr2=0x7f3e0c0c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.118+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 shutdown_connections 2024-03-14T09:04:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.119+0000 7f3e119bb700 1 -- 172.21.15.154:0/771869415 wait complete. 2024-03-14T09:04:31.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:31.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.256+0000 7ff35eca9700 1 Processor -- start 2024-03-14T09:04:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.256+0000 7ff35eca9700 1 -- start start 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35eca9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3580a9650 0x7ff3580c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35eca9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 0x7ff35815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35eca9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff358156050 con 0x7ff3581564e0 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35eca9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3580a86f0 con 0x7ff3580a9650 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35eca9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580c34b0 con 0x7ff358156960 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3580a9650 0x7ff3580c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3580a9650 0x7ff3580c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43446/0 (socket says 172.21.15.154:43446) 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 -- 172.21.15.154:0/2877929355 learned_addr learned my addr 172.21.15.154:0/2877929355 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35d246700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 0x7ff35815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 -- 172.21.15.154:0/2877929355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 msgr2=0x7ff35815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 0x7ff35815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 -- 172.21.15.154:0/2877929355 >> v1:172.21.15.191:6789/0 conn(0x7ff3581564e0 legacy=0x7ff35806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 -- 172.21.15.154:0/2877929355 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3580a8990 con 0x7ff3580a9650 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35d246700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 0x7ff35815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff35ca45700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3580a9650 0x7ff3580c1780 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0098f0 tx=0x7ff34c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7aa22e46731a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff3577fe700 1 -- 172.21.15.154:0/2877929355 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c004230 con 0x7ff3580a9650 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff3577fe700 1 -- 172.21.15.154:0/2877929355 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c004390 con 0x7ff3580a9650 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.257+0000 7ff3577fe700 1 -- 172.21.15.154:0/2877929355 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0036a0 con 0x7ff3580a9650 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 -- 172.21.15.154:0/2877929355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3580a9650 msgr2=0x7ff3580c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3580a9650 0x7ff3580c1780 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0098f0 tx=0x7ff34c009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 -- 172.21.15.154:0/2877929355 shutdown_connections 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff358156960 0x7ff35815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 --2- 172.21.15.154:0/2877929355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3580a9650 0x7ff3580c1780 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 -- 172.21.15.154:0/2877929355 >> 172.21.15.154:0/2877929355 conn(0x7ff3580c3880 msgr2=0x7ff3580c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 -- 172.21.15.154:0/2877929355 shutdown_connections 2024-03-14T09:04:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.258+0000 7ff35eca9700 1 -- 172.21.15.154:0/2877929355 wait complete. 2024-03-14T09:04:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.259+0000 7ff35eca9700 1 Processor -- start 2024-03-14T09:04:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.259+0000 7ff35eca9700 1 -- start start 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3580a9650 0x7ff358143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3581564e0 0x7ff3580c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358143ff0 con 0x7ff358156960 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358154f30 con 0x7ff3581564e0 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35eca9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580c50c0 con 0x7ff3580a9650 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43454/0 (socket says 172.21.15.154:43454) 2024-03-14T09:04:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 -- 172.21.15.154:0/60362579 learned_addr learned my addr 172.21.15.154:0/60362579 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35ca45700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3580a9650 0x7ff358143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff357fff700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3581564e0 0x7ff3580c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 -- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3580a9650 msgr2=0x7ff358143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3580a9650 0x7ff358143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 -- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3581564e0 msgr2=0x7ff3580c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3581564e0 0x7ff3580c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.260+0000 7ff35d246700 1 -- 172.21.15.154:0/60362579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c009550 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff35d246700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff35000cef0 tx=0x7ff35000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000ad40 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff350015440 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000aea0 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff358145990 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.261+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff358154c50 con 0x7ff358156960 2024-03-14T09:04:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.262+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3580c2db0 con 0x7ff358156960 2024-03-14T09:04:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.262+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff35001f440 con 0x7ff358156960 2024-03-14T09:04:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.263+0000 7ff3557fa700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 0x7ff33c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.263+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff350089590 con 0x7ff358156960 2024-03-14T09:04:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.263+0000 7ff35ca45700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 0x7ff33c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.263+0000 7ff35ca45700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 0x7ff33c06ccf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009130 tx=0x7ff34c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.267+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff3500545a0 con 0x7ff358156960 2024-03-14T09:04:31.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.459+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3580697d0 con 0x7ff358156960 2024-03-14T09:04:31.460 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.459+0000 7ff3557fa700 1 -- 172.21.15.154:0/60362579 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff35001a0a0 con 0x7ff358156960 2024-03-14T09:04:31.460 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 msgr2=0x7ff33c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 0x7ff33c06ccf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009130 tx=0x7ff34c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 msgr2=0x7ff35814af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff35000cef0 tx=0x7ff35000cf20 comp rx=0 tx=0).stop 2024-03-14T09:04:31.463 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 shutdown_connections 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff33c06a830 0x7ff33c06ccf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff358156960 0x7ff35814af60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3580a9650 0x7ff358143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 --2- 172.21.15.154:0/60362579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3581564e0 0x7ff3580c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 >> 172.21.15.154:0/60362579 conn(0x7ff3580c3880 msgr2=0x7ff358071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.464 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 shutdown_connections 2024-03-14T09:04:31.465 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.462+0000 7ff35eca9700 1 -- 172.21.15.154:0/60362579 wait complete. 2024-03-14T09:04:31.474 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.599+0000 7f524b795700 1 Processor -- start 2024-03-14T09:04:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.599+0000 7f524b795700 1 -- start start 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f524b795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f524414e980 0x7f52441552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f524b795700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5244158a80 0x7f52440c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f524b795700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524414ed60 con 0x7f52440c4a80 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f524b795700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52440c6b10 con 0x7f5244158a80 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f524b795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5244155820 con 0x7f524414e980 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5244158a80 0x7f52440c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5244158a80 0x7f52440c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43470/0 (socket says 172.21.15.154:43470) 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 -- 172.21.15.154:0/564495526 learned_addr learned my addr 172.21.15.154:0/564495526 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 -- 172.21.15.154:0/564495526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f524414e980 msgr2=0x7f52441552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 --2- 172.21.15.154:0/564495526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f524414e980 0x7f52441552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 -- 172.21.15.154:0/564495526 >> v1:172.21.15.191:6789/0 conn(0x7f52440c4a80 legacy=0x7f5244158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 -- 172.21.15.154:0/564495526 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440c6cf0 con 0x7f5244158a80 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f5249d32700 1 --2- 172.21.15.154:0/564495526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5244158a80 0x7f52440c4f30 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e50 tx=0x7f523c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2049052eb2c2ef5b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f523bfff700 1 -- 172.21.15.154:0/564495526 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c00e070 con 0x7f5244158a80 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f523bfff700 1 -- 172.21.15.154:0/564495526 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c003a20 con 0x7f5244158a80 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.600+0000 7f523bfff700 1 -- 172.21.15.154:0/564495526 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c003dd0 con 0x7f5244158a80 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 -- 172.21.15.154:0/564495526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5244158a80 msgr2=0x7f52440c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 --2- 172.21.15.154:0/564495526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5244158a80 0x7f52440c4f30 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e50 tx=0x7f523c00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 -- 172.21.15.154:0/564495526 shutdown_connections 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 --2- 172.21.15.154:0/564495526 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f524414e980 0x7f52441552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 --2- 172.21.15.154:0/564495526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5244158a80 0x7f52440c4f30 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 -- 172.21.15.154:0/564495526 >> 172.21.15.154:0/564495526 conn(0x7f52440abcf0 msgr2=0x7f52440ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 -- 172.21.15.154:0/564495526 shutdown_connections 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.601+0000 7f524b795700 1 -- 172.21.15.154:0/564495526 wait complete. 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.602+0000 7f524b795700 1 Processor -- start 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.602+0000 7f524b795700 1 -- start start 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440c4a80 0x7f52440af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5244158a80 0x7f524414a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440adbe0 con 0x7f524414e980 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524414ae00 con 0x7f52440c4a80 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52440acdf0 con 0x7f5244158a80 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43484/0 (socket says 172.21.15.154:43484) 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 -- 172.21.15.154:0/622070423 learned_addr learned my addr 172.21.15.154:0/622070423 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 -- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5244158a80 msgr2=0x7f524414a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5244158a80 0x7f524414a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5249531700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440c4a80 0x7f52440af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 -- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440c4a80 msgr2=0x7f52440af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440c4a80 0x7f52440af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 -- 172.21.15.154:0/622070423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c009b30 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5248d30700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f524000c4a0 tx=0x7f524000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400054e0 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5240003680 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52440addc0 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.603+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240003a30 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.604+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52440ae380 con 0x7f524414e980 2024-03-14T09:04:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.605+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52440ae5a0 con 0x7f524414e980 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.605+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5240003b90 con 0x7f524414e980 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.605+0000 7f5239ffb700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 0x7f522806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.605+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f52400885b0 con 0x7f524414e980 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.606+0000 7f5249531700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 0x7f522806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.606+0000 7f5249531700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 0x7f522806ccf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5234006fd0 tx=0x7f5234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.610+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f52400536e0 con 0x7f524414e980 2024-03-14T09:04:31.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.803+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52440697d0 con 0x7f524414e980 2024-03-14T09:04:31.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.804+0000 7f5239ffb700 1 -- 172.21.15.154:0/622070423 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5240056d00 con 0x7f524414e980 2024-03-14T09:04:31.804 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:31.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 msgr2=0x7f522806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 0x7f522806ccf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5234006fd0 tx=0x7f5234008040 comp rx=0 tx=0).stop 2024-03-14T09:04:31.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 msgr2=0x7f524414a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f524000c4a0 tx=0x7f524000c860 comp rx=0 tx=0).stop 2024-03-14T09:04:31.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 shutdown_connections 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5244158a80 0x7f524414a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440c4a80 0x7f52440af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f522806a830 0x7f522806ccf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 --2- 172.21.15.154:0/622070423 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f524414e980 0x7f524414a380 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 >> 172.21.15.154:0/622070423 conn(0x7f52440abcf0 msgr2=0x7f5244156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 shutdown_connections 2024-03-14T09:04:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.806+0000 7f524b795700 1 -- 172.21.15.154:0/622070423 wait complete. 2024-03-14T09:04:31.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.941+0000 7f3424028700 1 Processor -- start 2024-03-14T09:04:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f3424028700 1 -- start start 2024-03-14T09:04:31.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f3424028700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f341c1553d0 0x7f341c158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f3424028700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c154f40 con 0x7f341c0bf9c0 2024-03-14T09:04:31.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f3424028700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c158d80 con 0x7f341c0c1d50 2024-03-14T09:04:31.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f3424028700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0c16a0 con 0x7f341c1553d0 2024-03-14T09:04:31.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f34225c5700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f341c0bf9c0 0x7f341c15f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57600/0 (socket says 172.21.15.154:57600) 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.942+0000 7f34225c5700 1 -- 172.21.15.154:0/177722774 learned_addr learned my addr 172.21.15.154:0/177722774 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 --2- 172.21.15.154:0/177722774 >> v2:172.21.15.191:3300/0 conn(0x7f341c1553d0 0x7f341c158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238460477 0 0) 0x7f341c154f40 con 0x7f341c0bf9c0 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34080035b0 con 0x7f341c0bf9c0 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253699491 0 0) 0x7f34080035b0 con 0x7f341c0bf9c0 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 >> v2:172.21.15.191:3300/0 conn(0x7f341c1553d0 msgr2=0x7f341c158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 --2- 172.21.15.154:0/177722774 >> v2:172.21.15.191:3300/0 conn(0x7f341c1553d0 0x7f341c158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 >> v1:172.21.15.191:6789/0 conn(0x7f341c0c1d50 legacy=0x7f341c0c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c154930 con 0x7f341c0bf9c0 2024-03-14T09:04:31.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f34140019f0 con 0x7f341c0bf9c0 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 >> v1:172.21.15.154:6789/0 conn(0x7f341c0bf9c0 legacy=0x7f341c15f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 0x7f3408003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3408004850 0x7f3408004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408005210 0x7f340800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0c16a0 con 0x7f3408003a00 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340800c160 con 0x7f3408005210 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340800c320 con 0x7f3408004850 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 0x7f3408003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f34225c5700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408005210 0x7f340800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f34215c3700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3408004850 0x7f3408004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 -- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3408004850 msgr2=0x7f3408004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3408004850 0x7f3408004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 -- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408005210 msgr2=0x7f340800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408005210 0x7f340800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f3421dc4700 1 -- 172.21.15.154:0/177722774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080035b0 con 0x7f3408003a00 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f34225c5700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408005210 0x7f340800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.943+0000 7f34225c5700 1 -- 172.21.15.154:0/177722774 reap_dead start 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3421dc4700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 0x7f3408003e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f340c009890 tx=0x7f3408005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0dbe54d0625633 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c00c070 con 0x7f3408003a00 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c013610 con 0x7f3408003a00 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3420dc2700 1 -- 172.21.15.154:0/177722774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0139a0 con 0x7f3408003a00 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 -- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 msgr2=0x7f3408003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 0x7f3408003e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f340c009890 tx=0x7f3408005880 comp rx=0 tx=0).stop 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 -- 172.21.15.154:0/177722774 shutdown_connections 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 --2- 172.21.15.154:0/177722774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3408003a00 0x7f3408003e20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 -- 172.21.15.154:0/177722774 >> 172.21.15.154:0/177722774 conn(0x7f341c06a770 msgr2=0x7f341c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 -- 172.21.15.154:0/177722774 shutdown_connections 2024-03-14T09:04:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.944+0000 7f3424028700 1 -- 172.21.15.154:0/177722774 wait complete. 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.946+0000 7f3424028700 1 Processor -- start 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.946+0000 7f3424028700 1 -- start start 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408003a00 0x7f341c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 0x7f341c0abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0ab5a0 con 0x7f341c1eabb0 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0ab880 con 0x7f3408003a00 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c1539b0 con 0x7f341c071a90 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43488/0 (socket says 172.21.15.154:43488) 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 -- 172.21.15.154:0/330840811 learned_addr learned my addr 172.21.15.154:0/330840811 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3421dc4700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408003a00 0x7f341c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34225c5700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 0x7f341c0abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 -- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 msgr2=0x7f341c0abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 0x7f341c0abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 -- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408003a00 msgr2=0x7f341c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408003a00 0x7f341c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 -- 172.21.15.154:0/330840811 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c1de7a0 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34225c5700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 0x7f341c0abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34215c3700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f341800e550 tx=0x7f341800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180052e0 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418005440 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c1dea80 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.947+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180038a0 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.948+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c09a160 con 0x7f341c1eabb0 2024-03-14T09:04:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.948+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c0697d0 con 0x7f341c1eabb0 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.949+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3418003a00 con 0x7f341c1eabb0 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.949+0000 7f34127fc700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 0x7f340006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.950+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f3418088660 con 0x7f341c1eabb0 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.950+0000 7f3421dc4700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 0x7f340006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.950+0000 7f3421dc4700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 0x7f340006ccf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c00c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:31.954 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:31.954+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3418053840 con 0x7f341c1eabb0 2024-03-14T09:04:32.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.147+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f341c066550 con 0x7f341c1eabb0 2024-03-14T09:04:32.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.148+0000 7f34127fc700 1 -- 172.21.15.154:0/330840811 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3418019020 con 0x7f341c1eabb0 2024-03-14T09:04:32.148 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 msgr2=0x7f340006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 0x7f340006ccf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c00c000 comp rx=0 tx=0).stop 2024-03-14T09:04:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 msgr2=0x7f341c071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f341800e550 tx=0x7f341800e860 comp rx=0 tx=0).stop 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 shutdown_connections 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f341c071a90 0x7f341c0abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3408003a00 0x7f341c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f340006a830 0x7f340006ccf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 --2- 172.21.15.154:0/330840811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f341c1eabb0 0x7f341c071550 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 >> 172.21.15.154:0/330840811 conn(0x7f341c06a770 msgr2=0x7f341c0c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 shutdown_connections 2024-03-14T09:04:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.150+0000 7f3424028700 1 -- 172.21.15.154:0/330840811 wait complete. 2024-03-14T09:04:32.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:32.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.285+0000 7f26a6c59700 1 Processor -- start 2024-03-14T09:04:32.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.286+0000 7f26a6c59700 1 -- start start 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26a014e980 0x7f26a01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a014ed60 con 0x7f26a0158370 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a00c6b10 con 0x7f26a014e980 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0155820 con 0x7f26a00c4a80 2024-03-14T09:04:32.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26a014e980 0x7f26a01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a51f6700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f26a0158370 0x7f26a01587e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57612/0 (socket says 172.21.15.154:57612) 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a51f6700 1 -- 172.21.15.154:0/2471873444 learned_addr learned my addr 172.21.15.154:0/2471873444 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26a014e980 0x7f26a01552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43494/0 (socket says 172.21.15.154:43494) 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f269ffff700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 -- 172.21.15.154:0/2471873444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 msgr2=0x7f26a00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a00c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 -- 172.21.15.154:0/2471873444 >> v1:172.21.15.154:6789/0 conn(0x7f26a0158370 legacy=0x7f26a01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f269f7fe700 1 -- 172.21.15.154:0/2471873444 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762313292 0 0) 0x7f26a014ed60 con 0x7f26a0158370 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 -- 172.21.15.154:0/2471873444 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00c6cf0 con 0x7f26a014e980 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a49f5700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a014e980 0x7f26a01552e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2690009840 tx=0x7f2690009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d626d418c5c58662 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f269f7fe700 1 -- 172.21.15.154:0/2471873444 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690011830 con 0x7f26a014e980 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f269f7fe700 1 -- 172.21.15.154:0/2471873444 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2690011990 con 0x7f26a014e980 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f269f7fe700 1 -- 172.21.15.154:0/2471873444 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690011d20 con 0x7f26a014e980 2024-03-14T09:04:32.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- 172.21.15.154:0/2471873444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a014e980 msgr2=0x7f26a01552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a014e980 0x7f26a01552e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2690009840 tx=0x7f2690009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- 172.21.15.154:0/2471873444 shutdown_connections 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a014e980 0x7f26a01552e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 --2- 172.21.15.154:0/2471873444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a00c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- 172.21.15.154:0/2471873444 >> 172.21.15.154:0/2471873444 conn(0x7f26a00abcf0 msgr2=0x7f26a00ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- 172.21.15.154:0/2471873444 shutdown_connections 2024-03-14T09:04:32.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.288+0000 7f26a6c59700 1 -- 172.21.15.154:0/2471873444 wait complete. 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.289+0000 7f26a6c59700 1 Processor -- start 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.289+0000 7f26a6c59700 1 -- start start 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a014e980 0x7f26a01ed510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0146510 con 0x7f26a0158370 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01ede50 con 0x7f26a014e980 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a6c59700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a0146700 con 0x7f26a00c4a80 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43506/0 (socket says 172.21.15.154:43506) 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 -- 172.21.15.154:0/1286723957 learned_addr learned my addr 172.21.15.154:0/1286723957 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a49f5700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f269ffff700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a014e980 0x7f26a01ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 -- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 msgr2=0x7f26a0154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a0154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 -- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a014e980 msgr2=0x7f26a01ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a014e980 0x7f26a01ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.290+0000 7f26a51f6700 1 -- 172.21.15.154:0/1286723957 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26900094a0 con 0x7f26a0158370 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f26a51f6700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2698002230 tx=0x7f269800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269800aac0 con 0x7f26a0158370 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269800ac40 con 0x7f26a0158370 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a01469e0 con 0x7f26a0158370 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26980070d0 con 0x7f26a0158370 2024-03-14T09:04:32.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.291+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a014be00 con 0x7f26a0158370 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.292+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a00697d0 con 0x7f26a0158370 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.292+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2698007250 con 0x7f26a0158370 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.293+0000 7f269d7fa700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 0x7f268c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.293+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f2698088e50 con 0x7f26a0158370 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.293+0000 7f26a49f5700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 0x7f268c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.293+0000 7f26a49f5700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 0x7f268c06ccf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f26900090b0 tx=0x7f2690008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.297+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2698053f80 con 0x7f26a0158370 2024-03-14T09:04:32.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.493+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26a01579e0 con 0x7f26a0158370 2024-03-14T09:04:32.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.493+0000 7f269d7fa700 1 -- 172.21.15.154:0/1286723957 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7f2698018020 con 0x7f26a0158370 2024-03-14T09:04:32.493 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 msgr2=0x7f268c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 0x7f268c06ccf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f26900090b0 tx=0x7f2690008030 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 msgr2=0x7f26a01526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2698002230 tx=0x7f269800cf50 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 shutdown_connections 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f268c06a830 0x7f268c06ccf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26a0158370 0x7f26a01526d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a00c4a80 0x7f26a0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 --2- 172.21.15.154:0/1286723957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a014e980 0x7f26a01ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.495+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 >> 172.21.15.154:0/1286723957 conn(0x7f26a00abcf0 msgr2=0x7f26a0073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.496+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 shutdown_connections 2024-03-14T09:04:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.496+0000 7f26a6c59700 1 -- 172.21.15.154:0/1286723957 wait complete. 2024-03-14T09:04:32.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:32.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.632+0000 7fba34c46700 1 Processor -- start 2024-03-14T09:04:32.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.632+0000 7fba34c46700 1 -- start start 2024-03-14T09:04:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba34c46700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba30155df0 0x7fba30156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba34c46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 0x7fba3015f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba34c46700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba300ac8e0 con 0x7fba3014eaf0 2024-03-14T09:04:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba34c46700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3006b310 con 0x7fba30155df0 2024-03-14T09:04:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba34c46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3006de30 con 0x7fba3015d270 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba30155df0 0x7fba30156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba30155df0 0x7fba30156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43508/0 (socket says 172.21.15.154:43508) 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 -- 172.21.15.154:0/2530068861 learned_addr learned my addr 172.21.15.154:0/2530068861 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2ed9d700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 0x7fba3015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 -- 172.21.15.154:0/2530068861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 msgr2=0x7fba3015f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 0x7fba3015f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 -- 172.21.15.154:0/2530068861 >> v1:172.21.15.191:6789/0 conn(0x7fba3014eaf0 legacy=0x7fba3006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 -- 172.21.15.154:0/2530068861 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300acb30 con 0x7fba30155df0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2ed9d700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 0x7fba3015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.633+0000 7fba2dd9b700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba30155df0 0x7fba30156250 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fba200098f0 tx=0x7fba20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23492d2ba544f891 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba2d59a700 1 -- 172.21.15.154:0/2530068861 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2000b8a0 con 0x7fba30155df0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba2d59a700 1 -- 172.21.15.154:0/2530068861 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2000ba00 con 0x7fba30155df0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba2d59a700 1 -- 172.21.15.154:0/2530068861 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2000bd90 con 0x7fba30155df0 2024-03-14T09:04:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 -- 172.21.15.154:0/2530068861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba30155df0 msgr2=0x7fba30156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba30155df0 0x7fba30156250 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fba200098f0 tx=0x7fba20009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 -- 172.21.15.154:0/2530068861 shutdown_connections 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba3015d270 0x7fba3015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 --2- 172.21.15.154:0/2530068861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba30155df0 0x7fba30156250 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 -- 172.21.15.154:0/2530068861 >> 172.21.15.154:0/2530068861 conn(0x7fba300c3880 msgr2=0x7fba300c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 -- 172.21.15.154:0/2530068861 shutdown_connections 2024-03-14T09:04:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.634+0000 7fba34c46700 1 -- 172.21.15.154:0/2530068861 wait complete. 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.635+0000 7fba34c46700 1 Processor -- start 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 -- start start 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba30155df0 0x7fba300c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3015d270 0x7fba301ed800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30150da0 con 0x7fba3014eaf0 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30150770 con 0x7fba3015d270 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba34c46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3014f1f0 con 0x7fba30155df0 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba2e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba2e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43510/0 (socket says 172.21.15.154:43510) 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba2e59c700 1 -- 172.21.15.154:0/816720447 learned_addr learned my addr 172.21.15.154:0/816720447 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba2dd9b700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba30155df0 0x7fba300c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.636+0000 7fba2ed9d700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3015d270 0x7fba301ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 -- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba30155df0 msgr2=0x7fba300c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba30155df0 0x7fba300c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 -- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3015d270 msgr2=0x7fba301ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3015d270 0x7fba301ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 -- 172.21.15.154:0/816720447 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20009550 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba2e59c700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c4e0 tx=0x7fba2400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240052d0 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba24005430 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240038c0 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3014f4d0 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.637+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301509c0 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.638+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300697d0 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.638+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fba24003a20 con 0x7fba3014eaf0 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.639+0000 7fba1effd700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 0x7fba1006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.639+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fba240883f0 con 0x7fba3014eaf0 2024-03-14T09:04:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.639+0000 7fba2dd9b700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 0x7fba1006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.640+0000 7fba2dd9b700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 0x7fba1006ccf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fba20009c40 tx=0x7fba20008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.643 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.644+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fba24053520 con 0x7fba3014eaf0 2024-03-14T09:04:32.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.837+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7fba3006b470 con 0x7fba3014eaf0 2024-03-14T09:04:32.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.838+0000 7fba1effd700 1 -- 172.21.15.154:0/816720447 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7fba24056b40 con 0x7fba3014eaf0 2024-03-14T09:04:32.838 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 msgr2=0x7fba1006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 0x7fba1006ccf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fba20009c40 tx=0x7fba20008030 comp rx=0 tx=0).stop 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 msgr2=0x7fba301519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c4e0 tx=0x7fba2400c7f0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 shutdown_connections 2024-03-14T09:04:32.840 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba3015d270 0x7fba301ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fba1006a830 0x7fba1006ccf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3014eaf0 0x7fba301519d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 --2- 172.21.15.154:0/816720447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba30155df0 0x7fba300c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 >> 172.21.15.154:0/816720447 conn(0x7fba300c3880 msgr2=0x7fba301573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 shutdown_connections 2024-03-14T09:04:32.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.840+0000 7fba34c46700 1 -- 172.21.15.154:0/816720447 wait complete. 2024-03-14T09:04:32.852 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-14T09:04:32.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 Processor -- start 2024-03-14T09:04:32.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 -- start start 2024-03-14T09:04:32.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb72c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb72c072290 0x7fb72c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c070e10 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.983 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c0c1bb0 con 0x7fb72c072290 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.977+0000 7fb732549700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c6170 con 0x7fb72c14c5c0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb730ae6700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb72c0c1fd0 0x7fb72c0c2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57618/0 (socket says 172.21.15.154:57618) 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb730ae6700 1 -- 172.21.15.154:0/4207899043 learned_addr learned my addr 172.21.15.154:0/4207899043 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72bfff700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb72c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/4207899043 >> v2:172.21.15.191:3300/0 conn(0x7fb72c072290 0x7fb72c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181073174 0 0) 0x7fb72c070e10 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7140035b0 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997511969 0 0) 0x7fb7140035b0 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 msgr2=0x7fb72c152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb72c152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 >> v2:172.21.15.191:3300/0 conn(0x7fb72c072290 msgr2=0x7fb72c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 --2- 172.21.15.154:0/4207899043 >> v2:172.21.15.191:3300/0 conn(0x7fb72c072290 0x7fb72c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c14c9a0 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7fb7240019f0 con 0x7fb72c0c1fd0 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 >> v1:172.21.15.154:6789/0 conn(0x7fb72c0c1fd0 legacy=0x7fb72c0c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714003a00 0x7fb714003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb714004850 0x7fb714004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb714005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c1bb0 con 0x7fb714004850 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c6170 con 0x7fb714003a00 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71400ba10 con 0x7fb72c14c5c0 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb714004850 0x7fb714004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb72bfff700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714003a00 0x7fb714003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.978+0000 7fb730ae6700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb714005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 -- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 msgr2=0x7fb714005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14c5c0 0x7fb714005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 -- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714003a00 msgr2=0x7fb714003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714003a00 0x7fb714003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 -- 172.21.15.154:0/4207899043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140035b0 con 0x7fb714004850 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb714004850 0x7fb714004cd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb720009b90 tx=0x7fb714005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e279ca1cc9a4bc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000e6c0 con 0x7fb714004850 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72000e820 con 0x7fb714004850 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb72affd700 1 -- 172.21.15.154:0/4207899043 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000eb90 con 0x7fb714004850 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 -- 172.21.15.154:0/4207899043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb714004850 msgr2=0x7fb714004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 --2- 172.21.15.154:0/4207899043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb714004850 0x7fb714004cd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb720009b90 tx=0x7fb714005880 comp rx=0 tx=0).stop 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb730ae6700 1 -- 172.21.15.154:0/4207899043 reap_dead start 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 -- 172.21.15.154:0/4207899043 shutdown_connections 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 -- 172.21.15.154:0/4207899043 >> 172.21.15.154:0/4207899043 conn(0x7fb72c06a770 msgr2=0x7fb72c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 -- 172.21.15.154:0/4207899043 shutdown_connections 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.979+0000 7fb732549700 1 -- 172.21.15.154:0/4207899043 wait complete. 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 Processor -- start 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 -- start start 2024-03-14T09:04:32.986 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 0x7fb72c14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14d0f0 0x7fb72c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c14eab0 con 0x7fb72c148bb0 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c14fca0 con 0x7fb714004850 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.981+0000 7fb732549700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1df370 con 0x7fb72c14d0f0 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43516/0 (socket says 172.21.15.154:43516) 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 -- 172.21.15.154:0/129926681 learned_addr learned my addr 172.21.15.154:0/129926681 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 -- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14d0f0 msgr2=0x7fb72c155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb72bfff700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 0x7fb72c14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb72b7fe700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14d0f0 0x7fb72c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14d0f0 0x7fb72c155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 -- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 msgr2=0x7fb72c14cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 0x7fb72c14cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:32.987 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 -- 172.21.15.154:0/129926681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c1df650 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb72bfff700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 0x7fb72c14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb730ae6700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a9340 tx=0x7fb72400a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400ae90 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7240133f0 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c14f930 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724013780 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.982+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c156270 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.983+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c0697d0 con 0x7fb72c148bb0 2024-03-14T09:04:32.988 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.984+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fb7240138e0 con 0x7fb72c148bb0 2024-03-14T09:04:32.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.984+0000 7fb728ff9700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 0x7fb71806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:32.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.984+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fb724089a00 con 0x7fb72c148bb0 2024-03-14T09:04:32.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.984+0000 7fb72bfff700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 0x7fb71806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:32.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.985+0000 7fb72bfff700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 0x7fb71806ccf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006dd0 tx=0x7fb71c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:32.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:32.990+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fb724054aa0 con 0x7fb72c148bb0 2024-03-14T09:04:33.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.198+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb72c0c08c0 con 0x7fb72c148bb0 2024-03-14T09:04:33.197 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.198+0000 7fb728ff9700 1 -- 172.21.15.154:0/129926681 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb7240580c0 con 0x7fb72c148bb0 2024-03-14T09:04:33.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 msgr2=0x7fb71806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 0x7fb71806ccf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006dd0 tx=0x7fb71c008040 comp rx=0 tx=0).stop 2024-03-14T09:04:33.199 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 msgr2=0x7fb72c149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a9340 tx=0x7fb72400a600 comp rx=0 tx=0).stop 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 shutdown_connections 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fb71806a830 0x7fb71806ccf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb72c148bb0 0x7fb72c149030 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb714004850 0x7fb72c14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 --2- 172.21.15.154:0/129926681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb72c14d0f0 0x7fb72c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 >> 172.21.15.154:0/129926681 conn(0x7fb72c06a770 msgr2=0x7fb72c06ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 shutdown_connections 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.200+0000 7fb732549700 1 -- 172.21.15.154:0/129926681 wait complete. 2024-03-14T09:04:33.200 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2024-03-14T09:04:33.211 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.testuser... 2024-03-14T09:04:33.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:33.211 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-14T09:04:33.260 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:04:33.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:33.260 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-14T09:04:33.279 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-14T09:04:33.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:04:33.318 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-14T09:04:33.368 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-14T09:04:33.369 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:33.369 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:33.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:04:33.387 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:33.387 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:33.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:04:33.406 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:33.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.549+0000 7f34def64700 1 Processor -- start 2024-03-14T09:04:33.548 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.549+0000 7f34def64700 1 -- start start 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34def64700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34d80c8350 0x7f34d81547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34def64700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f34d8154d30 0x7f34d8155170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34def64700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d8157e80 con 0x7f34d80c7f70 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34def64700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d8158ac0 con 0x7f34d80c8350 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34def64700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d8158120 con 0x7f34d8154d30 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34d80c8350 0x7f34d81547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.550 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34d80c8350 0x7f34d81547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43522/0 (socket says 172.21.15.154:43522) 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 -- 172.21.15.154:0/4039359960 learned_addr learned my addr 172.21.15.154:0/4039359960 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34dcd00700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f34d80c7f70 0x7f34d80c49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57624/0 (socket says 172.21.15.154:57624) 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 -- 172.21.15.154:0/4039359960 >> v2:172.21.15.191:3300/0 conn(0x7f34d8154d30 msgr2=0x7f34d8155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 --2- 172.21.15.154:0/4039359960 >> v2:172.21.15.191:3300/0 conn(0x7f34d8154d30 0x7f34d8155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 -- 172.21.15.154:0/4039359960 >> v1:172.21.15.154:6789/0 conn(0x7f34d80c7f70 legacy=0x7f34d80c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 -- 172.21.15.154:0/4039359960 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d8158400 con 0x7f34d80c8350 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d77fe700 1 -- 172.21.15.154:0/4039359960 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230787226 0 0) 0x7f34d8157e80 con 0x7f34d80c7f70 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d7fff700 1 --2- 172.21.15.154:0/4039359960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80c8350 0x7f34d81547f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f34d80c0f70 tx=0x7f34c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413941f9a751081f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d77fe700 1 -- 172.21.15.154:0/4039359960 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c400b870 con 0x7f34d80c8350 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d77fe700 1 -- 172.21.15.154:0/4039359960 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c400b9d0 con 0x7f34d80c8350 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.550+0000 7f34d77fe700 1 -- 172.21.15.154:0/4039359960 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c400bd60 con 0x7f34d80c8350 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 -- 172.21.15.154:0/4039359960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80c8350 msgr2=0x7f34d81547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 --2- 172.21.15.154:0/4039359960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80c8350 0x7f34d81547f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f34d80c0f70 tx=0x7f34c4009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 -- 172.21.15.154:0/4039359960 shutdown_connections 2024-03-14T09:04:33.551 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 --2- 172.21.15.154:0/4039359960 >> v2:172.21.15.191:3300/0 conn(0x7f34d8154d30 0x7f34d8155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 --2- 172.21.15.154:0/4039359960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80c8350 0x7f34d81547f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 -- 172.21.15.154:0/4039359960 >> 172.21.15.154:0/4039359960 conn(0x7f34d8072080 msgr2=0x7f34d8072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:33.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 -- 172.21.15.154:0/4039359960 shutdown_connections 2024-03-14T09:04:33.552 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.551+0000 7f34def64700 1 -- 172.21.15.154:0/4039359960 wait complete. 2024-03-14T09:04:33.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.552+0000 7f34def64700 1 Processor -- start 2024-03-14T09:04:33.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.552+0000 7f34def64700 1 -- start start 2024-03-14T09:04:33.554 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34d80c7f70 0x7f34d80c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34d80c8350 0x7f34d814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80c1870 con 0x7f34d8154d30 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8150b70 con 0x7f34d80c8350 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34def64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8157c10 con 0x7f34d80c7f70 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43536/0 (socket says 172.21.15.154:43536) 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 -- 172.21.15.154:0/1741656070 learned_addr learned my addr 172.21.15.154:0/1741656070 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34d7fff700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34d80c8350 0x7f34d814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dcd00700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34d80c7f70 0x7f34d80c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 -- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34d80c7f70 msgr2=0x7f34d80c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34d80c7f70 0x7f34d80c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 -- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34d80c8350 msgr2=0x7f34d814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34d80c8350 0x7f34d814fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.555 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.553+0000 7f34dd501700 1 -- 172.21.15.154:0/1741656070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4009550 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34dd501700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b9a0 tx=0x7f34d000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000d850 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000d9d0 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d8150690 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000dd80 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.554+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d81eae40 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.555+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d8154b50 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.555+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f34d000e420 con 0x7f34d8154d30 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.555+0000 7f34d57fa700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 0x7f34bc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.556 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.556+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f34d0089230 con 0x7f34d8154d30 2024-03-14T09:04:33.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.556+0000 7f34dcd00700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 0x7f34bc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.557 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.556+0000 7f34dcd00700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 0x7f34bc06ccf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.560+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f34d00542c0 con 0x7f34d8154d30 2024-03-14T09:04:33.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.801+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34d80697d0 con 0x7f34d8154d30 2024-03-14T09:04:33.800 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.801+0000 7f34d57fa700 1 -- 172.21.15.154:0/1741656070 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5958 (secure 0 0 0) 0x7f34d00578e0 con 0x7f34d8154d30 2024-03-14T09:04:33.800 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:33.800 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:33.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 msgr2=0x7f34bc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 0x7f34bc06ccf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc007de0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 msgr2=0x7f34d814ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b9a0 tx=0x7f34d000bcb0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 shutdown_connections 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f34bc06a830 0x7f34bc06ccf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8154d30 0x7f34d814ff80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34d80c7f70 0x7f34d80c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 --2- 172.21.15.154:0/1741656070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34d80c8350 0x7f34d814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.803+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 >> 172.21.15.154:0/1741656070 conn(0x7f34d8072080 msgr2=0x7f34d80aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.804+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 shutdown_connections 2024-03-14T09:04:33.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.804+0000 7f34def64700 1 -- 172.21.15.154:0/1741656070 wait complete. 2024-03-14T09:04:33.804 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 50 2024-03-14T09:04:33.815 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:33.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.937+0000 7f069f31b700 1 Processor -- start 2024-03-14T09:04:33.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.937+0000 7f069f31b700 1 -- start start 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069f31b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 0x7f069813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069f31b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 0x7f06980c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069f31b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069813ade0 con 0x7f06980a8540 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069f31b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069806fb30 con 0x7f06980c3250 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069f31b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069806ec30 con 0x7f06980c1780 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d8b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 0x7f06980c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069c8b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 0x7f069813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d0b7700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f06980a8540 0x7f0698154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:52074/0 (socket says 172.21.15.154:52074) 2024-03-14T09:04:33.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d0b7700 1 -- 172.21.15.154:0/2634819455 learned_addr learned my addr 172.21.15.154:0/2634819455 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d8b8700 1 -- 172.21.15.154:0/2634819455 >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 msgr2=0x7f069813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d8b8700 1 --2- 172.21.15.154:0/2634819455 >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 0x7f069813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.938+0000 7f069d8b8700 1 -- 172.21.15.154:0/2634819455 >> v1:172.21.15.191:6789/0 conn(0x7f06980a8540 legacy=0x7f0698154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069d8b8700 1 -- 172.21.15.154:0/2634819455 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069806ef10 con 0x7f06980c1780 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f068ffff700 1 -- 172.21.15.154:0/2634819455 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003570765 0 0) 0x7f0694001110 con 0x7f06980a8540 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069c8b6700 1 --2- 172.21.15.154:0/2634819455 >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 0x7f069813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069d8b8700 1 --2- 172.21.15.154:0/2634819455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 0x7f06980c1c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0690009d90 tx=0x7f069000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9a2820d11ad955 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f068ffff700 1 -- 172.21.15.154:0/2634819455 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000e070 con 0x7f06980c1780 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f068ffff700 1 -- 172.21.15.154:0/2634819455 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000bb20 con 0x7f06980c1780 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f068ffff700 1 -- 172.21.15.154:0/2634819455 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690012430 con 0x7f06980c1780 2024-03-14T09:04:33.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 -- 172.21.15.154:0/2634819455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 msgr2=0x7f06980c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 --2- 172.21.15.154:0/2634819455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 0x7f06980c1c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0690009d90 tx=0x7f069000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 -- 172.21.15.154:0/2634819455 shutdown_connections 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 --2- 172.21.15.154:0/2634819455 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980c1780 0x7f06980c1c10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 --2- 172.21.15.154:0/2634819455 >> v2:172.21.15.191:3300/0 conn(0x7f06980c3250 0x7f069813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.939+0000 7f069f31b700 1 -- 172.21.15.154:0/2634819455 >> 172.21.15.154:0/2634819455 conn(0x7f06980c3880 msgr2=0x7f06980c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.940+0000 7f069f31b700 1 -- 172.21.15.154:0/2634819455 shutdown_connections 2024-03-14T09:04:33.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.940+0000 7f069f31b700 1 -- 172.21.15.154:0/2634819455 wait complete. 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 Processor -- start 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 -- start start 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980a8540 0x7f0698135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c1780 0x7f069814f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069814e730 con 0x7f06980c3250 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06981ea310 con 0x7f06980c1780 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069f31b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698152c50 con 0x7f06980a8540 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.941+0000 7f069d8b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43552/0 (socket says 172.21.15.154:43552) 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 -- 172.21.15.154:0/4148465100 learned_addr learned my addr 172.21.15.154:0/4148465100 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069c8b6700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c1780 0x7f069814f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d0b7700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980a8540 0x7f0698135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 -- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980a8540 msgr2=0x7f0698135ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980a8540 0x7f0698135ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 -- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c1780 msgr2=0x7f069814f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c1780 0x7f069814f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 -- 172.21.15.154:0/4148465100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880094a0 con 0x7f06980c3250 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069d8b8700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0690014040 tx=0x7f0690004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690012430 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690020070 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0690009a70 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690004280 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.942+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069814e450 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.943+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698072a20 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.944+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f06900043e0 con 0x7f06980c3250 2024-03-14T09:04:33.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.944+0000 7f068dffb700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 0x7f067c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:33.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.944+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f0690022070 con 0x7f06980c3250 2024-03-14T09:04:33.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.945+0000 7f069d0b7700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 0x7f067c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:33.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.945+0000 7f069d0b7700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 0x7f067c06ccf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0694000f60 tx=0x7f0694007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:33.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:33.949+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0690056230 con 0x7f06980c3250 2024-03-14T09:04:34.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.140+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06980697d0 con 0x7f06980c3250 2024-03-14T09:04:34.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.141+0000 7f068dffb700 1 -- 172.21.15.154:0/4148465100 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f0690059850 con 0x7f06980c3250 2024-03-14T09:04:34.141 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:34.141 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 msgr2=0x7f067c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 0x7f067c06ccf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0694000f60 tx=0x7f0694007c90 comp rx=0 tx=0).stop 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 msgr2=0x7f06981ea740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0690014040 tx=0x7f0690004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 shutdown_connections 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f067c06a830 0x7f067c06ccf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06980c3250 0x7f06981ea740 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980a8540 0x7f0698135ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.143 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 --2- 172.21.15.154:0/4148465100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c1780 0x7f069814f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 >> 172.21.15.154:0/4148465100 conn(0x7f06980c3880 msgr2=0x7f069806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 shutdown_connections 2024-03-14T09:04:34.144 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.144+0000 7f069f31b700 1 -- 172.21.15.154:0/4148465100 wait complete. 2024-03-14T09:04:34.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-14T09:03:54.982472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-14T09:03:56.122682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:34.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:34.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.280+0000 7f0bc4a2a700 1 Processor -- start 2024-03-14T09:04:34.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.280+0000 7f0bc4a2a700 1 -- start start 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bc4a2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0bc00c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bc4a2a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc006f5e0 con 0x7f0bc01550e0 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bc4a2a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc0154c50 con 0x7f0bc006ae80 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bc4a2a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc006dc90 con 0x7f0bc00c1080 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbdd9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0bc01550e0 0x7f0bc006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57630/0 (socket says 172.21.15.154:57630) 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbdd9b700 1 -- 172.21.15.154:0/863713249 learned_addr learned my addr 172.21.15.154:0/863713249 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0bc00c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101277251 0 0) 0x7f0bc006f5e0 con 0x7f0bc01550e0 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba80035b0 con 0x7f0bc01550e0 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453327737 0 0) 0x7f0ba80035b0 con 0x7f0bc01550e0 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 msgr2=0x7f0bc00c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0bc00c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 >> v1:172.21.15.191:6789/0 conn(0x7f0bc006ae80 legacy=0x7f0bc015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00c0860 con 0x7f0bc01550e0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f0bb4001670 con 0x7f0bc01550e0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 >> v1:172.21.15.154:6789/0 conn(0x7f0bc01550e0 legacy=0x7f0bc006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8003a30 0x7f0ba8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba80043e0 0x7f0ba80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0ba80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba800bac0 con 0x7f0ba8003a30 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba800bc00 con 0x7f0ba80043e0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.281+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba800bdc0 con 0x7f0bc00c1080 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8003a30 0x7f0ba8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbdd9b700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba80043e0 0x7f0ba80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbed9d700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0ba80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 -- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 msgr2=0x7f0ba80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc00c1080 0x7f0ba80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 -- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba80043e0 msgr2=0x7f0ba80048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba80043e0 0x7f0ba80048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 -- 172.21.15.154:0/863713249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80035b0 con 0x7f0ba8003a30 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8003a30 0x7f0ba8003e10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009b60 tx=0x7f0ba8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5edb7113b6f173 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000e6c0 con 0x7f0ba8003a30 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb000e820 con 0x7f0ba8003a30 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.282+0000 7f0bbd59a700 1 -- 172.21.15.154:0/863713249 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000eb90 con 0x7f0ba8003a30 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/863713249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8003a30 msgr2=0x7f0ba8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/863713249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ba8003a30 0x7f0ba8003e10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009b60 tx=0x7f0ba8005880 comp rx=0 tx=0).stop 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bbed9d700 1 -- 172.21.15.154:0/863713249 reap_dead start 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/863713249 shutdown_connections 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/863713249 >> 172.21.15.154:0/863713249 conn(0x7f0bc00c39f0 msgr2=0x7f0bc00c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/863713249 shutdown_connections 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.283+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/863713249 wait complete. 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.284+0000 7f0bc4a2a700 1 Processor -- start 2024-03-14T09:04:34.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 -- start start 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 0x7f0bc00a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc01424c0 0x7f0bc0142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00ab000 con 0x7f0bc01586a0 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01ee210 con 0x7f0ba8003a30 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bc4a2a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01e7930 con 0x7f0bc01424c0 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.285+0000 7f0bbdd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43572/0 (socket says 172.21.15.154:43572) 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 -- 172.21.15.154:0/2183426837 learned_addr learned my addr 172.21.15.154:0/2183426837 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 0x7f0bc00a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbed9d700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc01424c0 0x7f0bc0142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 -- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc01424c0 msgr2=0x7f0bc0142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc01424c0 0x7f0bc0142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 -- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 msgr2=0x7f0bc00a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.291 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 0x7f0bc00a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 -- 172.21.15.154:0/2183426837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00a8670 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 0x7f0bc00a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bbdd9b700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40072c0 tx=0x7f0bb4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40077a0 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400f8d0 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc0142e90 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400fc80 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.286+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0156800 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.287+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc004ea80 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.288+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0bb401d460 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.288+0000 7f0baeffd700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 0x7f0ba006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.288+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f0bb4089030 con 0x7f0bc01586a0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.288+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 0x7f0ba006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.289+0000 7f0bbe59c700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 0x7f0ba006ccf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00090b0 tx=0x7f0bb0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.293+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0bb4054160 con 0x7f0bc01586a0 2024-03-14T09:04:34.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.487+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bc0067bd0 con 0x7f0bc01586a0 2024-03-14T09:04:34.486 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.488+0000 7f0baeffd700 1 -- 172.21.15.154:0/2183426837 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bb4057780 con 0x7f0bc01586a0 2024-03-14T09:04:34.487 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 msgr2=0x7f0ba006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 0x7f0ba006ccf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00090b0 tx=0x7f0bb0008030 comp rx=0 tx=0).stop 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 msgr2=0x7f0bc0156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40072c0 tx=0x7f0bb4007680 comp rx=0 tx=0).stop 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 shutdown_connections 2024-03-14T09:04:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc01424c0 0x7f0bc0142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ba8003a30 0x7f0bc00a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f0ba006a830 0x7f0ba006ccf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 --2- 172.21.15.154:0/2183426837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0bc01586a0 0x7f0bc0156d20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 >> 172.21.15.154:0/2183426837 conn(0x7f0bc00c39f0 msgr2=0x7f0bc00c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 shutdown_connections 2024-03-14T09:04:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.490+0000 7f0bc4a2a700 1 -- 172.21.15.154:0/2183426837 wait complete. 2024-03-14T09:04:34.501 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.625+0000 7fa7715d8700 1 Processor -- start 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.625+0000 7fa7715d8700 1 -- start start 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa7715d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c14e980 0x7fa76c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa7715d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa76c158a80 0x7fa76c0c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa7715d8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c14ed60 con 0x7fa76c0c4a80 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa7715d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0c6b10 con 0x7fa76c158a80 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa7715d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c155820 con 0x7fa76c14e980 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa76c158a80 0x7fa76c0c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa76c158a80 0x7fa76c0c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43588/0 (socket says 172.21.15.154:43588) 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 -- 172.21.15.154:0/3748492902 learned_addr learned my addr 172.21.15.154:0/3748492902 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76affd700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c14e980 0x7fa76c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.633 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 -- 172.21.15.154:0/3748492902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c14e980 msgr2=0x7fa76c1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c14e980 0x7fa76c1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 -- 172.21.15.154:0/3748492902 >> v1:172.21.15.191:6789/0 conn(0x7fa76c0c4a80 legacy=0x7fa76c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.626+0000 7fa76b7fe700 1 -- 172.21.15.154:0/3748492902 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0c6cf0 con 0x7fa76c158a80 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa76b7fe700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c158a80 0x7fa76c0c4f30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa764009e50 tx=0x7fa76400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f500c674b7cb0a45 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa769ffb700 1 -- 172.21.15.154:0/3748492902 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400e070 con 0x7fa76c158a80 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa769ffb700 1 -- 172.21.15.154:0/3748492902 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76400b910 con 0x7fa76c158a80 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa769ffb700 1 -- 172.21.15.154:0/3748492902 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400bcc0 con 0x7fa76c158a80 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 -- 172.21.15.154:0/3748492902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c158a80 msgr2=0x7fa76c0c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c158a80 0x7fa76c0c4f30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa764009e50 tx=0x7fa76400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 -- 172.21.15.154:0/3748492902 shutdown_connections 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c14e980 0x7fa76c1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 --2- 172.21.15.154:0/3748492902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c158a80 0x7fa76c0c4f30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 -- 172.21.15.154:0/3748492902 >> 172.21.15.154:0/3748492902 conn(0x7fa76c0abcf0 msgr2=0x7fa76c0ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.634 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 -- 172.21.15.154:0/3748492902 shutdown_connections 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.627+0000 7fa7715d8700 1 -- 172.21.15.154:0/3748492902 wait complete. 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 Processor -- start 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 -- start start 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa76c0c4a80 0x7fa76c0acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c158a80 0x7fa76c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0c3a70 con 0x7fa76c14e980 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0c3d10 con 0x7fa76c0c4a80 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa7715d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0af7a0 con 0x7fa76c158a80 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa76a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa76a7fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43594/0 (socket says 172.21.15.154:43594) 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa76a7fc700 1 -- 172.21.15.154:0/2586862010 learned_addr learned my addr 172.21.15.154:0/2586862010 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa76b7fe700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c158a80 0x7fa76c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.629+0000 7fa76affd700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa76c0c4a80 0x7fa76c0acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 -- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c158a80 msgr2=0x7fa76c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c158a80 0x7fa76c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 -- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa76c0c4a80 msgr2=0x7fa76c0acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa76c0c4a80 0x7fa76c0acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 -- 172.21.15.154:0/2586862010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009b30 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa76a7fc700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c4e0 tx=0x7fa75c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0052d0 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c005430 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c0afa80 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0038c0 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.630+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c152da0 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.631+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c0697d0 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.631+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa75c003a20 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.632+0000 7fa7537fe700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 0x7fa75806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.632+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fa75c0883f0 con 0x7fa76c14e980 2024-03-14T09:04:34.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.632+0000 7fa76affd700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 0x7fa75806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.632+0000 7fa76affd700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 0x7fa75806ccf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa76c14df80 tx=0x7fa760009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.637 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.636+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fa75c053520 con 0x7fa76c14e980 2024-03-14T09:04:34.830 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.830+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa76c0c5250 con 0x7fa76c14e980 2024-03-14T09:04:34.830 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.830+0000 7fa7537fe700 1 -- 172.21.15.154:0/2586862010 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa75c056b40 con 0x7fa76c14e980 2024-03-14T09:04:34.830 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 msgr2=0x7fa75806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 0x7fa75806ccf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa76c14df80 tx=0x7fa760009210 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 msgr2=0x7fa76c0ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c4e0 tx=0x7fa75c00c7f0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 shutdown_connections 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa76c158a80 0x7fa76c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa76c0c4a80 0x7fa76c0acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa75806a830 0x7fa75806ccf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 --2- 172.21.15.154:0/2586862010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa76c14e980 0x7fa76c0ae9d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 >> 172.21.15.154:0/2586862010 conn(0x7fa76c0abcf0 msgr2=0x7fa76c06b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 shutdown_connections 2024-03-14T09:04:34.833 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.833+0000 7fa7715d8700 1 -- 172.21.15.154:0/2586862010 wait complete. 2024-03-14T09:04:34.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-14T09:04:34.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.968+0000 7f024f121700 1 Processor -- start 2024-03-14T09:04:34.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.968+0000 7f024f121700 1 -- start start 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f024806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f024806acb0 0x7f0248155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0248156330 0x7f0248157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024806b310 con 0x7f024806acb0 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480ac8e0 con 0x7f0248156330 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f024f121700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024806de30 con 0x7f024814eaf0 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f0247fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f024806acb0 0x7f0248155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f0247fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f024806acb0 0x7f0248155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43612/0 (socket says 172.21.15.154:43612) 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.969+0000 7f0247fff700 1 -- 172.21.15.154:0/3933578212 learned_addr learned my addr 172.21.15.154:0/3933578212 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:34.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f024d6be700 1 --2- 172.21.15.154:0/3933578212 >> v2:172.21.15.191:3300/0 conn(0x7f0248156330 0x7f0248157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f024cebd700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f024806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 -- 172.21.15.154:0/3933578212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 msgr2=0x7f024806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f024806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 -- 172.21.15.154:0/3933578212 >> v2:172.21.15.191:3300/0 conn(0x7f0248156330 msgr2=0x7f0248157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 --2- 172.21.15.154:0/3933578212 >> v2:172.21.15.191:3300/0 conn(0x7f0248156330 0x7f0248157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 -- 172.21.15.154:0/3933578212 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480acb30 con 0x7f024806acb0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f024cebd700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f024806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f0247fff700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f024806acb0 0x7f0248155df0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0248070e50 tx=0x7f0234009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e508d84a3b402075 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f02477fe700 1 -- 172.21.15.154:0/3933578212 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234011830 con 0x7f024806acb0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f02477fe700 1 -- 172.21.15.154:0/3933578212 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0234011990 con 0x7f024806acb0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.970+0000 7f02477fe700 1 -- 172.21.15.154:0/3933578212 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234011d20 con 0x7f024806acb0 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 -- 172.21.15.154:0/3933578212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f024806acb0 msgr2=0x7f0248155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f024806acb0 0x7f0248155df0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0248070e50 tx=0x7f0234009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 -- 172.21.15.154:0/3933578212 shutdown_connections 2024-03-14T09:04:34.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 --2- 172.21.15.154:0/3933578212 >> v2:172.21.15.191:3300/0 conn(0x7f0248156330 0x7f0248157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f024806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 --2- 172.21.15.154:0/3933578212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f024806acb0 0x7f0248155df0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 -- 172.21.15.154:0/3933578212 >> 172.21.15.154:0/3933578212 conn(0x7f02480c3880 msgr2=0x7f02480c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:34.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 -- 172.21.15.154:0/3933578212 shutdown_connections 2024-03-14T09:04:34.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.971+0000 7f024f121700 1 -- 172.21.15.154:0/3933578212 wait complete. 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.972+0000 7f024f121700 1 Processor -- start 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.972+0000 7f024f121700 1 -- start start 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f024806acb0 0x7f0248153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f02481ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481ee800 con 0x7f0248156330 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481ed910 con 0x7f024806acb0 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02481edbf0 con 0x7f024814eaf0 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43626/0 (socket says 172.21.15.154:43626) 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 -- 172.21.15.154:0/1554758380 learned_addr learned my addr 172.21.15.154:0/1554758380 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:34.976 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024cebd700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f024806acb0 0x7f0248153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f0247fff700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f02481ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 -- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 msgr2=0x7f02481ea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f02481ea6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 -- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f024806acb0 msgr2=0x7f0248153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f024806acb0 0x7f0248153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 -- 172.21.15.154:0/1554758380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234009550 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024d6be700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f024000efc0 tx=0x7f024000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000cd40 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024000f040 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02481eded0 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400146c0 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.973+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481ee490 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.974+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02480697d0 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.975+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f0240014820 con 0x7f0248156330 2024-03-14T09:04:34.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.976+0000 7f02457fa700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 0x7f022c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:34.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.976+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f024008a350 con 0x7f0248156330 2024-03-14T09:04:34.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.976+0000 7f024cebd700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 0x7f022c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:34.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.976+0000 7f024cebd700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 0x7f022c06cd20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f023c00ba10 tx=0x7f023c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:34.979 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:34.980+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f0240055340 con 0x7f0248156330 2024-03-14T09:04:35.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.172+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02481ea3a0 con 0x7f0248156330 2024-03-14T09:04:35.172 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.172+0000 7f02457fa700 1 -- 172.21.15.154:0/1554758380 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0240018020 con 0x7f0248156330 2024-03-14T09:04:35.172 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:35.174 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 msgr2=0x7f022c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 0x7f022c06cd20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f023c00ba10 tx=0x7f023c008040 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 msgr2=0x7f02481eac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f024000efc0 tx=0x7f024000c3a0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 shutdown_connections 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f022c06a860 0x7f022c06cd20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0248156330 0x7f02481eac30 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f024806acb0 0x7f0248153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 --2- 172.21.15.154:0/1554758380 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f024814eaf0 0x7f02481ea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 >> 172.21.15.154:0/1554758380 conn(0x7f02480c3880 msgr2=0x7f0248157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 shutdown_connections 2024-03-14T09:04:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.174+0000 7f024f121700 1 -- 172.21.15.154:0/1554758380 wait complete. 2024-03-14T09:04:35.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.310+0000 7f702fe7d700 1 Processor -- start 2024-03-14T09:04:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 -- start start 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f70280702f0 0x7f70280c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f702814fbd0 0x7f70280c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70280acd10 con 0x7f70280c78c0 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70280ac7b0 con 0x7f70280702f0 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.311+0000 7f702fe7d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7028150050 con 0x7f702814fbd0 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702e41a700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f70280c78c0 0x7f70280c5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57640/0 (socket says 172.21.15.154:57640) 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702e41a700 1 -- 172.21.15.154:0/2690650713 learned_addr learned my addr 172.21.15.154:0/2690650713 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702d418700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f702814fbd0 0x7f70280c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> v2:172.21.15.191:3300/0 conn(0x7f70280702f0 0x7f70280c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437947953 0 0) 0x7f70280acd10 con 0x7f70280c78c0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70140035b0 con 0x7f70280c78c0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825204206 0 0) 0x7f70140035b0 con 0x7f70280c78c0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f702814fbd0 msgr2=0x7f70280c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f702814fbd0 0x7f70280c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 >> v2:172.21.15.191:3300/0 conn(0x7f70280702f0 msgr2=0x7f70280c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 --2- 172.21.15.154:0/2690650713 >> v2:172.21.15.191:3300/0 conn(0x7f70280702f0 0x7f70280c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70280ac9c0 con 0x7f70280c78c0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f70200019f0 con 0x7f70280c78c0 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 >> v1:172.21.15.154:6789/0 conn(0x7f70280c78c0 legacy=0x7f70280c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7014003a00 0x7f7014003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7014004850 0x7f7014004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014005210 0x7f701400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70280ac7b0 con 0x7f7014003a00 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028150050 con 0x7f7014005210 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701400bf50 con 0x7f7014004850 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> v2:172.21.15.191:3300/0 conn(0x7f70280702f0 0x7f70280c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702d418700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f702814fbd0 0x7f70280c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.312+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7014003a00 0x7f7014003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702e41a700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014005210 0x7f701400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702d418700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7014004850 0x7f7014004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 -- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7014004850 msgr2=0x7f7014004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7014004850 0x7f7014004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 -- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014005210 msgr2=0x7f701400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014005210 0x7f701400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 -- 172.21.15.154:0/2690650713 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140035b0 con 0x7f7014003a00 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702dc19700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7014003a00 0x7f7014003e20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f701c009d40 tx=0x7f7014005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216187fe0ec8eef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c009e90 con 0x7f7014003a00 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c00e9a0 con 0x7f7014003a00 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.313+0000 7f702cc17700 1 -- 172.21.15.154:0/2690650713 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c00ed10 con 0x7f7014003a00 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 -- 172.21.15.154:0/2690650713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7014003a00 msgr2=0x7f7014003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 --2- 172.21.15.154:0/2690650713 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7014003a00 0x7f7014003e20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f701c009d40 tx=0x7f7014005880 comp rx=0 tx=0).stop 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702e41a700 1 -- 172.21.15.154:0/2690650713 reap_dead start 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 -- 172.21.15.154:0/2690650713 shutdown_connections 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 -- 172.21.15.154:0/2690650713 >> 172.21.15.154:0/2690650713 conn(0x7f70280a9bf0 msgr2=0x7f70280a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 -- 172.21.15.154:0/2690650713 shutdown_connections 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.314+0000 7f702fe7d700 1 -- 172.21.15.154:0/2690650713 wait complete. 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.315+0000 7f702fe7d700 1 Processor -- start 2024-03-14T09:04:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.315+0000 7f702fe7d700 1 -- start start 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 0x7f70281ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7028145010 0x7f702814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702814b580 con 0x7f7028153b80 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281ec1b0 con 0x7f7014003a00 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702fe7d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702814b250 con 0x7f7028145010 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43652/0 (socket says 172.21.15.154:43652) 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 -- 172.21.15.154:0/4006814291 learned_addr learned my addr 172.21.15.154:0/4006814291 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702e41a700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7028145010 0x7f702814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702dc19700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 0x7f70281ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 -- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7028145010 msgr2=0x7f702814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7028145010 0x7f702814ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 -- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 msgr2=0x7f70281ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 0x7f70281ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702d418700 1 -- 172.21.15.154:0/4006814291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028148830 con 0x7f7028153b80 2024-03-14T09:04:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.316+0000 7f702dc19700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 0x7f70281ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f702d418700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7024011c00 tx=0x7f7024009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400fac0 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702400fc20 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7028148b10 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024014440 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.317+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70281490d0 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.318+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702814b3c0 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.318+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f70240145a0 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.318+0000 7f701a7fc700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 0x7f700c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.319+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f7024089890 con 0x7f7028153b80 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.319+0000 7f702dc19700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 0x7f700c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.319+0000 7f702dc19700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 0x7f700c06cd20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f701c009c60 tx=0x7f701c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.323+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f702814b3c0 con 0x7f7028153b80 2024-03-14T09:04:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.518+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f702814b3c0 con 0x7f7028153b80 2024-03-14T09:04:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.518+0000 7f701a7fc700 1 -- 172.21.15.154:0/4006814291 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7f702814b3c0 con 0x7f7028153b80 2024-03-14T09:04:35.518 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:35.520 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 msgr2=0x7f700c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 0x7f700c06cd20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f701c009c60 tx=0x7f701c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 msgr2=0x7f7028144ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7024011c00 tx=0x7f7024009e40 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 shutdown_connections 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7028145010 0x7f702814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7014003a00 0x7f70281ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f700c06a860 0x7f700c06cd20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 --2- 172.21.15.154:0/4006814291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7028153b80 0x7f7028144ad0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 >> 172.21.15.154:0/4006814291 conn(0x7f70280a9bf0 msgr2=0x7f702806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 shutdown_connections 2024-03-14T09:04:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.520+0000 7f702fe7d700 1 -- 172.21.15.154:0/4006814291 wait complete. 2024-03-14T09:04:35.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:35.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.657+0000 7f6fa09c7700 1 Processor -- start 2024-03-14T09:04:35.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.657+0000 7f6fa09c7700 1 -- start start 2024-03-14T09:04:35.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6fa09c7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f9c13a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6fa09c7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c13ade0 con 0x7f6f9c0c1780 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6fa09c7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c06fb30 con 0x7f6f9c0c3250 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6fa09c7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c06ec30 con 0x7f6f9c0a8540 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6f9ad9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6f9c0c1780 0x7f6f9c0c1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57656/0 (socket says 172.21.15.154:57656) 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6f9ad9d700 1 -- 172.21.15.154:0/4273640082 learned_addr learned my addr 172.21.15.154:0/4273640082 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271487809 0 0) 0x7f6f94001490 con 0x7f6f9c0c1780 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f840035b0 con 0x7f6f9c0c1780 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.658+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f9c13a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813121714 0 0) 0x7f6f840035b0 con 0x7f6f9c0c1780 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 msgr2=0x7f6f9c13a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f9c13a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 >> v1:172.21.15.191:6789/0 conn(0x7f6f9c0c3250 legacy=0x7f6f9c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c06ef10 con 0x7f6f9c0c1780 2024-03-14T09:04:35.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f6f94001b80 con 0x7f6f9c0c1780 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 >> v1:172.21.15.154:6789/0 conn(0x7f6f9c0c1780 legacy=0x7f6f9c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f84003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f84003fb0 0x7f6f84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c06ec30 con 0x7f6f84004d90 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400bc50 con 0x7f6f84003fb0 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400e020 con 0x7f6f9c0a8540 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f84003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f99d9b700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f84003fb0 0x7f6f84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 -- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 msgr2=0x7f6f84003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f84003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 -- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f84003fb0 msgr2=0x7f6f84004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:35.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f84003fb0 0x7f6f84004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 -- 172.21.15.154:0/4273640082 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840035b0 con 0x7f6f84004d90 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c0a8540 0x7f6f84003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f84005210 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400a810 tx=0x7f6f84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f1077e42844d77 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400a470 con 0x7f6f84004d90 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94012620 con 0x7f6f84004d90 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.659+0000 7f6f9959a700 1 -- 172.21.15.154:0/4273640082 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940129d0 con 0x7f6f84004d90 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 -- 172.21.15.154:0/4273640082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 msgr2=0x7f6f84005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/4273640082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f84005210 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400a810 tx=0x7f6f84005880 comp rx=0 tx=0).stop 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6f9ad9d700 1 -- 172.21.15.154:0/4273640082 reap_dead start 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 -- 172.21.15.154:0/4273640082 shutdown_connections 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 -- 172.21.15.154:0/4273640082 >> 172.21.15.154:0/4273640082 conn(0x7f6f9c0c3880 msgr2=0x7f6f9c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 -- 172.21.15.154:0/4273640082 shutdown_connections 2024-03-14T09:04:35.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.660+0000 7f6fa09c7700 1 -- 172.21.15.154:0/4273640082 wait complete. 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.661+0000 7f6fa09c7700 1 Processor -- start 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.661+0000 7f6fa09c7700 1 -- start start 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f9c0abaa0 0x7f6f9c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 0x7f6f9c14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1533e0 con 0x7f6f84004d90 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1535c0 con 0x7f6f9c0abaa0 2024-03-14T09:04:35.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6fa09c7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1538a0 con 0x7f6f9c134ff0 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6f9a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6f9a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43668/0 (socket says 172.21.15.154:43668) 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6f9a59c700 1 -- 172.21.15.154:0/3845854975 learned_addr learned my addr 172.21.15.154:0/3845854975 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6f99d9b700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f9c0abaa0 0x7f6f9c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.662+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 0x7f6f9c14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 -- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 msgr2=0x7f6f9c14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 0x7f6f9c14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 -- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f9c0abaa0 msgr2=0x7f6f9c134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f9c0abaa0 0x7f6f9c134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 -- 172.21.15.154:0/3845854975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c153b80 con 0x7f6f84004d90 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9ad9d700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 0x7f6f9c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f9a59c700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6f900118d0 tx=0x7f6f90011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90004220 con 0x7f6f84004d90 2024-03-14T09:04:35.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f90004380 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c153e60 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f900036a0 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.663+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c136940 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.664+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c1541d0 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.664+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6f90003820 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.665+0000 7f6f8affd700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 0x7f6f7c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.665+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f6f900894b0 con 0x7f6f84004d90 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.665+0000 7f6f99d9b700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 0x7f6f7c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:35.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.665+0000 7f6f99d9b700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 0x7f6f7c06ccf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0acf60 tx=0x7f6f8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:35.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.669+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6f900545e0 con 0x7f6f84004d90 2024-03-14T09:04:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.861+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f6f9c153ff0 con 0x7f6f84004d90 2024-03-14T09:04:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.861+0000 7f6f8affd700 1 -- 172.21.15.154:0/3845854975 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v74) v1 ==== 101+0+11 (secure 0 0 0) 0x7f6f90057c00 con 0x7f6f84004d90 2024-03-14T09:04:35.862 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 msgr2=0x7f6f7c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 0x7f6f7c06ccf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0acf60 tx=0x7f6f8c009040 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 msgr2=0x7f6f9c0ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f6f900118d0 tx=0x7f6f90011be0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 shutdown_connections 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.863+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6f9c134ff0 0x7f6f9c14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6f7c06a830 0x7f6f7c06ccf0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f84004d90 0x7f6f9c0ac6a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 --2- 172.21.15.154:0/3845854975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f9c0abaa0 0x7f6f9c134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 >> 172.21.15.154:0/3845854975 conn(0x7f6f9c0c3880 msgr2=0x7f6f9c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 shutdown_connections 2024-03-14T09:04:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.864+0000 7f6fa09c7700 1 -- 172.21.15.154:0/3845854975 wait complete. 2024-03-14T09:04:35.875 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:35.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.996+0000 7f9bd61ee700 1 Processor -- start 2024-03-14T09:04:35.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.996+0000 7f9bd61ee700 1 -- start start 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bd61ee700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9bd00a8540 0x7f9bd013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bd61ee700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd013ade0 con 0x7f9bd00c3250 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bd61ee700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd006fb30 con 0x7f9bd00c1780 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bd61ee700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd006ec30 con 0x7f9bd00a8540 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bceffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9bd00c3250 0x7f9bd0154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57658/0 (socket says 172.21.15.154:57658) 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.997+0000 7f9bceffd700 1 -- 172.21.15.154:0/1514959693 learned_addr learned my addr 172.21.15.154:0/1514959693 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/1514959693 >> v2:172.21.15.191:3300/0 conn(0x7f9bd00a8540 0x7f9bd013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629018856 0 0) 0x7f9bc4001110 con 0x7f9bd00c3250 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb80035b0 con 0x7f9bd00c3250 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837182564 0 0) 0x7f9bb80035b0 con 0x7f9bd00c3250 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 >> v2:172.21.15.191:3300/0 conn(0x7f9bd00a8540 msgr2=0x7f9bd013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 --2- 172.21.15.154:0/1514959693 >> v2:172.21.15.191:3300/0 conn(0x7f9bd00a8540 0x7f9bd013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 >> v1:172.21.15.191:6789/0 conn(0x7f9bd00c1780 legacy=0x7f9bd00c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd006ef10 con 0x7f9bd00c3250 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f9bc4001800 con 0x7f9bd00c3250 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 >> v1:172.21.15.154:6789/0 conn(0x7f9bd00c3250 legacy=0x7f9bd0154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8004850 0x7f9bb8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bb8005330 0x7f9bb800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd006ec30 con 0x7f9bb8003a00 2024-03-14T09:04:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb800c200 con 0x7f9bb8004850 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb800e5d0 con 0x7f9bb8005330 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bceffd700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8004850 0x7f9bb8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcffff700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bb8005330 0x7f9bb800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bb8005330 msgr2=0x7f9bb800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bb8005330 0x7f9bb800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8004850 msgr2=0x7f9bb8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8004850 0x7f9bb8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcffff700 1 -- 172.21.15.154:0/1514959693 reap_dead start 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/1514959693 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80035b0 con 0x7f9bb8003a00 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.998+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bb8003e20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b30 tx=0x7f9bb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0bbbd592834c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000e6c0 con 0x7f9bb8003a00 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc000e820 con 0x7f9bb8003a00 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bce7fc700 1 -- 172.21.15.154:0/1514959693 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000eb90 con 0x7f9bb8003a00 2024-03-14T09:04:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 -- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 msgr2=0x7f9bb8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bb8003e20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b30 tx=0x7f9bb8005880 comp rx=0 tx=0).stop 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 -- 172.21.15.154:0/1514959693 shutdown_connections 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/1514959693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bb8003e20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 -- 172.21.15.154:0/1514959693 >> 172.21.15.154:0/1514959693 conn(0x7f9bd00c3880 msgr2=0x7f9bd00c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 -- 172.21.15.154:0/1514959693 shutdown_connections 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:35.999+0000 7f9bd61ee700 1 -- 172.21.15.154:0/1514959693 wait complete. 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.000+0000 7f9bd61ee700 1 Processor -- start 2024-03-14T09:04:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 -- start start 2024-03-14T09:04:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bd014e3b0 0x7f9bd014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bd014d820 0x7f9bd014dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd014f720 con 0x7f9bb8003a00 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd014fce0 con 0x7f9bd014e3b0 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bd61ee700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd014e1e0 con 0x7f9bd014d820 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bcf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bcf7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43674/0 (socket says 172.21.15.154:43674) 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/3934396813 learned_addr learned my addr 172.21.15.154:0/3934396813 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bcffff700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bd014d820 0x7f9bd014dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.001+0000 7f9bceffd700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bd014e3b0 0x7f9bd014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bd014d820 msgr2=0x7f9bd014dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bd014d820 0x7f9bd014dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bd014e3b0 msgr2=0x7f9bd014e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bd014e3b0 0x7f9bd014e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 -- 172.21.15.154:0/3934396813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd01e9db0 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bcf7fe700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e4b0 tx=0x7f9bc00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000e6c0 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc00038c0 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0003c70 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd014f720 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.002+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd014f970 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.003+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0153290 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.003+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9bc0013470 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.004+0000 7f9bb7fff700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 0x7f9bbc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.004+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f9bc008a6d0 con 0x7f9bb8003a00 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.004+0000 7f9bceffd700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 0x7f9bbc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.007 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.004+0000 7f9bceffd700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 0x7f9bbc06ccf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0154590 tx=0x7f9bc4007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.008+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9bc0055800 con 0x7f9bb8003a00 2024-03-14T09:04:36.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.248+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bd00697d0 con 0x7f9bb8003a00 2024-03-14T09:04:36.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.248+0000 7f9bb7fff700 1 -- 172.21.15.154:0/3934396813 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5958 (secure 0 0 0) 0x7f9bc0058e20 con 0x7f9bb8003a00 2024-03-14T09:04:36.248 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:36.248 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:36.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 msgr2=0x7f9bbc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 0x7f9bbc06ccf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0154590 tx=0x7f9bc4007040 comp rx=0 tx=0).stop 2024-03-14T09:04:36.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 msgr2=0x7f9bd01e9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e4b0 tx=0x7f9bc00047c0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 shutdown_connections 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bd014d820 0x7f9bd014dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f9bbc06a830 0x7f9bbc06ccf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bb8003a00 0x7f9bd01e9570 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 --2- 172.21.15.154:0/3934396813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bd014e3b0 0x7f9bd014e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 >> 172.21.15.154:0/3934396813 conn(0x7f9bd00c3880 msgr2=0x7f9bd00c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 shutdown_connections 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.250+0000 7f9bd61ee700 1 -- 172.21.15.154:0/3934396813 wait complete. 2024-03-14T09:04:36.251 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 50 2024-03-14T09:04:36.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:36.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.384+0000 7ff33bdae700 1 Processor -- start 2024-03-14T09:04:36.384 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.384+0000 7ff33bdae700 1 -- start start 2024-03-14T09:04:36.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff33bdae700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff33406d3c0 0x7ff3340c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff33bdae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff3340c1d90 0x7ff3340c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff33bdae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff334151860 con 0x7ff3340c1830 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff33bdae700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3340c50e0 con 0x7ff33406d3c0 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff33bdae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff334150f30 con 0x7ff3340c1d90 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff33406d3c0 0x7ff3340c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff33406d3c0 0x7ff3340c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43682/0 (socket says 172.21.15.154:43682) 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 -- 172.21.15.154:0/1960543415 learned_addr learned my addr 172.21.15.154:0/1960543415 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 -- 172.21.15.154:0/1960543415 >> v2:172.21.15.191:3300/0 conn(0x7ff3340c1d90 msgr2=0x7ff3340c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 --2- 172.21.15.154:0/1960543415 >> v2:172.21.15.191:3300/0 conn(0x7ff3340c1d90 0x7ff3340c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 -- 172.21.15.154:0/1960543415 >> v1:172.21.15.191:6789/0 conn(0x7ff3340c1830 legacy=0x7ff3340c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.385+0000 7ff339b4a700 1 -- 172.21.15.154:0/1960543415 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334151210 con 0x7ff33406d3c0 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff339b4a700 1 --2- 172.21.15.154:0/1960543415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff33406d3c0 0x7ff3340c12f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff3240098f0 tx=0x7ff324009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788c03c57e68eb7c server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff338b48700 1 -- 172.21.15.154:0/1960543415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324004230 con 0x7ff33406d3c0 2024-03-14T09:04:36.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff338b48700 1 -- 172.21.15.154:0/1960543415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff324004390 con 0x7ff33406d3c0 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff338b48700 1 -- 172.21.15.154:0/1960543415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240036a0 con 0x7ff33406d3c0 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 -- 172.21.15.154:0/1960543415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff33406d3c0 msgr2=0x7ff3340c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 --2- 172.21.15.154:0/1960543415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff33406d3c0 0x7ff3340c12f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff3240098f0 tx=0x7ff324009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 -- 172.21.15.154:0/1960543415 shutdown_connections 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 --2- 172.21.15.154:0/1960543415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff33406d3c0 0x7ff3340c12f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 --2- 172.21.15.154:0/1960543415 >> v2:172.21.15.191:3300/0 conn(0x7ff3340c1d90 0x7ff3340c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 -- 172.21.15.154:0/1960543415 >> 172.21.15.154:0/1960543415 conn(0x7ff33406a770 msgr2=0x7ff33406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 -- 172.21.15.154:0/1960543415 shutdown_connections 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.386+0000 7ff33bdae700 1 -- 172.21.15.154:0/1960543415 wait complete. 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.387+0000 7ff33bdae700 1 Processor -- start 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 -- start start 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 0x7ff3341468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3340c1830 0x7ff33414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.396 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334152800 con 0x7ff3340c1d90 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341520d0 con 0x7ff33406d3c0 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33bdae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff334149c60 con 0x7ff3340c1830 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33a34b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33a34b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43686/0 (socket says 172.21.15.154:43686) 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff33a34b700 1 -- 172.21.15.154:0/980900096 learned_addr learned my addr 172.21.15.154:0/980900096 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff339b4a700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 0x7ff3341468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.388+0000 7ff339349700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3340c1830 0x7ff33414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 -- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3340c1830 msgr2=0x7ff33414d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3340c1830 0x7ff33414d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 -- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 msgr2=0x7ff3341468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 0x7ff3341468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 -- 172.21.15.154:0/980900096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff324009550 con 0x7ff3340c1d90 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff339b4a700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 0x7ff3341468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:36.397 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33a34b700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c7a0 tx=0x7ff32c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003e10 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c005070 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff334148be0 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005420 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.389+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff334154e20 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.390+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3340697d0 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.390+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff32c005580 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.391+0000 7ff32a7fc700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 0x7ff31806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.391+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff32c0885a0 con 0x7ff3340c1d90 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.391+0000 7ff339b4a700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 0x7ff31806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.391+0000 7ff339b4a700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 0x7ff31806ccf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff324009130 tx=0x7ff324008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.395+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff32c0535a0 con 0x7ff3340c1d90 2024-03-14T09:04:36.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.591+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff33414b580 con 0x7ff3340c1d90 2024-03-14T09:04:36.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.593+0000 7ff32a7fc700 1 -- 172.21.15.154:0/980900096 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7ff32c056bc0 con 0x7ff3340c1d90 2024-03-14T09:04:36.593 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:36.593 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:36.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 msgr2=0x7ff31806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.595 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 0x7ff31806ccf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff324009130 tx=0x7ff324008030 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 msgr2=0x7ff3341e9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c7a0 tx=0x7ff32c00cb60 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 shutdown_connections 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff31806a830 0x7ff31806ccf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3340c1d90 0x7ff3341e9e30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff33406d3c0 0x7ff3341468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 --2- 172.21.15.154:0/980900096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff3340c1830 0x7ff33414d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 >> 172.21.15.154:0/980900096 conn(0x7ff33406a770 msgr2=0x7ff334147400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 shutdown_connections 2024-03-14T09:04:36.596 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.595+0000 7ff33bdae700 1 -- 172.21.15.154:0/980900096 wait complete. 2024-03-14T09:04:36.607 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T09:04:36.607 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.728+0000 7f72a2359700 1 Processor -- start 2024-03-14T09:04:36.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 -- start start 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f729c155df0 0x7f729c156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 0x7f729c15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729c0ac8e0 con 0x7f729c14eaf0 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c06b310 con 0x7f729c155df0 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f72a2359700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729c06de30 con 0x7f729c15d270 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.729+0000 7f729b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f729c155df0 0x7f729c156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f729c155df0 0x7f729c156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43690/0 (socket says 172.21.15.154:43690) 2024-03-14T09:04:36.731 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 -- 172.21.15.154:0/2394765915 learned_addr learned my addr 172.21.15.154:0/2394765915 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f72a08f6700 1 --2- 172.21.15.154:0/2394765915 >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 0x7f729c15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 -- 172.21.15.154:0/2394765915 >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 msgr2=0x7f729c15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 --2- 172.21.15.154:0/2394765915 >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 0x7f729c15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 -- 172.21.15.154:0/2394765915 >> v1:172.21.15.191:6789/0 conn(0x7f729c14eaf0 legacy=0x7f729c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 -- 172.21.15.154:0/2394765915 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0acb30 con 0x7f729c155df0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f72a08f6700 1 --2- 172.21.15.154:0/2394765915 >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 0x7f729c15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729b7fe700 1 --2- 172.21.15.154:0/2394765915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c155df0 0x7f729c156250 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f729c0c4920 tx=0x7f7290009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f56d07f9c06bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729affd700 1 -- 172.21.15.154:0/2394765915 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000b870 con 0x7f729c155df0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729affd700 1 -- 172.21.15.154:0/2394765915 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729000b9d0 con 0x7f729c155df0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.730+0000 7f729affd700 1 -- 172.21.15.154:0/2394765915 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000bd60 con 0x7f729c155df0 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 -- 172.21.15.154:0/2394765915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c155df0 msgr2=0x7f729c156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 --2- 172.21.15.154:0/2394765915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c155df0 0x7f729c156250 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f729c0c4920 tx=0x7f7290009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 -- 172.21.15.154:0/2394765915 shutdown_connections 2024-03-14T09:04:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 --2- 172.21.15.154:0/2394765915 >> v2:172.21.15.191:3300/0 conn(0x7f729c15d270 0x7f729c15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 --2- 172.21.15.154:0/2394765915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c155df0 0x7f729c156250 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 -- 172.21.15.154:0/2394765915 >> 172.21.15.154:0/2394765915 conn(0x7f729c0c3880 msgr2=0x7f729c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 -- 172.21.15.154:0/2394765915 shutdown_connections 2024-03-14T09:04:36.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.731+0000 7f72a2359700 1 -- 172.21.15.154:0/2394765915 wait complete. 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.732+0000 7f72a2359700 1 Processor -- start 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.732+0000 7f72a2359700 1 -- start start 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 0x7f729c0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f729c15d270 0x7f729c146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1455e0 con 0x7f729c14eaf0 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1458c0 con 0x7f729c155df0 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a2359700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729c147df0 con 0x7f729c15d270 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f729bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f729bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43698/0 (socket says 172.21.15.154:43698) 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f729bfff700 1 -- 172.21.15.154:0/968898937 learned_addr learned my addr 172.21.15.154:0/968898937 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:36.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f729b7fe700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 0x7f729c0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.733+0000 7f72a08f6700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f729c15d270 0x7f729c146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 -- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f729c15d270 msgr2=0x7f729c146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f729c15d270 0x7f729c146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 -- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 msgr2=0x7f729c0c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 0x7f729c0c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 -- 172.21.15.154:0/968898937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290009550 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729b7fe700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 0x7f729c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f729bfff700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c4e0 tx=0x7f728c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0052d0 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c005430 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c1480d0 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0038c0 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.734+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c0c2810 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.735+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c0697d0 con 0x7f729c14eaf0 2024-03-14T09:04:36.737 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.735+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f728c003a20 con 0x7f729c14eaf0 2024-03-14T09:04:36.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.735+0000 7f7298ff9700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 0x7f728806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:36.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.736+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f728c0885a0 con 0x7f729c14eaf0 2024-03-14T09:04:36.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.736+0000 7f729b7fe700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 0x7f728806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:36.738 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.736+0000 7f729b7fe700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 0x7f728806ccf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f7290008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:36.739 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.740+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f728c0536d0 con 0x7f729c14eaf0 2024-03-14T09:04:36.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.981+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729c145ca0 con 0x7f729c14eaf0 2024-03-14T09:04:36.981 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.982+0000 7f7298ff9700 1 -- 172.21.15.154:0/968898937 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5958 (secure 0 0 0) 0x7f728c056cf0 con 0x7f729c14eaf0 2024-03-14T09:04:36.982 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:36.982 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 msgr2=0x7f728806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 0x7f728806ccf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7290000c00 tx=0x7f7290008030 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 msgr2=0x7f729c1ea5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c4e0 tx=0x7f728c00c8a0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 shutdown_connections 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f729c15d270 0x7f729c146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f728806a830 0x7f728806ccf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f729c14eaf0 0x7f729c1ea5f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 --2- 172.21.15.154:0/968898937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729c155df0 0x7f729c0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 >> 172.21.15.154:0/968898937 conn(0x7f729c0c3880 msgr2=0x7f729c1568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:36.984 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 shutdown_connections 2024-03-14T09:04:36.985 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:36.984+0000 7f72a2359700 1 -- 172.21.15.154:0/968898937 wait complete. 2024-03-14T09:04:36.985 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 50 2024-03-14T09:04:36.996 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:37.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.116+0000 7f295ada0700 1 Processor -- start 2024-03-14T09:04:37.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 -- start start 2024-03-14T09:04:37.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2954151e50 0x7f29540ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 0x7f295415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29540c2e30 con 0x7f29540ace20 2024-03-14T09:04:37.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2954154c70 con 0x7f2954151e50 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f295ada0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2954154f50 con 0x7f2954158870 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f2958b3c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2954151e50 0x7f29540ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f2958b3c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2954151e50 0x7f29540ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43700/0 (socket says 172.21.15.154:43700) 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f2958b3c700 1 -- 172.21.15.154:0/1775068369 learned_addr learned my addr 172.21.15.154:0/1775068369 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.117+0000 7f2953fff700 1 --1- 172.21.15.154:0/1775068369 >> v1:172.21.15.154:6789/0 conn(0x7f29540ace20 0x7f2954158160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57666/0 (socket says 172.21.15.154:57666) 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f295933d700 1 --2- 172.21.15.154:0/1775068369 >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 0x7f295415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f2958b3c700 1 -- 172.21.15.154:0/1775068369 >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 msgr2=0x7f295415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f2958b3c700 1 --2- 172.21.15.154:0/1775068369 >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 0x7f295415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f2958b3c700 1 -- 172.21.15.154:0/1775068369 >> v1:172.21.15.154:6789/0 conn(0x7f29540ace20 legacy=0x7f2954158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f2958b3c700 1 -- 172.21.15.154:0/1775068369 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954155230 con 0x7f2954151e50 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f29537fe700 1 -- 172.21.15.154:0/1775068369 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980396329 0 0) 0x7f29540c2e30 con 0x7f29540ace20 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f295933d700 1 --2- 172.21.15.154:0/1775068369 >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 0x7f295415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f2958b3c700 1 --2- 172.21.15.154:0/1775068369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29540ac8e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f29540c0d80 tx=0x7f2948009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb7f229a21b037f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f29537fe700 1 -- 172.21.15.154:0/1775068369 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800b870 con 0x7f2954151e50 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f29537fe700 1 -- 172.21.15.154:0/1775068369 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800b9d0 con 0x7f2954151e50 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f29537fe700 1 -- 172.21.15.154:0/1775068369 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800bd60 con 0x7f2954151e50 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f295ada0700 1 -- 172.21.15.154:0/1775068369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 msgr2=0x7f29540ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.118+0000 7f295ada0700 1 --2- 172.21.15.154:0/1775068369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29540ac8e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f29540c0d80 tx=0x7f2948009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 -- 172.21.15.154:0/1775068369 shutdown_connections 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 --2- 172.21.15.154:0/1775068369 >> v2:172.21.15.191:3300/0 conn(0x7f2954158870 0x7f295415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 --2- 172.21.15.154:0/1775068369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29540ac8e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 -- 172.21.15.154:0/1775068369 >> 172.21.15.154:0/1775068369 conn(0x7f29540c3ad0 msgr2=0x7f29540c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 -- 172.21.15.154:0/1775068369 shutdown_connections 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.119+0000 7f295ada0700 1 -- 172.21.15.154:0/1775068369 wait complete. 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.120+0000 7f295ada0700 1 Processor -- start 2024-03-14T09:04:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.120+0000 7f295ada0700 1 -- start start 2024-03-14T09:04:37.124 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 0x7f295414f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2954158870 0x7f295414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29541eb0e0 con 0x7f2954151e50 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29540c5490 con 0x7f2954158870 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29541ea120 con 0x7f29540ace20 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43706/0 (socket says 172.21.15.154:43706) 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 -- 172.21.15.154:0/2787396458 learned_addr learned my addr 172.21.15.154:0/2787396458 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2958b3c700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 0x7f295414f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295933d700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2954158870 0x7f295414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 -- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 msgr2=0x7f295414f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 0x7f295414f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 -- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2954158870 msgr2=0x7f295414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2954158870 0x7f295414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 -- 172.21.15.154:0/2787396458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009550 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2958b3c700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 0x7f295414f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f2953fff700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f294000c550 tx=0x7f294000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940004c80 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940004e00 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940007310 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29541574f0 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.121+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2954157aa0 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.122+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29540697d0 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.123+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2940007470 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.123+0000 7f29517fa700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 0x7f293806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.123+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f2940088dd0 con 0x7f2954151e50 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.123+0000 7f2958b3c700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 0x7f293806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.124+0000 7f2958b3c700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 0x7f293806ccf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.128+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2940053e70 con 0x7f2954151e50 2024-03-14T09:04:37.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.369+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f295406f500 con 0x7f2954151e50 2024-03-14T09:04:37.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.370+0000 7f29517fa700 1 -- 172.21.15.154:0/2787396458 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5958 (secure 0 0 0) 0x7f2940057490 con 0x7f2954151e50 2024-03-14T09:04:37.370 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:37.370 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:03:57.159417+0000","modified":"2024-03-14T09:04:00.171013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6441":{"gid":6441,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/1688485831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1688485831},{"type":"v1","addr":"172.21.15.154:6835","nonce":1688485831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6},{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:37.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 msgr2=0x7f293806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 0x7f293806ccf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948008030 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 msgr2=0x7f29541eaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f294000c550 tx=0x7f294000c910 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 shutdown_connections 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2954158870 0x7f295414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29540ace20 0x7f295414f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f293806a830 0x7f293806ccf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 --2- 172.21.15.154:0/2787396458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2954151e50 0x7f29541eaba0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 >> 172.21.15.154:0/2787396458 conn(0x7f29540c3ad0 msgr2=0x7f29540c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 shutdown_connections 2024-03-14T09:04:37.374 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.372+0000 7f295ada0700 1 -- 172.21.15.154:0/2787396458 wait complete. 2024-03-14T09:04:37.374 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 50 2024-03-14T09:04:37.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:37.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.508+0000 7f2fdf484700 1 Processor -- start 2024-03-14T09:04:37.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.508+0000 7f2fdf484700 1 -- start start 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdf484700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fd814e980 0x7f2fd81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdf484700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fd80c4a80 0x7f2fd80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdf484700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd814ed60 con 0x7f2fd8158370 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdf484700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80c6b10 con 0x7f2fd80c4a80 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdf484700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd8155820 con 0x7f2fd814e980 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdca1f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fd80c4a80 0x7f2fd80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdca1f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fd80c4a80 0x7f2fd80c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43718/0 (socket says 172.21.15.154:43718) 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3576811143 learned_addr learned my addr 172.21.15.154:0/3576811143 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.509+0000 7f2fdd220700 1 --2- 172.21.15.154:0/3576811143 >> v2:172.21.15.191:3300/0 conn(0x7f2fd814e980 0x7f2fd81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3576811143 >> v2:172.21.15.191:3300/0 conn(0x7f2fd814e980 msgr2=0x7f2fd81552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdca1f700 1 --2- 172.21.15.154:0/3576811143 >> v2:172.21.15.191:3300/0 conn(0x7f2fd814e980 0x7f2fd81552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3576811143 >> v1:172.21.15.191:6789/0 conn(0x7f2fd8158370 legacy=0x7f2fd8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3576811143 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80c6cf0 con 0x7f2fd80c4a80 2024-03-14T09:04:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdca1f700 1 --2- 172.21.15.154:0/3576811143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd80c4a80 0x7f2fd80c4f10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80c7870 tx=0x7f2fd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944aa08e0dbd76b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fcffff700 1 -- 172.21.15.154:0/3576811143 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4011830 con 0x7f2fd80c4a80 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fcffff700 1 -- 172.21.15.154:0/3576811143 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4011990 con 0x7f2fd80c4a80 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fcffff700 1 -- 172.21.15.154:0/3576811143 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4011d20 con 0x7f2fd80c4a80 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdf484700 1 -- 172.21.15.154:0/3576811143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd80c4a80 msgr2=0x7f2fd80c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.510+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3576811143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd80c4a80 0x7f2fd80c4f10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80c7870 tx=0x7f2fd4009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 -- 172.21.15.154:0/3576811143 shutdown_connections 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3576811143 >> v2:172.21.15.191:3300/0 conn(0x7f2fd814e980 0x7f2fd81552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3576811143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd80c4a80 0x7f2fd80c4f10 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 -- 172.21.15.154:0/3576811143 >> 172.21.15.154:0/3576811143 conn(0x7f2fd80abcf0 msgr2=0x7f2fd80ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 -- 172.21.15.154:0/3576811143 shutdown_connections 2024-03-14T09:04:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.511+0000 7f2fdf484700 1 -- 172.21.15.154:0/3576811143 wait complete. 2024-03-14T09:04:37.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 Processor -- start 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 -- start start 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd80c4a80 0x7f2fd80ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 0x7f2fd81ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81eb3f0 con 0x7f2fd814e980 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81ea770 con 0x7f2fd8158370 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.512+0000 7f2fdf484700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81eaa50 con 0x7f2fd80c4a80 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43728/0 (socket says 172.21.15.154:43728) 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3920535317 learned_addr learned my addr 172.21.15.154:0/3920535317 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdd220700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd80c4a80 0x7f2fd80ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdda21700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 0x7f2fd81ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd80c4a80 msgr2=0x7f2fd80ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd80c4a80 0x7f2fd80ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 msgr2=0x7f2fd81ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 0x7f2fd81ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 -- 172.21.15.154:0/3920535317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009550 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdda21700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 0x7f2fd81ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdca1f700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009c10 tx=0x7f2fd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004590 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd401e070 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd81ead30 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40038e0 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.513+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd80ab600 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.515+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80697d0 con 0x7f2fd814e980 2024-03-14T09:04:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.515+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2fd4003a40 con 0x7f2fd814e980 2024-03-14T09:04:37.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.515+0000 7f2fcdffb700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.515+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f2fd4089f10 con 0x7f2fd814e980 2024-03-14T09:04:37.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.515+0000 7f2fdd220700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.516+0000 7f2fdd220700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80c9e00 tx=0x7f2fc8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.520+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2fd4055040 con 0x7f2fd814e980 2024-03-14T09:04:37.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.716+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fd80c61a0 con 0x7f2fd814e980 2024-03-14T09:04:37.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.718+0000 7f2fcdffb700 1 -- 172.21.15.154:0/3920535317 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f2fd4020090 con 0x7f2fd814e980 2024-03-14T09:04:37.718 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:37.718 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:10.770571+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-14T09:03:54.982472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-14T09:03:56.122682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 msgr2=0x7f2fbc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80c9e00 tx=0x7f2fc8009210 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 msgr2=0x7f2fd81ebb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009c10 tx=0x7f2fd4004750 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 shutdown_connections 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fd8158370 0x7f2fd81ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fd80c4a80 0x7f2fd80ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f2fbc06a830 0x7f2fbc06ccf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 --2- 172.21.15.154:0/3920535317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fd814e980 0x7f2fd81ebb60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.720+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 >> 172.21.15.154:0/3920535317 conn(0x7f2fd80abcf0 msgr2=0x7f2fd8158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.721+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 shutdown_connections 2024-03-14T09:04:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.721+0000 7f2fdf484700 1 -- 172.21.15.154:0/3920535317 wait complete. 2024-03-14T09:04:37.733 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-14T09:04:37.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 Processor -- start 2024-03-14T09:04:37.856 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 -- start start 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58814eaf0 0x7fe58806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe588156500 0x7fe58815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880ac8e0 con 0x7fe58806acb0 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58806b310 con 0x7fe588156500 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.856+0000 7fe58f705700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58806de30 con 0x7fe58814eaf0 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe588156500 0x7fe58815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe588156500 0x7fe58815f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43730/0 (socket says 172.21.15.154:43730) 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 -- 172.21.15.154:0/812838376 learned_addr learned my addr 172.21.15.154:0/812838376 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58d4a1700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58814eaf0 0x7fe58806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 -- 172.21.15.154:0/812838376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58814eaf0 msgr2=0x7fe58806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58814eaf0 0x7fe58806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.859 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 -- 172.21.15.154:0/812838376 >> v1:172.21.15.191:6789/0 conn(0x7fe58806acb0 legacy=0x7fe588155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 -- 172.21.15.154:0/812838376 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588156950 con 0x7fe588156500 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe58dca2700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe588156500 0x7fe58815f280 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe580009e50 tx=0x7fe58000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4988afd233a5a28 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe57ffff700 1 -- 172.21.15.154:0/812838376 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000e070 con 0x7fe588156500 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe57ffff700 1 -- 172.21.15.154:0/812838376 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580003a20 con 0x7fe588156500 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.857+0000 7fe57ffff700 1 -- 172.21.15.154:0/812838376 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580003dd0 con 0x7fe588156500 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 -- 172.21.15.154:0/812838376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe588156500 msgr2=0x7fe58815f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe588156500 0x7fe58815f280 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fe580009e50 tx=0x7fe58000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 -- 172.21.15.154:0/812838376 shutdown_connections 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58814eaf0 0x7fe58806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 --2- 172.21.15.154:0/812838376 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe588156500 0x7fe58815f280 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 -- 172.21.15.154:0/812838376 >> 172.21.15.154:0/812838376 conn(0x7fe5880c3880 msgr2=0x7fe5880c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 -- 172.21.15.154:0/812838376 shutdown_connections 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.858+0000 7fe58f705700 1 -- 172.21.15.154:0/812838376 wait complete. 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.859+0000 7fe58f705700 1 Processor -- start 2024-03-14T09:04:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.859+0000 7fe58f705700 1 -- start start 2024-03-14T09:04:37.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58806acb0 0x7fe5880c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe588156500 0x7fe5881eb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880c5540 con 0x7fe58814eaf0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880c5cf0 con 0x7fe588156500 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58f705700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880c5730 con 0x7fe58806acb0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43748/0 (socket says 172.21.15.154:43748) 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 -- 172.21.15.154:0/2650772735 learned_addr learned my addr 172.21.15.154:0/2650772735 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58d4a1700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58806acb0 0x7fe5880c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58dca2700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe588156500 0x7fe5881eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 -- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58806acb0 msgr2=0x7fe5880c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58806acb0 0x7fe5880c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 -- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe588156500 msgr2=0x7fe5881eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe588156500 0x7fe5881eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 -- 172.21.15.154:0/2650772735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009b30 con 0x7fe58814eaf0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe58cca0700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b4a0 tx=0x7fe58400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.860+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584011830 con 0x7fe58814eaf0 2024-03-14T09:04:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.861+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584011990 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.861+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584011d40 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.861+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5880c5a10 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.861+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5880c2b00 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.862+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880c17e0 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.862+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe58401a400 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.862+0000 7fe57dffb700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 0x7fe56c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.863+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fe584088c90 con 0x7fe58814eaf0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.863+0000 7fe58d4a1700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 0x7fe56c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:37.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.863+0000 7fe58d4a1700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 0x7fe56c06ccf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe578006b20 tx=0x7fe578008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:37.866 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:37.867+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe5880c17e0 con 0x7fe58814eaf0 2024-03-14T09:04:38.101 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.101+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe5880c17e0 con 0x7fe58814eaf0 2024-03-14T09:04:38.102 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:04:38.102+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:04:38.113 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.114+0000 7fe57dffb700 1 -- 172.21.15.154:0/2650772735 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe5880c17e0 con 0x7fe58814eaf0 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 msgr2=0x7fe56c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 0x7fe56c06ccf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe578006b20 tx=0x7fe578008040 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 msgr2=0x7fe58814a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b4a0 tx=0x7fe58400b7b0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 shutdown_connections 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe588156500 0x7fe5881eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe58806acb0 0x7fe5880c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe56c06a830 0x7fe56c06ccf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 --2- 172.21.15.154:0/2650772735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe58814eaf0 0x7fe58814a3b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 >> 172.21.15.154:0/2650772735 conn(0x7fe5880c3880 msgr2=0x7fe58814d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 shutdown_connections 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.116+0000 7fe58f705700 1 -- 172.21.15.154:0/2650772735 wait complete. 2024-03-14T09:04:38.117 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:04:38.118 INFO:tasks.ceph.mds.a.smithi154.stderr: -6241> 2024-03-14T09:04:02.670+0000 7fa034edb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:04:38.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-14T09:04:38.150 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2024-03-14T09:04:38.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.254+0000 7fbe97422700 1 Processor -- start 2024-03-14T09:04:38.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.254+0000 7fbe97422700 1 -- start start 2024-03-14T09:04:38.257 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe97422700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbe9014eaf0 0x7fbe9006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe97422700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe900ac8e0 con 0x7fbe9006acb0 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe97422700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9006de30 con 0x7fbe90156500 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe97422700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9006b310 con 0x7fbe9014eaf0 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe949bd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbe9006acb0 0x7fbe90155df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57670/0 (socket says 172.21.15.154:57670) 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.255+0000 7fbe949bd700 1 -- 172.21.15.154:0/1503517106 learned_addr learned my addr 172.21.15.154:0/1503517106 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe951be700 1 --2- 172.21.15.154:0/1503517106 >> v2:172.21.15.191:3300/0 conn(0x7fbe9014eaf0 0x7fbe9006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494949504 0 0) 0x7fbe900ac8e0 con 0x7fbe9006acb0 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe7c0035b0 con 0x7fbe9006acb0 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151853121 0 0) 0x7fbe9006de30 con 0x7fbe90156500 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe900ac8e0 con 0x7fbe90156500 2024-03-14T09:04:38.258 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe951be700 1 -- 172.21.15.154:0/1503517106 >> v1:172.21.15.191:6789/0 conn(0x7fbe90156500 legacy=0x7fbe9015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe951be700 1 -- 172.21.15.154:0/1503517106 >> v1:172.21.15.154:6789/0 conn(0x7fbe9006acb0 legacy=0x7fbe90155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294279612 0 0) 0x7fbe7c0035b0 con 0x7fbe9006acb0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe951be700 1 -- 172.21.15.154:0/1503517106 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90156910 con 0x7fbe9014eaf0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe951be700 1 --2- 172.21.15.154:0/1503517106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9014eaf0 0x7fbe9006a770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009840 tx=0x7fbe80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c4d78c813354b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c001670 con 0x7fbe9014eaf0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80004030 con 0x7fbe9014eaf0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.256+0000 7fbe87fff700 1 -- 172.21.15.154:0/1503517106 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe800043c0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 -- 172.21.15.154:0/1503517106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9014eaf0 msgr2=0x7fbe9006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 --2- 172.21.15.154:0/1503517106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9014eaf0 0x7fbe9006a770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009840 tx=0x7fbe80009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 -- 172.21.15.154:0/1503517106 shutdown_connections 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 --2- 172.21.15.154:0/1503517106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe9014eaf0 0x7fbe9006a770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 -- 172.21.15.154:0/1503517106 >> 172.21.15.154:0/1503517106 conn(0x7fbe900c3880 msgr2=0x7fbe900c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 -- 172.21.15.154:0/1503517106 shutdown_connections 2024-03-14T09:04:38.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.257+0000 7fbe97422700 1 -- 172.21.15.154:0/1503517106 wait complete. 2024-03-14T09:04:38.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.258+0000 7fbe97422700 1 Processor -- start 2024-03-14T09:04:38.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.258+0000 7fbe97422700 1 -- start start 2024-03-14T09:04:38.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9006acb0 0x7fbe9014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:38.262 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe90156500 0x7fbe90144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901520e0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901491a0 con 0x7fbe90156500 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90144280 con 0x7fbe9006acb0 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43780/0 (socket says 172.21.15.154:43780) 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 -- 172.21.15.154:0/3230730299 learned_addr learned my addr 172.21.15.154:0/3230730299 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe959bf700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe90156500 0x7fbe90144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe951be700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9006acb0 0x7fbe9014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 -- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9006acb0 msgr2=0x7fbe9014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9006acb0 0x7fbe9014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 -- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe90156500 msgr2=0x7fbe90144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe90156500 0x7fbe90144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 -- 172.21.15.154:0/3230730299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800094a0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe949bd700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c750 tx=0x7fbe8c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c007560 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0076c0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.259+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9014b9c0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.260+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c004ab0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.260+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90148e30 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.261+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe900697d0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.261+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbe8c004c10 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.262+0000 7fbe85ffb700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 0x7fbe7406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.262+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6680+0+0 (secure 0 0 0) 0x7fbe8c0885e0 con 0x7fbe9014eaf0 2024-03-14T09:04:38.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.262+0000 7fbe951be700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 0x7fbe7406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:38.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.262+0000 7fbe951be700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 0x7fbe7406ccf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe800090b0 tx=0x7fbe80008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:38.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.266+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbe8c053620 con 0x7fbe9014eaf0 2024-03-14T09:04:38.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:38.508+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbe90148c20 con 0x7fbe9014eaf0 2024-03-14T09:04:39.115 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.115+0000 7fbe85ffb700 1 -- 172.21.15.154:0/3230730299 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbe8c056c40 con 0x7fbe9014eaf0 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 msgr2=0x7fbe7406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 0x7fbe7406ccf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe800090b0 tx=0x7fbe80008030 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 msgr2=0x7fbe90151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00c750 tx=0x7fbe8c00cb10 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 shutdown_connections 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe90156500 0x7fbe90144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe9006acb0 0x7fbe9014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbe7406a830 0x7fbe7406ccf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 --2- 172.21.15.154:0/3230730299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe9014eaf0 0x7fbe90151ba0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.118+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 >> 172.21.15.154:0/3230730299 conn(0x7fbe900c3880 msgr2=0x7fbe90155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.119+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 shutdown_connections 2024-03-14T09:04:39.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.119+0000 7fbe97422700 1 -- 172.21.15.154:0/3230730299 wait complete. 2024-03-14T09:04:39.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-14T09:04:39.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.301+0000 7f81b2ec0700 1 Processor -- start 2024-03-14T09:04:39.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.301+0000 7f81b2ec0700 1 -- start start 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b2ec0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81ac06d4d0 0x7f81ac158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b2ec0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b2ec0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac157ed0 con 0x7f81ac155360 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b2ec0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac158490 con 0x7f81ac06d4d0 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b2ec0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac06d8b0 con 0x7f81ac0c1380 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81ac06d4d0 0x7f81ac158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81ac06d4d0 0x7f81ac158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43796/0 (socket says 172.21.15.154:43796) 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 -- 172.21.15.154:0/1924001793 learned_addr learned my addr 172.21.15.154:0/1924001793 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b145d700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 -- 172.21.15.154:0/1924001793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 msgr2=0x7f81ac0c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 -- 172.21.15.154:0/1924001793 >> v1:172.21.15.191:6789/0 conn(0x7f81ac155360 legacy=0x7f81ac06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 -- 172.21.15.154:0/1924001793 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac06cd20 con 0x7f81ac06d4d0 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b145d700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.302+0000 7f81b0c5c700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac158920 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098f0 tx=0x7f81a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeb0a2e8e2a2dfc server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81ab7fe700 1 -- 172.21.15.154:0/1924001793 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004230 con 0x7f81ac06d4d0 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81ab7fe700 1 -- 172.21.15.154:0/1924001793 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0004390 con 0x7f81ac06d4d0 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81ab7fe700 1 -- 172.21.15.154:0/1924001793 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a00036a0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 -- 172.21.15.154:0/1924001793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 msgr2=0x7f81ac158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac158920 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098f0 tx=0x7f81a0009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 -- 172.21.15.154:0/1924001793 shutdown_connections 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/1924001793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac158920 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 -- 172.21.15.154:0/1924001793 >> 172.21.15.154:0/1924001793 conn(0x7f81ac0a9d00 msgr2=0x7f81ac0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 -- 172.21.15.154:0/1924001793 shutdown_connections 2024-03-14T09:04:39.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.303+0000 7f81b2ec0700 1 -- 172.21.15.154:0/1924001793 wait complete. 2024-03-14T09:04:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.304+0000 7f81b2ec0700 1 Processor -- start 2024-03-14T09:04:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.304+0000 7f81b2ec0700 1 -- start start 2024-03-14T09:04:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac155360 0x7f81ac152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac14b1a0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac154c00 con 0x7f81ac155360 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b2ec0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0701b0 con 0x7f81ac0c1380 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b0c5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b0c5c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43812/0 (socket says 172.21.15.154:43812) 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b0c5c700 1 -- 172.21.15.154:0/3085441495 learned_addr learned my addr 172.21.15.154:0/3085441495 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81b145d700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac155360 0x7f81ac152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.305+0000 7f81abfff700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 -- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 msgr2=0x7f81ac0aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 -- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac155360 msgr2=0x7f81ac152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac155360 0x7f81ac152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 -- 172.21.15.154:0/3085441495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0009550 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b0c5c700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009130 tx=0x7f81a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003d10 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a001f070 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a000b8a0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac06f9e0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.306+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac1ee250 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.307+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac1557a0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.307+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f81a0021030 con 0x7f81ac06d4d0 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.308+0000 7f81a97fa700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 0x7f819006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.308+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f81a008a690 con 0x7f81ac06d4d0 2024-03-14T09:04:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.308+0000 7f81abfff700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 0x7f819006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.308+0000 7f81abfff700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 0x7f819006ccf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f81ac14edd0 tx=0x7f81980090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.312+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f81ac1557a0 con 0x7f81ac06d4d0 2024-03-14T09:04:39.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:39.508+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81ac1557a0 con 0x7f81ac06d4d0 2024-03-14T09:04:40.469 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.469+0000 7f81a97fa700 1 -- 172.21.15.154:0/3085441495 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7f81ac1557a0 con 0x7f81ac06d4d0 2024-03-14T09:04:40.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.473+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 msgr2=0x7f819006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.473+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 0x7f819006ccf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f81ac14edd0 tx=0x7f81980090e0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.473+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 msgr2=0x7f81ac06f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.473+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009130 tx=0x7f81a0004750 comp rx=0 tx=0).stop 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 shutdown_connections 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81ac155360 0x7f81ac152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f819006a830 0x7f819006ccf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81ac06d4d0 0x7f81ac06f4a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 --2- 172.21.15.154:0/3085441495 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ac0c1380 0x7f81ac0aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 >> 172.21.15.154:0/3085441495 conn(0x7f81ac0a9d00 msgr2=0x7f81ac0a9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 shutdown_connections 2024-03-14T09:04:40.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.474+0000 7f81b2ec0700 1 -- 172.21.15.154:0/3085441495 wait complete. 2024-03-14T09:04:40.475 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2024-03-14T09:04:40.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-14T09:04:40.615 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.616+0000 7f8e34541700 1 Processor -- start 2024-03-14T09:04:40.616 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.616+0000 7f8e34541700 1 -- start start 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e34541700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e2c0ac6c0 0x7f8e2c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e34541700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c152970 con 0x7f8e2c0c5310 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e34541700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2c0acb50 con 0x7f8e2c0c5de0 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e34541700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2c0c56f0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e32ade700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e2c0ac6c0 0x7f8e2c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e322dd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8e2c0c5310 0x7f8e2c152e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44954/0 (socket says 172.21.15.154:44954) 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e322dd700 1 -- 172.21.15.154:0/3183628502 learned_addr learned my addr 172.21.15.154:0/3183628502 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e32ade700 1 -- 172.21.15.154:0/3183628502 >> v1:172.21.15.191:6789/0 conn(0x7f8e2c0c5de0 legacy=0x7f8e2c070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:40.618 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e32ade700 1 -- 172.21.15.154:0/3183628502 >> v1:172.21.15.154:6789/0 conn(0x7f8e2c0c5310 legacy=0x7f8e2c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e312db700 1 -- 172.21.15.154:0/3183628502 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719021106 0 0) 0x7f8e2c152970 con 0x7f8e2c0c5310 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e32ade700 1 -- 172.21.15.154:0/3183628502 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0c25c0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e32ade700 1 --2- 172.21.15.154:0/3183628502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c15dfb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009dc0 tx=0x7f8e2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5a1d032ce58f53 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e312db700 1 -- 172.21.15.154:0/3183628502 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400e070 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e312db700 1 -- 172.21.15.154:0/3183628502 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2400bb50 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.617+0000 7f8e312db700 1 -- 172.21.15.154:0/3183628502 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24003a20 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 -- 172.21.15.154:0/3183628502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 msgr2=0x7f8e2c15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 --2- 172.21.15.154:0/3183628502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c15dfb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009dc0 tx=0x7f8e2400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 -- 172.21.15.154:0/3183628502 shutdown_connections 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 --2- 172.21.15.154:0/3183628502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c15dfb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 -- 172.21.15.154:0/3183628502 >> 172.21.15.154:0/3183628502 conn(0x7f8e2c0c7ee0 msgr2=0x7f8e2c0c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 -- 172.21.15.154:0/3183628502 shutdown_connections 2024-03-14T09:04:40.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.618+0000 7f8e34541700 1 -- 172.21.15.154:0/3183628502 wait complete. 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.619+0000 7f8e34541700 1 Processor -- start 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.619+0000 7f8e34541700 1 -- start start 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 0x7f8e2c1eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e2c0c5de0 0x7f8e2c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c154400 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c153c40 con 0x7f8e2c0c5de0 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e34541700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2c1ec000 con 0x7f8e2c0c5310 2024-03-14T09:04:40.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60940/0 (socket says 172.21.15.154:60940) 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 -- 172.21.15.154:0/4253262512 learned_addr learned my addr 172.21.15.154:0/4253262512 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e32ade700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e2c0c5de0 0x7f8e2c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e31adc700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 0x7f8e2c1eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 -- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 msgr2=0x7f8e2c1eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 0x7f8e2c1eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 -- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e2c0c5de0 msgr2=0x7f8e2c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e2c0c5de0 0x7f8e2c14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 -- 172.21.15.154:0/4253262512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e24009a70 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e31adc700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 0x7f8e2c1eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.620+0000 7f8e322dd700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8e280011e0 tx=0x7f8e2800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28004c80 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28004de0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28007310 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c153ee0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2c0a7000 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.621+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2c0697d0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.622+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8e28007470 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.622+0000 7f8e22ffd700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 0x7f8e1006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.622+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6242+0+0 (secure 0 0 0) 0x7f8e28088bf0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.622+0000 7f8e31adc700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 0x7f8e1006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.623+0000 7f8e31adc700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 0x7f8e1006ccf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0c6e60 tx=0x7f8e1c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.627+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8e28053e80 con 0x7f8e2c0ac6c0 2024-03-14T09:04:40.829 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:40.829+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8e2c0c82c0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:41.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.471+0000 7f8e22ffd700 1 -- 172.21.15.154:0/4253262512 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8e280574a0 con 0x7f8e2c0ac6c0 2024-03-14T09:04:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.474+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 msgr2=0x7f8e1006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.474+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 0x7f8e1006ccf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0c6e60 tx=0x7f8e1c008040 comp rx=0 tx=0).stop 2024-03-14T09:04:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.474+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 msgr2=0x7f8e2c1df840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.474+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8e280011e0 tx=0x7f8e2800ca30 comp rx=0 tx=0).stop 2024-03-14T09:04:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 shutdown_connections 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e2c0c5de0 0x7f8e2c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f8e1006a830 0x7f8e1006ccf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e2c0ac6c0 0x7f8e2c1df840 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 --2- 172.21.15.154:0/4253262512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e2c0c5310 0x7f8e2c1eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 >> 172.21.15.154:0/4253262512 conn(0x7f8e2c0c7ee0 msgr2=0x7f8e2c0c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 shutdown_connections 2024-03-14T09:04:41.475 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.475+0000 7f8e34541700 1 -- 172.21.15.154:0/4253262512 wait complete. 2024-03-14T09:04:41.476 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2024-03-14T09:04:41.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.611+0000 7fac9e0ce700 1 Processor -- start 2024-03-14T09:04:41.612 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 -- start start 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 0x7fac98070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac980c62f0 0x7fac980c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980c7ce0 con 0x7fac9806c6c0 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980c7fc0 con 0x7fac980c62f0 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac9e0ce700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980c82a0 con 0x7fac980c31c0 2024-03-14T09:04:41.619 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac97fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac980c62f0 0x7fac980c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac977fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fac9806c6c0 0x7fac9806d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44960/0 (socket says 172.21.15.154:44960) 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac97fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac980c62f0 0x7fac980c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60950/0 (socket says 172.21.15.154:60950) 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.612+0000 7fac977fe700 1 -- 172.21.15.154:0/2128437304 learned_addr learned my addr 172.21.15.154:0/2128437304 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac96ffd700 1 --2- 172.21.15.154:0/2128437304 >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 0x7fac98070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac97fff700 1 -- 172.21.15.154:0/2128437304 >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 msgr2=0x7fac98070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac97fff700 1 --2- 172.21.15.154:0/2128437304 >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 0x7fac98070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac97fff700 1 -- 172.21.15.154:0/2128437304 >> v1:172.21.15.154:6789/0 conn(0x7fac9806c6c0 legacy=0x7fac9806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac97fff700 1 -- 172.21.15.154:0/2128437304 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980c8580 con 0x7fac980c62f0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac967fc700 1 -- 172.21.15.154:0/2128437304 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411490010 0 0) 0x7fac980c7ce0 con 0x7fac9806c6c0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac96ffd700 1 --2- 172.21.15.154:0/2128437304 >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 0x7fac98070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac97fff700 1 --2- 172.21.15.154:0/2128437304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c62f0 0x7fac980c6760 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fac9806db60 tx=0x7fac9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88633a163394b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac967fc700 1 -- 172.21.15.154:0/2128437304 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e070 con 0x7fac980c62f0 2024-03-14T09:04:41.620 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac967fc700 1 -- 172.21.15.154:0/2128437304 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9000bbd0 con 0x7fac980c62f0 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac967fc700 1 -- 172.21.15.154:0/2128437304 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90012430 con 0x7fac980c62f0 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac9e0ce700 1 -- 172.21.15.154:0/2128437304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c62f0 msgr2=0x7fac980c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.613+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/2128437304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c62f0 0x7fac980c6760 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fac9806db60 tx=0x7fac9000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 -- 172.21.15.154:0/2128437304 shutdown_connections 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/2128437304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c62f0 0x7fac980c6760 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/2128437304 >> v2:172.21.15.191:3300/0 conn(0x7fac980c31c0 0x7fac98070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 -- 172.21.15.154:0/2128437304 >> 172.21.15.154:0/2128437304 conn(0x7fac980a9bf0 msgr2=0x7fac980a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 -- 172.21.15.154:0/2128437304 shutdown_connections 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.614+0000 7fac9e0ce700 1 -- 172.21.15.154:0/2128437304 wait complete. 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.615+0000 7fac9e0ce700 1 Processor -- start 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.615+0000 7fac9e0ce700 1 -- start start 2024-03-14T09:04:41.621 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9806c6c0 0x7fac981dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac980c62f0 0x7fac981ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac981ee200 con 0x7fac980c31c0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98099de0 con 0x7fac9806c6c0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9809a5f0 con 0x7fac980c62f0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60960/0 (socket says 172.21.15.154:60960) 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 -- 172.21.15.154:0/750840266 learned_addr learned my addr 172.21.15.154:0/750840266 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac977fe700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9806c6c0 0x7fac981dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac97fff700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac980c62f0 0x7fac981ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.622 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 -- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac980c62f0 msgr2=0x7fac981ea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac980c62f0 0x7fac981ea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 -- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9806c6c0 msgr2=0x7fac981dc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9806c6c0 0x7fac981dc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 -- 172.21.15.154:0/750840266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90009a70 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac96ffd700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009fd0 tx=0x7fac8c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c013610 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c013770 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9809a8d0 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c013b20 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.616+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac981dca20 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.617+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98099f70 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.618+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fac8c013c80 con 0x7fac980c31c0 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.618+0000 7fac7ffff700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 0x7fac8406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.623 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.618+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fac8c088a10 con 0x7fac980c31c0 2024-03-14T09:04:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.618+0000 7fac977fe700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 0x7fac8406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.619+0000 7fac977fe700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 0x7fac8406ccf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.624 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.623+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fac8c053de0 con 0x7fac980c31c0 2024-03-14T09:04:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.821+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac980697d0 con 0x7fac980c31c0 2024-03-14T09:04:41.821 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.822+0000 7fac7ffff700 1 -- 172.21.15.154:0/750840266 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16223 (secure 0 0 0) 0x7fac8c057400 con 0x7fac980c31c0 2024-03-14T09:04:41.822 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:41.822 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":77,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:41.467922+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/1688485831":"2024-03-15T09:04:38.103230+0000","172.21.15.154:6833/1688485831":"2024-03-15T09:04:38.103230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 msgr2=0x7fac8406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 0x7fac8406ccf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007de0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 msgr2=0x7fac981e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009fd0 tx=0x7fac8c00eac0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 shutdown_connections 2024-03-14T09:04:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac980c62f0 0x7fac981ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac9806c6c0 0x7fac981dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fac8406a830 0x7fac8406ccf0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 --2- 172.21.15.154:0/750840266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac980c31c0 0x7fac981e9cb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 >> 172.21.15.154:0/750840266 conn(0x7fac980a9bf0 msgr2=0x7fac980a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 shutdown_connections 2024-03-14T09:04:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.824+0000 7fac9e0ce700 1 -- 172.21.15.154:0/750840266 wait complete. 2024-03-14T09:04:41.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-14T09:01:13.575425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-14T09:04:07.885228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-14T09:04:08.739578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-14T09:04:41.837 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-14T09:04:41.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.964+0000 7f4d0a007700 1 Processor -- start 2024-03-14T09:04:41.964 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.964+0000 7f4d0a007700 1 -- start start 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d0a007700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d040a8540 0x7f4d0413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d0a007700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d040c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d0a007700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0413ade0 con 0x7f4d040c3250 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d0a007700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0406fb30 con 0x7f4d040a8540 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d0a007700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0406ec30 con 0x7f4d040c1780 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d040a8540 0x7f4d0413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4d040a8540 0x7f4d0413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60974/0 (socket says 172.21.15.154:60974) 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 -- 172.21.15.154:0/631166026 learned_addr learned my addr 172.21.15.154:0/631166026 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d03fff700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d040c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 -- 172.21.15.154:0/631166026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 msgr2=0x7f4d040c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d040c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 -- 172.21.15.154:0/631166026 >> v1:172.21.15.191:6789/0 conn(0x7f4d040c3250 legacy=0x7f4d04154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 -- 172.21.15.154:0/631166026 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0406ef10 con 0x7f4d040a8540 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d037fe700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040a8540 0x7f4d0413a8a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098f0 tx=0x7f4cf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f516fc139bbbace server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d027fc700 1 -- 172.21.15.154:0/631166026 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400b8a0 con 0x7f4d040a8540 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d027fc700 1 -- 172.21.15.154:0/631166026 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400ba00 con 0x7f4d040a8540 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.965+0000 7f4d027fc700 1 -- 172.21.15.154:0/631166026 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400bd90 con 0x7f4d040a8540 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 -- 172.21.15.154:0/631166026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040a8540 msgr2=0x7f4d0413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040a8540 0x7f4d0413a8a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098f0 tx=0x7f4cf4009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 -- 172.21.15.154:0/631166026 shutdown_connections 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d040c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 --2- 172.21.15.154:0/631166026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040a8540 0x7f4d0413a8a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.967 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 -- 172.21.15.154:0/631166026 >> 172.21.15.154:0/631166026 conn(0x7f4d040c3880 msgr2=0x7f4d040c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:41.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 -- 172.21.15.154:0/631166026 shutdown_connections 2024-03-14T09:04:41.968 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.966+0000 7f4d0a007700 1 -- 172.21.15.154:0/631166026 wait complete. 2024-03-14T09:04:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.967+0000 7f4d0a007700 1 Processor -- start 2024-03-14T09:04:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 -- start start 2024-03-14T09:04:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 0x7f4d040ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d04151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c6ae0 con 0x7f4d040c3250 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041eb800 con 0x7f4d040a8540 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.968+0000 7f4d0a007700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041ebae0 con 0x7f4d040c1780 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60980/0 (socket says 172.21.15.154:60980) 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 -- 172.21.15.154:0/311495647 learned_addr learned my addr 172.21.15.154:0/311495647 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d037fe700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 0x7f4d040ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d02ffd700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d04151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 -- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 msgr2=0x7f4d04151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d04151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 -- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 msgr2=0x7f4d040ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 0x7f4d040ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 -- 172.21.15.154:0/311495647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009550 con 0x7f4d040c3250 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d037fe700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 0x7f4d040ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:41.971 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d03fff700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc007e60 tx=0x7f4cfc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00ede0 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc014420 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d041ebdc0 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0147d0 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.969+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d041ec380 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.970+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040697d0 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.970+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4cfc014930 con 0x7f4d040c3250 2024-03-14T09:04:41.972 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.971+0000 7f4cebfff700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 0x7f4cf006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:41.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.971+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f4cfc089520 con 0x7f4d040c3250 2024-03-14T09:04:41.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.971+0000 7f4d037fe700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 0x7f4cf006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:41.973 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.972+0000 7f4d037fe700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 0x7f4cf006cd20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4014040 tx=0x7f4cf4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:41.975 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:41.976+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4cfc0548e0 con 0x7f4d040c3250 2024-03-14T09:04:42.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.166+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d040c2e10 con 0x7f4d040c3250 2024-03-14T09:04:42.166 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.166+0000 7f4cebfff700 1 -- 172.21.15.154:0/311495647 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cfc057f00 con 0x7f4d040c3250 2024-03-14T09:04:42.166 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.168+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 msgr2=0x7f4cf006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.168+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 0x7f4cf006cd20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4014040 tx=0x7f4cf4008030 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.168+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 msgr2=0x7f4d040a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.168+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc007e60 tx=0x7f4cfc00e3c0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.168+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 shutdown_connections 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4cf006a860 0x7f4cf006cd20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d040c3250 0x7f4d040a9db0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d040a8540 0x7f4d040ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 --2- 172.21.15.154:0/311495647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d040c1780 0x7f4d04151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 >> 172.21.15.154:0/311495647 conn(0x7f4d040c3880 msgr2=0x7f4d04139790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.169 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 shutdown_connections 2024-03-14T09:04:42.170 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.169+0000 7f4d0a007700 1 -- 172.21.15.154:0/311495647 wait complete. 2024-03-14T09:04:42.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-14T09:04:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 Processor -- start 2024-03-14T09:04:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 -- start start 2024-03-14T09:04:42.315 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f815006c670 0x7f815006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81500c7bb0 con 0x7f815006d510 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81500c7e90 con 0x7f81500c31c0 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.310+0000 7f81562d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500c3e00 con 0x7f815006c670 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814f7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f815006d510 0x7f81500c6140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44964/0 (socket says 172.21.15.154:44964) 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814f7fe700 1 -- 172.21.15.154:0/1447142719 learned_addr learned my addr 172.21.15.154:0/1447142719 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814ffff700 1 --2- 172.21.15.154:0/1447142719 >> v2:172.21.15.191:3300/0 conn(0x7f815006c670 0x7f815006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207623617 0 0) 0x7f81500c7bb0 con 0x7f815006d510 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81380035b0 con 0x7f815006d510 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054575339 0 0) 0x7f81380035b0 con 0x7f815006d510 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 >> v2:172.21.15.191:3300/0 conn(0x7f815006c670 msgr2=0x7f815006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 --2- 172.21.15.154:0/1447142719 >> v2:172.21.15.191:3300/0 conn(0x7f815006c670 0x7f815006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 >> v1:172.21.15.191:6789/0 conn(0x7f81500c31c0 legacy=0x7f81500c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500c8170 con 0x7f815006d510 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f8144001670 con 0x7f815006d510 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 >> v1:172.21.15.154:6789/0 conn(0x7f815006d510 legacy=0x7f81500c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8138003a00 0x7f8138003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138004850 0x7f8138004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 0x7f813800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500c3e00 con 0x7f8138005330 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813800c200 con 0x7f8138003a00 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813800c3c0 con 0x7f8138004850 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f8154872700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 0x7f813800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814ffff700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8138003a00 0x7f8138003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f814f7fe700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138004850 0x7f8138004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.311+0000 7f8154872700 1 -- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138004850 msgr2=0x7f8138004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138004850 0x7f8138004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.317 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 -- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8138003a00 msgr2=0x7f8138003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8138003a00 0x7f8138003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 -- 172.21.15.154:0/1447142719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380035b0 con 0x7f8138005330 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 -- 172.21.15.154:0/1447142719 reap_dead start 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f8154872700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 0x7f813800ba10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f814800c160 tx=0x7f8138005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97721d6b2c3eaa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148001db0 con 0x7f8138005330 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800cca0 con 0x7f8138005330 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f814effd700 1 -- 172.21.15.154:0/1447142719 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148003c00 con 0x7f8138005330 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 -- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 msgr2=0x7f813800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 0x7f813800ba10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f814800c160 tx=0x7f8138005880 comp rx=0 tx=0).stop 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 -- 172.21.15.154:0/1447142719 shutdown_connections 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 --2- 172.21.15.154:0/1447142719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8138005330 0x7f813800ba10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 -- 172.21.15.154:0/1447142719 >> 172.21.15.154:0/1447142719 conn(0x7f81500a9bf0 msgr2=0x7f81500a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 -- 172.21.15.154:0/1447142719 shutdown_connections 2024-03-14T09:04:42.318 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.312+0000 7f81562d5700 1 -- 172.21.15.154:0/1447142719 wait complete. 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.313+0000 7f81562d5700 1 Processor -- start 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 -- start start 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138005330 0x7f81501df920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81501e9a90 0x7f81500a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81501d79d0 con 0x7f81500aaf80 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81501ed0a0 con 0x7f81501e9a90 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f81562d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81501dd2b0 con 0x7f8138005330 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f8154872700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f8154872700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32770/0 (socket says 172.21.15.154:32770) 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.314+0000 7f8154872700 1 -- 172.21.15.154:0/180439665 learned_addr learned my addr 172.21.15.154:0/180439665 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f814ffff700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138005330 0x7f81501df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f814f7fe700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81501e9a90 0x7f81500a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 -- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138005330 msgr2=0x7f81501df920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.319 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138005330 0x7f81501df920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 -- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81501e9a90 msgr2=0x7f81500a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81501e9a90 0x7f81500a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 -- 172.21.15.154:0/180439665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81501dd590 con 0x7f81500aaf80 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f8154872700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f81500aa8c0 tx=0x7f81480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148001650 con 0x7f81500aaf80 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8148003e90 con 0x7f81500aaf80 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81480052b0 con 0x7f81500aaf80 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81500a95e0 con 0x7f81500aaf80 2024-03-14T09:04:42.320 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.315+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81501dd870 con 0x7f81500aaf80 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.316+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81501dda60 con 0x7f81500aaf80 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.316+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8148005410 con 0x7f81500aaf80 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.317+0000 7f814cff9700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 0x7f813c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.317+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f8148089ba0 con 0x7f81500aaf80 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.317+0000 7f814ffff700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 0x7f813c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.321 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.318+0000 7f814ffff700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 0x7f813c06ccf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8140009be0 tx=0x7f8140009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.322 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.322+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f81501dda60 con 0x7f81500aaf80 2024-03-14T09:04:42.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.515+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81501dda60 con 0x7f81500aaf80 2024-03-14T09:04:42.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.516+0000 7f814cff9700 1 -- 172.21.15.154:0/180439665 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v77) v1 ==== 105+0+11 (secure 0 0 0) 0x7f81501dda60 con 0x7f81500aaf80 2024-03-14T09:04:42.516 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 msgr2=0x7f813c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 0x7f813c06ccf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8140009be0 tx=0x7f8140009210 comp rx=0 tx=0).stop 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 msgr2=0x7f81501ed580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f81500aa8c0 tx=0x7f81480014a0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 shutdown_connections 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f813c06a830 0x7f813c06ccf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81500aaf80 0x7f81501ed580 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8138005330 0x7f81501df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 --2- 172.21.15.154:0/180439665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81501e9a90 0x7f81500a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 >> 172.21.15.154:0/180439665 conn(0x7f81500a9bf0 msgr2=0x7f81501d9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 shutdown_connections 2024-03-14T09:04:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.518+0000 7f81562d5700 1 -- 172.21.15.154:0/180439665 wait complete. 2024-03-14T09:04:42.530 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-14T09:04:42.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.654+0000 7f6558bdd700 1 Processor -- start 2024-03-14T09:04:42.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.654+0000 7f6558bdd700 1 -- start start 2024-03-14T09:04:42.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6558bdd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540c3250 0x7f655413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6558bdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540c1780 0x7f65540c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.656 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6558bdd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655413ade0 con 0x7f65540a8540 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6558bdd700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655406fb30 con 0x7f65540c3250 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6558bdd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655406ec30 con 0x7f65540c1780 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540c3250 0x7f655413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f655259c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f65540a8540 0x7f6554154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44972/0 (socket says 172.21.15.154:44972) 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f655259c700 1 -- 172.21.15.154:0/3114066619 learned_addr learned my addr 172.21.15.154:0/3114066619 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540c3250 0x7f655413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32774/0 (socket says 172.21.15.154:32774) 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 -- 172.21.15.154:0/3114066619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540c1780 msgr2=0x7f65540c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 --2- 172.21.15.154:0/3114066619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540c1780 0x7f65540c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 -- 172.21.15.154:0/3114066619 >> v1:172.21.15.154:6789/0 conn(0x7f65540a8540 legacy=0x7f6554154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 -- 172.21.15.154:0/3114066619 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655406ef10 con 0x7f65540c3250 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f655159a700 1 -- 172.21.15.154:0/3114066619 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155287416 0 0) 0x7f6548001110 con 0x7f65540a8540 2024-03-14T09:04:42.657 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.655+0000 7f6551d9b700 1 --2- 172.21.15.154:0/3114066619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f655413a8a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6544009840 tx=0x7f6544009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7b761dd32230a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f655159a700 1 -- 172.21.15.154:0/3114066619 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544011830 con 0x7f65540c3250 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f655159a700 1 -- 172.21.15.154:0/3114066619 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544011990 con 0x7f65540c3250 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f655159a700 1 -- 172.21.15.154:0/3114066619 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544011d20 con 0x7f65540c3250 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 -- 172.21.15.154:0/3114066619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 msgr2=0x7f655413a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 --2- 172.21.15.154:0/3114066619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f655413a8a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6544009840 tx=0x7f6544009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 -- 172.21.15.154:0/3114066619 shutdown_connections 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 --2- 172.21.15.154:0/3114066619 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540c1780 0x7f65540c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 --2- 172.21.15.154:0/3114066619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f655413a8a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 -- 172.21.15.154:0/3114066619 >> 172.21.15.154:0/3114066619 conn(0x7f65540c3880 msgr2=0x7f65540c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.658 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 -- 172.21.15.154:0/3114066619 shutdown_connections 2024-03-14T09:04:42.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.656+0000 7f6558bdd700 1 -- 172.21.15.154:0/3114066619 wait complete. 2024-03-14T09:04:42.661 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 Processor -- start 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 -- start start 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540a8540 0x7f65541ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65540c1780 0x7f655414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541ee800 con 0x7f65540c3250 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541543e0 con 0x7f65540c1780 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6558bdd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6554155080 con 0x7f65540a8540 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6552d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6552d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32784/0 (socket says 172.21.15.154:32784) 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6552d9d700 1 -- 172.21.15.154:0/2069898272 learned_addr learned my addr 172.21.15.154:0/2069898272 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.658+0000 7f6551d9b700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65540c1780 0x7f655414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f655259c700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540a8540 0x7f65541ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 -- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540a8540 msgr2=0x7f65541ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540a8540 0x7f65541ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 -- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65540c1780 msgr2=0x7f655414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65540c1780 0x7f655414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 -- 172.21.15.154:0/2069898272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440094a0 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6552d9d700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b970 tx=0x7f654c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00d8e0 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c00da60 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6554153b40 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00de10 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.659+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6554154100 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.660+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65541555a0 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.661+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f654c015420 con 0x7f65540c3250 2024-03-14T09:04:42.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.661+0000 7f6542ffd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 0x7f653406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.661+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f654c011070 con 0x7f65540c3250 2024-03-14T09:04:42.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.661+0000 7f655259c700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 0x7f653406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.662+0000 7f655259c700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 0x7f653406ccf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6548000f60 tx=0x7f6548007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.665+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f654c04f4f0 con 0x7f65540c3250 2024-03-14T09:04:42.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.859+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f65540697d0 con 0x7f65540c3250 2024-03-14T09:04:42.863 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.859+0000 7f6542ffd700 1 -- 172.21.15.154:0/2069898272 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v77) v1 ==== 101+0+11 (secure 0 0 0) 0x7f654c0540c0 con 0x7f65540c3250 2024-03-14T09:04:42.863 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.861+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 msgr2=0x7f653406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.861+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 0x7f653406ccf0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6548000f60 tx=0x7f6548007c90 comp rx=0 tx=0).stop 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.861+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 msgr2=0x7f65541ea970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.861+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b970 tx=0x7f654c00bc80 comp rx=0 tx=0).stop 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 shutdown_connections 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f653406a830 0x7f653406ccf0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.864 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540c3250 0x7f65541ea970 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65540a8540 0x7f65541ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 --2- 172.21.15.154:0/2069898272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65540c1780 0x7f655414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 >> 172.21.15.154:0/2069898272 conn(0x7f65540c3880 msgr2=0x7f65541336d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 shutdown_connections 2024-03-14T09:04:42.865 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.862+0000 7f6558bdd700 1 -- 172.21.15.154:0/2069898272 wait complete. 2024-03-14T09:04:42.874 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 Processor -- start 2024-03-14T09:04:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 -- start start 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e7814c5c0 0x7f7e78152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e78070e10 con 0x7f7e78072290 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7814c9a0 con 0x7f7e78072710 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e7e034700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e780c1bb0 con 0x7f7e7814c5c0 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.996+0000 7f7e76ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7e78072290 0x7f7e780c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44974/0 (socket says 172.21.15.154:44974) 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e76ffd700 1 -- 172.21.15.154:0/3553203677 learned_addr learned my addr 172.21.15.154:0/3553203677 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e777fe700 1 --2- 172.21.15.154:0/3553203677 >> v2:172.21.15.191:3300/0 conn(0x7f7e7814c5c0 0x7f7e78152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563158341 0 0) 0x7f7e78070e10 con 0x7f7e78072290 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e600035b0 con 0x7f7e78072290 2024-03-14T09:04:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e777fe700 1 -- 172.21.15.154:0/3553203677 >> v1:172.21.15.191:6789/0 conn(0x7f7e78072710 legacy=0x7f7e780c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034369865 0 0) 0x7f7e600035b0 con 0x7f7e78072290 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e777fe700 1 -- 172.21.15.154:0/3553203677 >> v1:172.21.15.154:6789/0 conn(0x7f7e78072290 legacy=0x7f7e780c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e777fe700 1 -- 172.21.15.154:0/3553203677 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e780c1580 con 0x7f7e7814c5c0 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e777fe700 1 --2- 172.21.15.154:0/3553203677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e78152f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009870 tx=0x7f7e68009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0a6dfd274de7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c001670 con 0x7f7e7814c5c0 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68004030 con 0x7f7e7814c5c0 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.997+0000 7f7e767fc700 1 -- 172.21.15.154:0/3553203677 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680043c0 con 0x7f7e7814c5c0 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 -- 172.21.15.154:0/3553203677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 msgr2=0x7f7e78152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 --2- 172.21.15.154:0/3553203677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e78152f80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009870 tx=0x7f7e68009b80 comp rx=0 tx=0).stop 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 -- 172.21.15.154:0/3553203677 shutdown_connections 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 --2- 172.21.15.154:0/3553203677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e78152f80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 -- 172.21.15.154:0/3553203677 >> 172.21.15.154:0/3553203677 conn(0x7f7e7806a770 msgr2=0x7f7e7806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 -- 172.21.15.154:0/3553203677 shutdown_connections 2024-03-14T09:04:43.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.998+0000 7f7e7e034700 1 -- 172.21.15.154:0/3553203677 wait complete. 2024-03-14T09:04:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.999+0000 7f7e7e034700 1 Processor -- start 2024-03-14T09:04:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:42.999+0000 7f7e7e034700 1 -- start start 2024-03-14T09:04:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e78072290 0x7f7e7814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e781dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78150030 con 0x7f7e78072710 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7814f820 con 0x7f7e7814c5c0 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e7e034700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7814d000 con 0x7f7e78072290 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32798/0 (socket says 172.21.15.154:32798) 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 -- 172.21.15.154:0/1969985126 learned_addr learned my addr 172.21.15.154:0/1969985126 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e777fe700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e78072290 0x7f7e7814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e77fff700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e781dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 -- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e78072290 msgr2=0x7f7e7814dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e78072290 0x7f7e7814dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 -- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 msgr2=0x7f7e781dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e781dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.000+0000 7f7e76ffd700 1 -- 172.21.15.154:0/1969985126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680094a0 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e76ffd700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c780 tx=0x7f7e6c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c004c80 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c004e00 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7814e1e0 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c007310 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.001+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7814fa70 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.002+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e780697d0 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.002+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7e6c007470 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.003+0000 7f7e5ffff700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 0x7f7e6406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.003+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f7e6c088a30 con 0x7f7e78072710 2024-03-14T09:04:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.003+0000 7f7e777fe700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 0x7f7e6406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.005 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.003+0000 7f7e777fe700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 0x7f7e6406cd20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009840 tx=0x7f7e68008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.006 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.007+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7e6c053e80 con 0x7f7e78072710 2024-03-14T09:04:43.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.244+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e780c5a00 con 0x7f7e78072710 2024-03-14T09:04:43.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.245+0000 7f7e5ffff700 1 -- 172.21.15.154:0/1969985126 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f7e6c0574a0 con 0x7f7e78072710 2024-03-14T09:04:43.245 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:43.245 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1132667086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1132667086},{"type":"v1","addr":"172.21.15.154:6835","nonce":1132667086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:43.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 msgr2=0x7f7e6406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.247 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 0x7f7e6406cd20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009840 tx=0x7f7e68008030 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 msgr2=0x7f7e7814d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c780 tx=0x7f7e6c00ca90 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 shutdown_connections 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e7814c5c0 0x7f7e781dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e78072290 0x7f7e7814dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7e6406a860 0x7f7e6406cd20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 --2- 172.21.15.154:0/1969985126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e78072710 0x7f7e7814d490 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 >> 172.21.15.154:0/1969985126 conn(0x7f7e7806a770 msgr2=0x7f7e78155900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 shutdown_connections 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.247+0000 7f7e7e034700 1 -- 172.21.15.154:0/1969985126 wait complete. 2024-03-14T09:04:43.248 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 52 2024-03-14T09:04:43.259 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:43.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.379+0000 7f865c8d4700 1 Processor -- start 2024-03-14T09:04:43.379 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.379+0000 7f865c8d4700 1 -- start start 2024-03-14T09:04:43.381 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.380+0000 7f865c8d4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86580702f0 0x7f86580c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.380+0000 7f865c8d4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86580acd10 con 0x7f865814fbd0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.380+0000 7f865c8d4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8658150050 con 0x7f86580c5d60 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.380+0000 7f865c8d4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86580ac7b0 con 0x7f86580702f0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.380+0000 7f865659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86580702f0 0x7f86580c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86580702f0 0x7f86580c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32800/0 (socket says 172.21.15.154:32800) 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f8655d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f865814fbd0 0x7f86580c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44976/0 (socket says 172.21.15.154:44976) 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 -- 172.21.15.154:0/745098792 learned_addr learned my addr 172.21.15.154:0/745098792 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 -- 172.21.15.154:0/745098792 >> v1:172.21.15.191:6789/0 conn(0x7f86580c5d60 legacy=0x7f865815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 -- 172.21.15.154:0/745098792 >> v1:172.21.15.154:6789/0 conn(0x7f865814fbd0 legacy=0x7f86580c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865559a700 1 -- 172.21.15.154:0/745098792 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185449206 0 0) 0x7f86580acd10 con 0x7f865814fbd0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 -- 172.21.15.154:0/745098792 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580ac9c0 con 0x7f86580702f0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865659c700 1 --2- 172.21.15.154:0/745098792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86580c4380 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f86400098f0 tx=0x7f8640009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f084b0836f703b server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.382 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865559a700 1 -- 172.21.15.154:0/745098792 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640004230 con 0x7f86580702f0 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865559a700 1 -- 172.21.15.154:0/745098792 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8640004390 con 0x7f86580702f0 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.381+0000 7f865559a700 1 -- 172.21.15.154:0/745098792 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400036a0 con 0x7f86580702f0 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 -- 172.21.15.154:0/745098792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 msgr2=0x7f86580c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 --2- 172.21.15.154:0/745098792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86580c4380 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f86400098f0 tx=0x7f8640009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 -- 172.21.15.154:0/745098792 shutdown_connections 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 --2- 172.21.15.154:0/745098792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86580c4380 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 -- 172.21.15.154:0/745098792 >> 172.21.15.154:0/745098792 conn(0x7f86580a9bf0 msgr2=0x7f86580a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 -- 172.21.15.154:0/745098792 shutdown_connections 2024-03-14T09:04:43.383 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.382+0000 7f865c8d4700 1 -- 172.21.15.154:0/745098792 wait complete. 2024-03-14T09:04:43.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.383+0000 7f865c8d4700 1 Processor -- start 2024-03-14T09:04:43.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.383+0000 7f865c8d4700 1 -- start start 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 0x7f8658155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f865814fbd0 0x7f86581ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658151ef0 con 0x7f86580702f0 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86581561c0 con 0x7f865814fbd0 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658150750 con 0x7f86580c5d60 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32814/0 (socket says 172.21.15.154:32814) 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 -- 172.21.15.154:0/609056069 learned_addr learned my addr 172.21.15.154:0/609056069 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f8655d9b700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 0x7f8658155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f8656d9d700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f865814fbd0 0x7f86581ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 -- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 msgr2=0x7f8658155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.386 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 0x7f8658155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 -- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f865814fbd0 msgr2=0x7f86581ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f865814fbd0 0x7f86581ebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 -- 172.21.15.154:0/609056069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009550 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f8655d9b700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 0x7f8658155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865659c700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f865806b710 tx=0x7f8640004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640003d80 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864001f070 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8658150a30 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.384+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000b8e0 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.385+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8658155480 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.385+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8658074110 con 0x7f86580702f0 2024-03-14T09:04:43.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.386+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8640021030 con 0x7f86580702f0 2024-03-14T09:04:43.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.386+0000 7f864effd700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 0x7f863806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.386+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f864008a4d0 con 0x7f86580702f0 2024-03-14T09:04:43.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.387+0000 7f8655d9b700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 0x7f863806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.387+0000 7f8655d9b700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 0x7f863806cd20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8658149940 tx=0x7f8648007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.391+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8640055920 con 0x7f86580702f0 2024-03-14T09:04:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.588+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86580c3b90 con 0x7f86580702f0 2024-03-14T09:04:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.589+0000 7f864effd700 1 -- 172.21.15.154:0/609056069 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f8640058f40 con 0x7f86580702f0 2024-03-14T09:04:43.589 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:43.589 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":77,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:41.467922+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/1688485831":"2024-03-15T09:04:38.103230+0000","172.21.15.154:6833/1688485831":"2024-03-15T09:04:38.103230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 msgr2=0x7f863806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 0x7f863806cd20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8658149940 tx=0x7f8648007d30 comp rx=0 tx=0).stop 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 msgr2=0x7f86581519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f865806b710 tx=0x7f8640004750 comp rx=0 tx=0).stop 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 shutdown_connections 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f865814fbd0 0x7f86581ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f863806a860 0x7f863806cd20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86580702f0 0x7f86581519b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 --2- 172.21.15.154:0/609056069 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86580c5d60 0x7f8658155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 >> 172.21.15.154:0/609056069 conn(0x7f86580a9bf0 msgr2=0x7f86580c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 shutdown_connections 2024-03-14T09:04:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.591+0000 7f865c8d4700 1 -- 172.21.15.154:0/609056069 wait complete. 2024-03-14T09:04:43.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-14T09:04:43.604 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:43.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.725+0000 7f3b8da3e700 1 Processor -- start 2024-03-14T09:04:43.725 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.725+0000 7f3b8da3e700 1 -- start start 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b8da3e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b8814e980 0x7f3b881552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b8da3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b88158a80 0x7f3b880c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b8da3e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8814ed60 con 0x7f3b880c4a80 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b8da3e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b880c6b10 con 0x7f3b8814e980 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b8da3e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88155820 con 0x7f3b88158a80 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b8814e980 0x7f3b881552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b8814e980 0x7f3b881552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32816/0 (socket says 172.21.15.154:32816) 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 -- 172.21.15.154:0/589953916 learned_addr learned my addr 172.21.15.154:0/589953916 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b877fe700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b88158a80 0x7f3b880c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 -- 172.21.15.154:0/589953916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b88158a80 msgr2=0x7f3b880c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b88158a80 0x7f3b880c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 -- 172.21.15.154:0/589953916 >> v1:172.21.15.191:6789/0 conn(0x7f3b880c4a80 legacy=0x7f3b88158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.726+0000 7f3b86ffd700 1 -- 172.21.15.154:0/589953916 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880c6cf0 con 0x7f3b8814e980 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b86ffd700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8814e980 0x7f3b881552e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0098f0 tx=0x7f3b7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e5d39a5bbaa401 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b85ffb700 1 -- 172.21.15.154:0/589953916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00b8a0 con 0x7f3b8814e980 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b85ffb700 1 -- 172.21.15.154:0/589953916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00ba00 con 0x7f3b8814e980 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b85ffb700 1 -- 172.21.15.154:0/589953916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00bd90 con 0x7f3b8814e980 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 -- 172.21.15.154:0/589953916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8814e980 msgr2=0x7f3b881552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8814e980 0x7f3b881552e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0098f0 tx=0x7f3b7c009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 -- 172.21.15.154:0/589953916 shutdown_connections 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b88158a80 0x7f3b880c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/589953916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b8814e980 0x7f3b881552e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 -- 172.21.15.154:0/589953916 >> 172.21.15.154:0/589953916 conn(0x7f3b880abcf0 msgr2=0x7f3b880ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 -- 172.21.15.154:0/589953916 shutdown_connections 2024-03-14T09:04:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.727+0000 7f3b8da3e700 1 -- 172.21.15.154:0/589953916 wait complete. 2024-03-14T09:04:43.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 Processor -- start 2024-03-14T09:04:43.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 -- start start 2024-03-14T09:04:43.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880c4a80 0x7f3b881ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 0x7f3b8814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.732 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b881eb650 con 0x7f3b88158a80 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b881edb90 con 0x7f3b8814e980 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b8da3e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b881ede70 con 0x7f3b880c4a80 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.729+0000 7f3b877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32818/0 (socket says 172.21.15.154:32818) 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 -- 172.21.15.154:0/2929113482 learned_addr learned my addr 172.21.15.154:0/2929113482 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b867fc700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 0x7f3b8814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b86ffd700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880c4a80 0x7f3b881ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 -- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880c4a80 msgr2=0x7f3b881ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880c4a80 0x7f3b881ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 -- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 msgr2=0x7f3b8814b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 0x7f3b8814b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 -- 172.21.15.154:0/2929113482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c009550 con 0x7f3b88158a80 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b867fc700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 0x7f3b8814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:43.733 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b877fe700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3b80007e40 tx=0x7f3b8000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000eea0 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b80014420 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b881ee150 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800147b0 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.730+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b88155310 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.731+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b881eb790 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.732+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f3b80014910 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.732+0000 7f3b6ffff700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 0x7f3b7406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.732+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f3b800894e0 con 0x7f3b88158a80 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.732+0000 7f3b86ffd700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 0x7f3b7406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.733+0000 7f3b86ffd700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 0x7f3b7406ccf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3b8814cd30 tx=0x7f3b7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:43.736 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.737+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f3b80054930 con 0x7f3b88158a80 2024-03-14T09:04:43.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.974+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b880697d0 con 0x7f3b88158a80 2024-03-14T09:04:43.974 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.975+0000 7f3b6ffff700 1 -- 172.21.15.154:0/2929113482 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f3b80057f50 con 0x7f3b88158a80 2024-03-14T09:04:43.975 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:43.975 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1132667086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1132667086},{"type":"v1","addr":"172.21.15.154:6835","nonce":1132667086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 msgr2=0x7f3b7406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 0x7f3b7406ccf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3b8814cd30 tx=0x7f3b7c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 msgr2=0x7f3b880aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3b80007e40 tx=0x7f3b8000e3c0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 shutdown_connections 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f3b7406a830 0x7f3b7406ccf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.977 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b88158a80 0x7f3b880aa6a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b880c4a80 0x7f3b881ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 --2- 172.21.15.154:0/2929113482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b8814e980 0x7f3b8814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 >> 172.21.15.154:0/2929113482 conn(0x7f3b880abcf0 msgr2=0x7f3b8806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 shutdown_connections 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:43.977+0000 7f3b8da3e700 1 -- 172.21.15.154:0/2929113482 wait complete. 2024-03-14T09:04:43.978 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 52 2024-03-14T09:04:43.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:04:44.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.110+0000 7f8670a4c700 1 Processor -- start 2024-03-14T09:04:44.110 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.110+0000 7f8670a4c700 1 -- start start 2024-03-14T09:04:44.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8670a4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f866c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8670a4c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c156bf0 con 0x7f866c0c1780 2024-03-14T09:04:44.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8670a4c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c156050 con 0x7f866c0a9650 2024-03-14T09:04:44.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8670a4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0c1c00 con 0x7f866c15bfd0 2024-03-14T09:04:44.116 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8669d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f866c0c1780 0x7f866c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44980/0 (socket says 172.21.15.154:44980) 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f8669d9b700 1 -- 172.21.15.154:0/511511884 learned_addr learned my addr 172.21.15.154:0/511511884 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866ad9d700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f866c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280381965 0 0) 0x7f865c001110 con 0x7f866c0c1780 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86540035b0 con 0x7f866c0c1780 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019191411 0 0) 0x7f866c156050 con 0x7f866c0a9650 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f865c001110 con 0x7f866c0a9650 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.111+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1343948720 0 0) 0x7f86540035b0 con 0x7f866c0c1780 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 msgr2=0x7f866c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f866c15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 >> v1:172.21.15.191:6789/0 conn(0x7f866c0a9650 legacy=0x7f866c1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0a86f0 con 0x7f866c0c1780 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f865c001800 con 0x7f866c0c1780 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 >> v1:172.21.15.154:6789/0 conn(0x7f866c0c1780 legacy=0x7f866c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8654003bc0 0x7f8654003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f8654004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f86540052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0c1c00 con 0x7f8654004850 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865400bc00 con 0x7f8654003bc0 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865400bdc0 con 0x7f866c15bfd0 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f8654004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866a59c700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8654003bc0 0x7f8654003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866ad9d700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f86540052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 -- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 msgr2=0x7f86540052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c15bfd0 0x7f86540052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 -- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8654003bc0 msgr2=0x7f8654003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8654003bc0 0x7f8654003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 -- 172.21.15.154:0/511511884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540035b0 con 0x7f8654004850 2024-03-14T09:04:44.118 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f8669d9b700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f8654004cd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c1c0 tx=0x7f8654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246f5d1e04d7ed58 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00cd00 con 0x7f8654004850 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00ce60 con 0x7f8654004850 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.112+0000 7f866959a700 1 -- 172.21.15.154:0/511511884 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c005ba0 con 0x7f8654004850 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 -- 172.21.15.154:0/511511884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 msgr2=0x7f8654004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 --2- 172.21.15.154:0/511511884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f8654004cd0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c1c0 tx=0x7f8654005880 comp rx=0 tx=0).stop 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f866ad9d700 1 -- 172.21.15.154:0/511511884 reap_dead start 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 -- 172.21.15.154:0/511511884 shutdown_connections 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 -- 172.21.15.154:0/511511884 >> 172.21.15.154:0/511511884 conn(0x7f866c0c3880 msgr2=0x7f866c0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 -- 172.21.15.154:0/511511884 shutdown_connections 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.113+0000 7f8670a4c700 1 -- 172.21.15.154:0/511511884 wait complete. 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.114+0000 7f8670a4c700 1 Processor -- start 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.114+0000 7f8670a4c700 1 -- start start 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f866c14a160 0x7f866c1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.119 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 0x7f866c143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c14e6a0 con 0x7f8654004850 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1431d0 con 0x7f866c14a160 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8670a4c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c13b650 con 0x7f866c14b750 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32830/0 (socket says 172.21.15.154:32830) 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 -- 172.21.15.154:0/2444656410 learned_addr learned my addr 172.21.15.154:0/2444656410 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f8669d9b700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f866c14a160 0x7f866c1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866ad9d700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 0x7f866c143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 -- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 msgr2=0x7f866c143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 0x7f866c143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 -- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f866c14a160 msgr2=0x7f866c1473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f866c14a160 0x7f866c1473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 -- 172.21.15.154:0/2444656410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c143420 con 0x7f8654004850 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866ad9d700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 0x7f866c143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:44.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.115+0000 7f866a59c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f866000f770 tx=0x7f866000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.116+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660005320 con 0x7f8654004850 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.116+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660005480 con 0x7f8654004850 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.116+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c143700 con 0x7f8654004850 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.116+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660006aa0 con 0x7f8654004850 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.116+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c147900 con 0x7f8654004850 2024-03-14T09:04:44.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.117+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c0697d0 con 0x7f8654004850 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.117+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f8660006c00 con 0x7f8654004850 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.117+0000 7f865affd700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 0x7f864c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.117+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f86600885b0 con 0x7f8654004850 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.118+0000 7f8669d9b700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 0x7f864c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.118+0000 7f8669d9b700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 0x7f864c06ccf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d010 tx=0x7f865c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.122+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f8660053a00 con 0x7f8654004850 2024-03-14T09:04:44.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.360+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f866c06b470 con 0x7f8654004850 2024-03-14T09:04:44.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.360+0000 7f865affd700 1 -- 172.21.15.154:0/2444656410 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4750 (secure 0 0 0) 0x7f8660057020 con 0x7f8654004850 2024-03-14T09:04:44.360 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:44.360 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1132667086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1132667086},{"type":"v1","addr":"172.21.15.154:6835","nonce":1132667086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-14T09:04:09.768470+0000","modified":"2024-03-14T09:04:12.782892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6438":{"gid":6438,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.154:6834/3041722362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3041722362},{"type":"v1","addr":"172.21.15.154:6834","nonce":3041722362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 msgr2=0x7f864c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 0x7f864c06ccf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d010 tx=0x7f865c00a000 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 msgr2=0x7f866c14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f866000f770 tx=0x7f866000fb30 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 shutdown_connections 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f866c14b750 0x7f866c143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f864c06a830 0x7f864c06ccf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8654004850 0x7f866c14e160 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 --2- 172.21.15.154:0/2444656410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f866c14a160 0x7f866c1473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.362+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 >> 172.21.15.154:0/2444656410 conn(0x7f866c0c3880 msgr2=0x7f866c1563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.363+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 shutdown_connections 2024-03-14T09:04:44.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.363+0000 7f8670a4c700 1 -- 172.21.15.154:0/2444656410 wait complete. 2024-03-14T09:04:44.364 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 52 2024-03-14T09:04:44.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.498+0000 7fe5eccdc700 1 Processor -- start 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.498+0000 7fe5eccdc700 1 -- start start 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5eccdc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe5e814e980 0x7fe5e81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5eccdc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 0x7fe5e80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5eccdc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e814ed60 con 0x7fe5e80c4a80 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5eccdc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e80c6b10 con 0x7fe5e814e980 2024-03-14T09:04:44.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5eccdc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8155820 con 0x7fe5e8158a80 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe5e814e980 0x7fe5e81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe5e814e980 0x7fe5e81552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32838/0 (socket says 172.21.15.154:32838) 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 -- 172.21.15.154:0/1185428614 learned_addr learned my addr 172.21.15.154:0/1185428614 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e6d9d700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 0x7fe5e80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 -- 172.21.15.154:0/1185428614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 msgr2=0x7fe5e80c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 0x7fe5e80c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 -- 172.21.15.154:0/1185428614 >> v1:172.21.15.191:6789/0 conn(0x7fe5e80c4a80 legacy=0x7fe5e8158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 -- 172.21.15.154:0/1185428614 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80c6cf0 con 0x7fe5e814e980 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e6d9d700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 0x7fe5e80c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e659c700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e81552e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098f0 tx=0x7fe5d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc3fb48f857888 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e559a700 1 -- 172.21.15.154:0/1185428614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800b8a0 con 0x7fe5e814e980 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e559a700 1 -- 172.21.15.154:0/1185428614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d800ba00 con 0x7fe5e814e980 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.499+0000 7fe5e559a700 1 -- 172.21.15.154:0/1185428614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800bd90 con 0x7fe5e814e980 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 -- 172.21.15.154:0/1185428614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 msgr2=0x7fe5e81552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e81552e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098f0 tx=0x7fe5d8009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 -- 172.21.15.154:0/1185428614 shutdown_connections 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e8158a80 0x7fe5e80c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/1185428614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e81552e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 -- 172.21.15.154:0/1185428614 >> 172.21.15.154:0/1185428614 conn(0x7fe5e80abcf0 msgr2=0x7fe5e80ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 -- 172.21.15.154:0/1185428614 shutdown_connections 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.500+0000 7fe5eccdc700 1 -- 172.21.15.154:0/1185428614 wait complete. 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.501+0000 7fe5eccdc700 1 Processor -- start 2024-03-14T09:04:44.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.501+0000 7fe5eccdc700 1 -- start start 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e80c4a80 0x7fe5e80ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8158a80 0x7fe5e80ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e81ed990 con 0x7fe5e814e980 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80af620 con 0x7fe5e8158a80 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5eccdc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80af900 con 0x7fe5e80c4a80 2024-03-14T09:04:44.506 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32842/0 (socket says 172.21.15.154:32842) 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 -- 172.21.15.154:0/2786036774 learned_addr learned my addr 172.21.15.154:0/2786036774 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e659c700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e80c4a80 0x7fe5e80ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e6d9d700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8158a80 0x7fe5e80ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 -- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e80c4a80 msgr2=0x7fe5e80ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e80c4a80 0x7fe5e80ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 -- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8158a80 msgr2=0x7fe5e80ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8158a80 0x7fe5e80ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.502+0000 7fe5e5d9b700 1 -- 172.21.15.154:0/2786036774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8009550 con 0x7fe5e814e980 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5e5d9b700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00c450 tx=0x7fe5dc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0052d0 con 0x7fe5e814e980 2024-03-14T09:04:44.507 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc005430 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e80afbe0 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0038c0 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.503+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80adf80 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.504+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e81edb00 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.504+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe5dc003a20 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.505+0000 7fe5d6ffd700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 0x7fe5c806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.505+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7fe5dc088070 con 0x7fe5e814e980 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.505+0000 7fe5e659c700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 0x7fe5c806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.505+0000 7fe5e659c700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 0x7fe5c806ccf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009c40 tx=0x7fe5d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.509 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.509+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe5dc053440 con 0x7fe5e814e980 2024-03-14T09:04:44.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.703+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e80697d0 con 0x7fe5e814e980 2024-03-14T09:04:44.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.704+0000 7fe5d6ffd700 1 -- 172.21.15.154:0/2786036774 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16223 (secure 0 0 0) 0x7fe5dc00ce60 con 0x7fe5e814e980 2024-03-14T09:04:44.705 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:44.705 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":77,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:41.467922+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_metadata","create_time":"2024-03-14T09:04:07.885228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs2_data","create_time":"2024-03-14T09:04:08.739578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/1688485831":"2024-03-15T09:04:38.103230+0000","172.21.15.154:6833/1688485831":"2024-03-15T09:04:38.103230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 msgr2=0x7fe5c806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 0x7fe5c806ccf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009c40 tx=0x7fe5d8008030 comp rx=0 tx=0).stop 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 msgr2=0x7fe5e8153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00c450 tx=0x7fe5dc00c760 comp rx=0 tx=0).stop 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 shutdown_connections 2024-03-14T09:04:44.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5e8158a80 0x7fe5e80ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5e80c4a80 0x7fe5e80ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe5c806a830 0x7fe5c806ccf0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 --2- 172.21.15.154:0/2786036774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5e814e980 0x7fe5e8153bf0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 >> 172.21.15.154:0/2786036774 conn(0x7fe5e80abcf0 msgr2=0x7fe5e8155e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 shutdown_connections 2024-03-14T09:04:44.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.707+0000 7fe5eccdc700 1 -- 172.21.15.154:0/2786036774 wait complete. 2024-03-14T09:04:44.719 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-14T09:04:44.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.843+0000 7f486265c700 1 Processor -- start 2024-03-14T09:04:44.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 -- start start 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c7f70 0x7f485c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f485c0c8350 0x7f485c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c157e80 con 0x7f485c0c4f30 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c158ac0 con 0x7f485c0c8350 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.844+0000 7f486265c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c158120 con 0x7f485c0c7f70 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f485c0c8350 0x7f485c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c7f70 0x7f485c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485b7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f485c0c4f30 0x7f485c1547f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:54300/0 (socket says 172.21.15.154:54300) 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f485c0c8350 0x7f485c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:40288/0 (socket says 172.21.15.154:40288) 2024-03-14T09:04:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485b7fe700 1 -- 172.21.15.154:0/363448767 learned_addr learned my addr 172.21.15.154:0/363448767 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 -- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c7f70 msgr2=0x7f485c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 --2- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c7f70 0x7f485c0c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 -- 172.21.15.154:0/363448767 >> v1:172.21.15.191:6789/0 conn(0x7f485c0c4f30 legacy=0x7f485c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485affd700 1 -- 172.21.15.154:0/363448767 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853158559 0 0) 0x7f485c157e80 con 0x7f485c0c4f30 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 -- 172.21.15.154:0/363448767 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c158400 con 0x7f485c0c8350 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f4860bf9700 1 --2- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c8350 0x7f485c154f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4854009d90 tx=0x7f485400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f6ad5b3803e6b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485affd700 1 -- 172.21.15.154:0/363448767 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400e070 con 0x7f485c0c8350 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485affd700 1 -- 172.21.15.154:0/363448767 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485400bb20 con 0x7f485c0c8350 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.845+0000 7f485affd700 1 -- 172.21.15.154:0/363448767 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854012430 con 0x7f485c0c8350 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 -- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c8350 msgr2=0x7f485c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 --2- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c8350 0x7f485c154f00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4854009d90 tx=0x7f485400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 -- 172.21.15.154:0/363448767 shutdown_connections 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 --2- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c8350 0x7f485c154f00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 --2- 172.21.15.154:0/363448767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c7f70 0x7f485c0c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 -- 172.21.15.154:0/363448767 >> 172.21.15.154:0/363448767 conn(0x7f485c072080 msgr2=0x7f485c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 -- 172.21.15.154:0/363448767 shutdown_connections 2024-03-14T09:04:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.846+0000 7f486265c700 1 -- 172.21.15.154:0/363448767 wait complete. 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.847+0000 7f486265c700 1 Processor -- start 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.847+0000 7f486265c700 1 -- start start 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c7f70 0x7f485c149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c8350 0x7f485c149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1491a0 con 0x7f485c0c4f30 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c148590 con 0x7f485c0c7f70 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f486265c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c148870 con 0x7f485c0c8350 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32844/0 (socket says 172.21.15.154:32844) 2024-03-14T09:04:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 -- 172.21.15.154:0/4154988385 learned_addr learned my addr 172.21.15.154:0/4154988385 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485b7fe700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c7f70 0x7f485c149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f4860bf9700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c8350 0x7f485c149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 -- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c8350 msgr2=0x7f485c149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c8350 0x7f485c149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 -- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c7f70 msgr2=0x7f485c149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c7f70 0x7f485c149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.848+0000 7f485bfff700 1 -- 172.21.15.154:0/4154988385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009a70 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f485bfff700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f485000e870 tx=0x7f485000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850009590 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850013680 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c148b50 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48500139f0 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.849+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c14b0d0 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.850+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c149480 con 0x7f485c0c4f30 2024-03-14T09:04:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.850+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4850013b50 con 0x7f485c0c4f30 2024-03-14T09:04:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.851+0000 7f4858ff9700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 0x7f484806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.851+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5799+0+0 (secure 0 0 0) 0x7f4850089080 con 0x7f485c0c4f30 2024-03-14T09:04:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.851+0000 7f485b7fe700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 0x7f484806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.851+0000 7f485b7fe700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 0x7f484806ccf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f485c14c3d0 tx=0x7f484c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:44.855+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f48500bc4f0 con 0x7f485c0c4f30 2024-03-14T09:04:45.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.092+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f485c0697d0 con 0x7f485c0c4f30 2024-03-14T09:04:45.093 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-03-14T09:04:45.093+0000 7f2fd022d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-14T09:04:45.100 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.100+0000 7f4858ff9700 1 -- 172.21.15.154:0/4154988385 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 163+0+0 (secure 0 0 0) 0x7f48500543b0 con 0x7f485c0c4f30 2024-03-14T09:04:45.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 msgr2=0x7f484806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 0x7f484806ccf0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f485c14c3d0 tx=0x7f484c007e70 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 msgr2=0x7f485c14ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f485000e870 tx=0x7f485000eb80 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 shutdown_connections 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f485c0c8350 0x7f485c149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f484806a830 0x7f484806ccf0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f485c0c4f30 0x7f485c14ab90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 --2- 172.21.15.154:0/4154988385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f485c0c7f70 0x7f485c149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.102+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 >> 172.21.15.154:0/4154988385 conn(0x7f485c072080 msgr2=0x7f485c156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.103+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 shutdown_connections 2024-03-14T09:04:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.103+0000 7f486265c700 1 -- 172.21.15.154:0/4154988385 wait complete. 2024-03-14T09:04:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-14T09:04:45.104 INFO:tasks.ceph.mds.c.smithi154.stderr: -5818> 2024-03-14T09:04:12.671+0000 7f6783001700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-14T09:04:45.116 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-14T09:04:45.136 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2024-03-14T09:04:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.241+0000 7fdeaad0e700 1 Processor -- start 2024-03-14T09:04:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.241+0000 7fdeaad0e700 1 -- start start 2024-03-14T09:04:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.241+0000 7fdeaad0e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdea40c26c0 0x7fdea414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.241+0000 7fdeaad0e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdea40c5a30 0x7fdea4151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdeaad0e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea4155a40 con 0x7fdea40c55a0 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdeaad0e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40c5e60 con 0x7fdea40c26c0 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdeaad0e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea40c2f10 con 0x7fdea40c5a30 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdea40c26c0 0x7fdea414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdea40c26c0 0x7fdea414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32860/0 (socket says 172.21.15.154:32860) 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 -- 172.21.15.154:0/909577797 learned_addr learned my addr 172.21.15.154:0/909577797 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea92ab700 1 --2- 172.21.15.154:0/909577797 >> v2:172.21.15.191:3300/0 conn(0x7fdea40c5a30 0x7fdea4151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 -- 172.21.15.154:0/909577797 >> v2:172.21.15.191:3300/0 conn(0x7fdea40c5a30 msgr2=0x7fdea4151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 --2- 172.21.15.154:0/909577797 >> v2:172.21.15.191:3300/0 conn(0x7fdea40c5a30 0x7fdea4151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 -- 172.21.15.154:0/909577797 >> v1:172.21.15.191:6789/0 conn(0x7fdea40c55a0 legacy=0x7fdea41518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 -- 172.21.15.154:0/909577797 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea4072a90 con 0x7fdea40c26c0 2024-03-14T09:04:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea8aaa700 1 --2- 172.21.15.154:0/909577797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea414edf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdea406f450 tx=0x7fde98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841ed39ef735dc07 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea37fe700 1 -- 172.21.15.154:0/909577797 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800b860 con 0x7fdea40c26c0 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea37fe700 1 -- 172.21.15.154:0/909577797 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800b9c0 con 0x7fdea40c26c0 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.242+0000 7fdea37fe700 1 -- 172.21.15.154:0/909577797 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800bd50 con 0x7fdea40c26c0 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 -- 172.21.15.154:0/909577797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 msgr2=0x7fdea414edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/909577797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea414edf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdea406f450 tx=0x7fde98009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 -- 172.21.15.154:0/909577797 shutdown_connections 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/909577797 >> v2:172.21.15.191:3300/0 conn(0x7fdea40c5a30 0x7fdea4151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/909577797 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea414edf0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 -- 172.21.15.154:0/909577797 >> 172.21.15.154:0/909577797 conn(0x7fdea406a770 msgr2=0x7fdea406ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 -- 172.21.15.154:0/909577797 shutdown_connections 2024-03-14T09:04:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.243+0000 7fdeaad0e700 1 -- 172.21.15.154:0/909577797 wait complete. 2024-03-14T09:04:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.244+0000 7fdeaad0e700 1 Processor -- start 2024-03-14T09:04:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.244+0000 7fdeaad0e700 1 -- start start 2024-03-14T09:04:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.245+0000 7fdeaad0e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.245+0000 7fdeaad0e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 0x7fdea414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.245+0000 7fdeaad0e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40c5a30 0x7fdea41ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.245+0000 7fdeaad0e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea40abb90 con 0x7fdea40c26c0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.245+0000 7fdeaad0e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea414c1e0 con 0x7fdea40c5a30 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdeaad0e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41ee770 con 0x7fdea40c55a0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32876/0 (socket says 172.21.15.154:32876) 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 -- 172.21.15.154:0/274079696 learned_addr learned my addr 172.21.15.154:0/274079696 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea92ab700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40c5a30 0x7fdea41ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea3fff700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 0x7fdea414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 -- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 msgr2=0x7fdea414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 0x7fdea414bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 -- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40c5a30 msgr2=0x7fdea41ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40c5a30 0x7fdea41ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 -- 172.21.15.154:0/274079696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98009550 con 0x7fdea40c26c0 2024-03-14T09:04:45.249 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea3fff700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 0x7fdea414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea8aaa700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fde98009c40 tx=0x7fde98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980036a0 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98003820 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea4150d90 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98003bd0 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.246+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea40a9a50 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.247+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea41512b0 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.248+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fde98021070 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.248+0000 7fdea17fa700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 0x7fde8806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.248+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5885+0+0 (secure 0 0 0) 0x7fde9808a2f0 con 0x7fdea40c26c0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.248+0000 7fdea3fff700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 0x7fde8806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:45.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.249+0000 7fdea3fff700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 0x7fde8806ccf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdea4151870 tx=0x7fde90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:45.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.253+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fdea41512b0 con 0x7fdea40c26c0 2024-03-14T09:04:45.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:45.492+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdea40a98b0 con 0x7fdea40c26c0 2024-03-14T09:04:46.102 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.102+0000 7fdea17fa700 1 -- 172.21.15.154:0/274079696 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdea40a98b0 con 0x7fdea40c26c0 2024-03-14T09:04:46.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 msgr2=0x7fde8806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:46.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 0x7fde8806ccf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdea4151870 tx=0x7fde90009040 comp rx=0 tx=0).stop 2024-03-14T09:04:46.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 msgr2=0x7fdea41507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:46.105 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fde98009c40 tx=0x7fde98004750 comp rx=0 tx=0).stop 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 shutdown_connections 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdea40c5a30 0x7fdea41ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fde8806a830 0x7fde8806ccf0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdea40c26c0 0x7fdea41507b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 --2- 172.21.15.154:0/274079696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdea40c55a0 0x7fdea414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 >> 172.21.15.154:0/274079696 conn(0x7fdea406a770 msgr2=0x7fdea406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 shutdown_connections 2024-03-14T09:04:46.106 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.105+0000 7fdeaad0e700 1 -- 172.21.15.154:0/274079696 wait complete. 2024-03-14T09:04:46.120 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-14T09:04:46.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.293+0000 7f1c7712d700 1 Processor -- start 2024-03-14T09:04:46.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.293+0000 7f1c7712d700 1 -- start start 2024-03-14T09:04:46.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c7712d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1c7013cbf0 0x7f1c7013d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c7712d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c70157e40 0x7f1c701582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c7712d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7006b3f0 con 0x7f1c7006cfe0 2024-03-14T09:04:46.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c7712d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7013d680 con 0x7f1c70157e40 2024-03-14T09:04:46.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c7712d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c700c4500 con 0x7f1c7013cbf0 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c70157e40 0x7f1c701582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1c70157e40 0x7f1c701582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32892/0 (socket says 172.21.15.154:32892) 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 -- 172.21.15.154:0/1143890366 learned_addr learned my addr 172.21.15.154:0/1143890366 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c6ffff700 1 --2- 172.21.15.154:0/1143890366 >> v2:172.21.15.191:3300/0 conn(0x7f1c7013cbf0 0x7f1c7013d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 -- 172.21.15.154:0/1143890366 >> v2:172.21.15.191:3300/0 conn(0x7f1c7013cbf0 msgr2=0x7f1c7013d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 --2- 172.21.15.154:0/1143890366 >> v2:172.21.15.191:3300/0 conn(0x7f1c7013cbf0 0x7f1c7013d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 -- 172.21.15.154:0/1143890366 >> v1:172.21.15.191:6789/0 conn(0x7f1c7006cfe0 legacy=0x7f1c7006a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 -- 172.21.15.154:0/1143890366 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7013dd70 con 0x7f1c70157e40 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c756ca700 1 --2- 172.21.15.154:0/1143890366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701582b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1c70072a00 tx=0x7f1c6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b592be8593ec2a87 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c6f7fe700 1 -- 172.21.15.154:0/1143890366 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6800e070 con 0x7f1c70157e40 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c6f7fe700 1 -- 172.21.15.154:0/1143890366 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68003a20 con 0x7f1c70157e40 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.294+0000 7f1c6f7fe700 1 -- 172.21.15.154:0/1143890366 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68003dd0 con 0x7f1c70157e40 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 -- 172.21.15.154:0/1143890366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 msgr2=0x7f1c701582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:46.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 --2- 172.21.15.154:0/1143890366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701582b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1c70072a00 tx=0x7f1c6800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 -- 172.21.15.154:0/1143890366 shutdown_connections 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 --2- 172.21.15.154:0/1143890366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701582b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 --2- 172.21.15.154:0/1143890366 >> v2:172.21.15.191:3300/0 conn(0x7f1c7013cbf0 0x7f1c7013d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 -- 172.21.15.154:0/1143890366 >> 172.21.15.154:0/1143890366 conn(0x7f1c700c3980 msgr2=0x7f1c700c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 -- 172.21.15.154:0/1143890366 shutdown_connections 2024-03-14T09:04:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.295+0000 7f1c7712d700 1 -- 172.21.15.154:0/1143890366 wait complete. 2024-03-14T09:04:46.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.296+0000 7f1c7712d700 1 Processor -- start 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.296+0000 7f1c7712d700 1 -- start start 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1c7006cfe0 0x7f1c700c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 0x7f1c701e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70151c20 con 0x7f1c70157e40 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c70153a30 con 0x7f1c7013cbf0 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c7712d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c701ea7b0 con 0x7f1c7006cfe0 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32896/0 (socket says 172.21.15.154:32896) 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 -- 172.21.15.154:0/3820798336 learned_addr learned my addr 172.21.15.154:0/3820798336 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c74ec9700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1c7006cfe0 0x7f1c700c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c6ffff700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 0x7f1c701e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 -- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1c7006cfe0 msgr2=0x7f1c700c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1c7006cfe0 0x7f1c700c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 -- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 msgr2=0x7f1c701e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.297+0000 7f1c756ca700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 0x7f1c701e9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c756ca700 1 -- 172.21.15.154:0/3820798336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009b30 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c6ffff700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 0x7f1c701e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c756ca700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006c860 tx=0x7f1c68004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004440 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680045c0 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c70153590 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004970 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.298+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c701532b0 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.299+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c700697d0 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.299+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1c68009400 con 0x7f1c70157e40 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.300+0000 7f1c6d7fa700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 0x7f1c5406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.300+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5885+0+0 (secure 0 0 0) 0x7f1c68089fc0 con 0x7f1c70157e40 2024-03-14T09:04:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.300+0000 7f1c74ec9700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 0x7f1c5406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.300+0000 7f1c74ec9700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 0x7f1c5406ccf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1c70151d80 tx=0x7f1c64007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.304+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f1c680bd4f0 con 0x7f1c70157e40 2024-03-14T09:04:46.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:46.499+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c7006dcf0 con 0x7f1c70157e40 2024-03-14T09:04:47.510 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.509+0000 7f1c6d7fa700 1 -- 172.21.15.154:0/3820798336 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v79) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1c68058e50 con 0x7f1c70157e40 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 msgr2=0x7f1c5406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 0x7f1c5406ccf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1c70151d80 tx=0x7f1c64007ec0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 msgr2=0x7f1c701ea270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1c7006c860 tx=0x7f1c68004ad0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 shutdown_connections 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f1c5406a830 0x7f1c5406ccf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1c70157e40 0x7f1c701ea270 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1c7006cfe0 0x7f1c700c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 --2- 172.21.15.154:0/3820798336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c7013cbf0 0x7f1c701e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.514+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 >> 172.21.15.154:0/3820798336 conn(0x7f1c700c3980 msgr2=0x7f1c700c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:47.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.515+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 shutdown_connections 2024-03-14T09:04:47.516 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.515+0000 7f1c7712d700 1 -- 172.21.15.154:0/3820798336 wait complete. 2024-03-14T09:04:47.516 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_metadata' removed 2024-03-14T09:04:47.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-14T09:04:47.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.662+0000 7f7303fbc700 1 Processor -- start 2024-03-14T09:04:47.662 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.662+0000 7f7303fbc700 1 -- start start 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 0x7f72fc14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72fc154a40 0x7f72fc154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72fc06ef60 con 0x7f72fc154a40 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72fc155a10 con 0x7f72fc0a84d0 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7303fbc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc06f140 con 0x7f72fc0c18f0 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7301557700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72fc154a40 0x7f72fc154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7301557700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f72fc154a40 0x7f72fc154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32910/0 (socket says 172.21.15.154:32910) 2024-03-14T09:04:47.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.663+0000 7f7301557700 1 -- 172.21.15.154:0/3893840176 learned_addr learned my addr 172.21.15.154:0/3893840176 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 -- 172.21.15.154:0/3893840176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 msgr2=0x7f72fc0c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7302559700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301d58700 1 --2- 172.21.15.154:0/3893840176 >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 0x7f72fc14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc0c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 -- 172.21.15.154:0/3893840176 >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 msgr2=0x7f72fc14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 --2- 172.21.15.154:0/3893840176 >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 0x7f72fc14ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 -- 172.21.15.154:0/3893840176 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc06f620 con 0x7f72fc154a40 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301d58700 1 --2- 172.21.15.154:0/3893840176 >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 0x7f72fc14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7301557700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc154a40 0x7f72fc154ef0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c4890 tx=0x7f72f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530f064a32910b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7300d56700 1 -- 172.21.15.154:0/3893840176 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800b860 con 0x7f72fc154a40 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7300d56700 1 -- 172.21.15.154:0/3893840176 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f800b9c0 con 0x7f72fc154a40 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.664+0000 7f7300d56700 1 -- 172.21.15.154:0/3893840176 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800bd50 con 0x7f72fc154a40 2024-03-14T09:04:47.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 -- 172.21.15.154:0/3893840176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc154a40 msgr2=0x7f72fc154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc154a40 0x7f72fc154ef0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c4890 tx=0x7f72f8009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 -- 172.21.15.154:0/3893840176 shutdown_connections 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 --2- 172.21.15.154:0/3893840176 >> v2:172.21.15.191:3300/0 conn(0x7f72fc0a84d0 0x7f72fc14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 --2- 172.21.15.154:0/3893840176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc154a40 0x7f72fc154ef0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 -- 172.21.15.154:0/3893840176 >> 172.21.15.154:0/3893840176 conn(0x7f72fc0c39f0 msgr2=0x7f72fc0c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 -- 172.21.15.154:0/3893840176 shutdown_connections 2024-03-14T09:04:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.665+0000 7f7303fbc700 1 -- 172.21.15.154:0/3893840176 wait complete. 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.666+0000 7f7303fbc700 1 Processor -- start 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.666+0000 7f7303fbc700 1 -- start start 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc1424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc154a40 0x7f72fc1ed880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc1eab60 con 0x7f72fc0a84d0 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc1ed110 con 0x7f72fc154a40 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc150520 con 0x7f72fc0c18f0 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32914/0 (socket says 172.21.15.154:32914) 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 -- 172.21.15.154:0/395073382 learned_addr learned my addr 172.21.15.154:0/395073382 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301557700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc1424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7302559700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc154a40 0x7f72fc1ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 -- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 msgr2=0x7f72fc1424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc1424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 -- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc154a40 msgr2=0x7f72fc1ed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc154a40 0x7f72fc1ed880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 -- 172.21.15.154:0/395073382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8009550 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7301d58700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00c7f0 tx=0x7f72ec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec011800 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec011960 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec011d10 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc150800 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.667+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc1ed360 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.668+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc0697d0 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.669+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f72ec018360 con 0x7f72fc0a84d0 2024-03-14T09:04:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.669+0000 7f72f27fc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 0x7f72e006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:47.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.669+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5445+0+0 (secure 0 0 0) 0x7f72ec088b40 con 0x7f72fc0a84d0 2024-03-14T09:04:47.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.669+0000 7f7301557700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 0x7f72e006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:47.672 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.670+0000 7f7301557700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 0x7f72e006ccf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000c00 tx=0x7f72f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:47.673 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.674+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f72ec0540f0 con 0x7f72fc0a84d0 2024-03-14T09:04:47.868 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:47.868+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72fc06d390 con 0x7f72fc0a84d0 2024-03-14T09:04:48.514 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.514+0000 7f72f27fc700 1 -- 172.21.15.154:0/395073382 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v80) v1 ==== 172+0+0 (secure 0 0 0) 0x7f72ec01f0d0 con 0x7f72fc0a84d0 2024-03-14T09:04:48.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.517+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 msgr2=0x7f72e006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.517+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 0x7f72e006ccf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000c00 tx=0x7f72f8008030 comp rx=0 tx=0).stop 2024-03-14T09:04:48.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.517+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 msgr2=0x7f72fc1ea620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.517+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00c7f0 tx=0x7f72ec00cbb0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 shutdown_connections 2024-03-14T09:04:48.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72fc154a40 0x7f72fc1ed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f72e006a830 0x7f72e006ccf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72fc0a84d0 0x7f72fc1ea620 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 --2- 172.21.15.154:0/395073382 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72fc0c18f0 0x7f72fc1424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 >> 172.21.15.154:0/395073382 conn(0x7f72fc0c39f0 msgr2=0x7f72fc14a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:48.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.518+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 shutdown_connections 2024-03-14T09:04:48.519 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.519+0000 7f7303fbc700 1 -- 172.21.15.154:0/395073382 wait complete. 2024-03-14T09:04:48.521 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs2_data' removed 2024-03-14T09:04:48.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-14T09:04:48.659 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.659+0000 7fbaf65a4700 1 Processor -- start 2024-03-14T09:04:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 -- start start 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 0x7fbaf006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbaf006d600 0x7fbaf00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf00c5f10 con 0x7fbaf006d600 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf00c7d50 con 0x7fbaf006c6e0 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf65a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c8030 con 0x7fbaf0070a10 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf4b41700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbaf006d600 0x7fbaf00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf4b41700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbaf006d600 0x7fbaf00c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32926/0 (socket says 172.21.15.154:32926) 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.660+0000 7fbaf4b41700 1 -- 172.21.15.154:0/1604394498 learned_addr learned my addr 172.21.15.154:0/1604394498 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 -- 172.21.15.154:0/1604394498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 msgr2=0x7fbaf00c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaeffff700 1 --2- 172.21.15.154:0/1604394498 >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 0x7fbaf006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaef7fe700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf00c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 -- 172.21.15.154:0/1604394498 >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 msgr2=0x7fbaf006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 --2- 172.21.15.154:0/1604394498 >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 0x7fbaf006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 -- 172.21.15.154:0/1604394498 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00c8310 con 0x7fbaf006d600 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaef7fe700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf00c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaeffff700 1 --2- 172.21.15.154:0/1604394498 >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 0x7fbaf006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf4b41700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf00c68e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0073f80 tx=0x7fbae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e62e8e7c63252 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaeeffd700 1 -- 172.21.15.154:0/1604394498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae800e070 con 0x7fbaf006d600 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaeeffd700 1 -- 172.21.15.154:0/1604394498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae800b9c0 con 0x7fbaf006d600 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaeeffd700 1 -- 172.21.15.154:0/1604394498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae800bd70 con 0x7fbaf006d600 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf65a4700 1 -- 172.21.15.154:0/1604394498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 msgr2=0x7fbaf00c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf00c68e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0073f80 tx=0x7fbae800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.661+0000 7fbaf65a4700 1 -- 172.21.15.154:0/1604394498 shutdown_connections 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf00c68e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/1604394498 >> v2:172.21.15.191:3300/0 conn(0x7fbaf006c6e0 0x7fbaf006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/1604394498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf00c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 -- 172.21.15.154:0/1604394498 >> 172.21.15.154:0/1604394498 conn(0x7fbaf00a9bf0 msgr2=0x7fbaf00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 -- 172.21.15.154:0/1604394498 shutdown_connections 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.662+0000 7fbaf65a4700 1 -- 172.21.15.154:0/1604394498 wait complete. 2024-03-14T09:04:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.663+0000 7fbaf65a4700 1 Processor -- start 2024-03-14T09:04:48.666 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.663+0000 7fbaf65a4700 1 -- start start 2024-03-14T09:04:48.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf006c6e0 0x7fbaf01ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.668 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf01dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf0099fd0 con 0x7fbaf006d600 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf01ede90 con 0x7fbaf006c6e0 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf65a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf009bce0 con 0x7fbaf0070a10 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32930/0 (socket says 172.21.15.154:32930) 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 -- 172.21.15.154:0/167542282 learned_addr learned my addr 172.21.15.154:0/167542282 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaeffff700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf006c6e0 0x7fbaf01ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaf4b41700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf01dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 -- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 msgr2=0x7fbaf01dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf01dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 -- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf006c6e0 msgr2=0x7fbaf01ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.664+0000 7fbaef7fe700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf006c6e0 0x7fbaf01ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.669 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaef7fe700 1 -- 172.21.15.154:0/167542282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae8009b30 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaf4b41700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf01dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaef7fe700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c7f0 tx=0x7fbae000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011800 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0011960 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011cf0 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf009a240 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.665+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf009a800 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.666+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf00697d0 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.666+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fbae0018360 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.667+0000 7fbaecff9700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 0x7fbadc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.667+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbae00889b0 con 0x7fbaf006d600 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.667+0000 7fbaeffff700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 0x7fbadc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:48.670 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.667+0000 7fbaeffff700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 0x7fbadc06ccf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:48.671 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.671+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fbae0054080 con 0x7fbaf006d600 2024-03-14T09:04:48.874 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.874+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbaf00c3190 con 0x7fbaf006d600 2024-03-14T09:04:48.877 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.878+0000 7fbaecff9700 1 -- 172.21.15.154:0/167542282 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbae00576a0 con 0x7fbaf006d600 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 msgr2=0x7fbadc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 0x7fbadc06ccf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 msgr2=0x7fbaf009b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c7f0 tx=0x7fbae000cb00 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 shutdown_connections 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbaf0070a10 0x7fbaf01dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf006c6e0 0x7fbaf01ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fbadc06a830 0x7fbadc06ccf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 --2- 172.21.15.154:0/167542282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaf006d600 0x7fbaf009b7a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:48.880 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 >> 172.21.15.154:0/167542282 conn(0x7fbaf00a9bf0 msgr2=0x7fbaf00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:48.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 shutdown_connections 2024-03-14T09:04:48.881 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:48.880+0000 7fbaf65a4700 1 -- 172.21.15.154:0/167542282 wait complete. 2024-03-14T09:04:48.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap' 2024-03-14T09:04:49.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.013+0000 7efe9fe91700 1 Processor -- start 2024-03-14T09:04:49.013 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.013+0000 7efe9fe91700 1 -- start start 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9fe91700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe98155df0 0x7efe98156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9fe91700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe980ac8e0 con 0x7efe9815d270 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9fe91700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9806de30 con 0x7efe9814eaf0 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9fe91700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9806b310 con 0x7efe98155df0 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9d42c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe98155df0 0x7efe98156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9e42e700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efe9815d270 0x7efe9815f660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44996/0 (socket says 172.21.15.154:44996) 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9e42e700 1 -- 172.21.15.154:0/2991428868 learned_addr learned my addr 172.21.15.154:0/2991428868 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9d42c700 1 -- 172.21.15.154:0/2991428868 >> v1:172.21.15.191:6789/0 conn(0x7efe9814eaf0 legacy=0x7efe9806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9d42c700 1 -- 172.21.15.154:0/2991428868 >> v1:172.21.15.154:6789/0 conn(0x7efe9815d270 legacy=0x7efe9815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.015 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.014+0000 7efe9d42c700 1 -- 172.21.15.154:0/2991428868 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980acb30 con 0x7efe98155df0 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9d42c700 1 --2- 172.21.15.154:0/2991428868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98155df0 0x7efe98156250 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efe94009840 tx=0x7efe94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d70f4da9026e46 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9cc2b700 1 -- 172.21.15.154:0/2991428868 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94011830 con 0x7efe98155df0 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9cc2b700 1 -- 172.21.15.154:0/2991428868 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe94011990 con 0x7efe98155df0 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9cc2b700 1 -- 172.21.15.154:0/2991428868 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94011d20 con 0x7efe98155df0 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 -- 172.21.15.154:0/2991428868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98155df0 msgr2=0x7efe98156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 --2- 172.21.15.154:0/2991428868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98155df0 0x7efe98156250 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efe94009840 tx=0x7efe94009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 -- 172.21.15.154:0/2991428868 shutdown_connections 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 --2- 172.21.15.154:0/2991428868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98155df0 0x7efe98156250 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 -- 172.21.15.154:0/2991428868 >> 172.21.15.154:0/2991428868 conn(0x7efe980c3880 msgr2=0x7efe980c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 -- 172.21.15.154:0/2991428868 shutdown_connections 2024-03-14T09:04:49.016 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.015+0000 7efe9fe91700 1 -- 172.21.15.154:0/2991428868 wait complete. 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.016+0000 7efe9fe91700 1 Processor -- start 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.016+0000 7efe9fe91700 1 -- start start 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 0x7efe9814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 0x7efe98151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9815ae20 con 0x7efe9814eaf0 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe98144fb0 con 0x7efe98155df0 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9fe91700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe98148d90 con 0x7efe9815d270 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9dc2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9dc2d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32944/0 (socket says 172.21.15.154:32944) 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9dc2d700 1 -- 172.21.15.154:0/4140056553 learned_addr learned my addr 172.21.15.154:0/4140056553 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.017+0000 7efe9d42c700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 0x7efe9814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9e42e700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 0x7efe98151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 -- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 msgr2=0x7efe98151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 0x7efe98151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 -- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 msgr2=0x7efe9814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 0x7efe9814a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 -- 172.21.15.154:0/4140056553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940094a0 con 0x7efe9814eaf0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9d42c700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 0x7efe9814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9e42e700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 0x7efe98151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9dc2d700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c450 tx=0x7efe8800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880052d0 con 0x7efe9814eaf0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe88005430 con 0x7efe9814eaf0 2024-03-14T09:04:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9814b3a0 con 0x7efe9814eaf0 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880038c0 con 0x7efe9814eaf0 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.018+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe981eb6a0 con 0x7efe9814eaf0 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.019+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe980697d0 con 0x7efe9814eaf0 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.019+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7efe88003a20 con 0x7efe9814eaf0 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.020+0000 7efe8e7fc700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 0x7efe7c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.020+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efe88087de0 con 0x7efe9814eaf0 2024-03-14T09:04:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.020+0000 7efe9d42c700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 0x7efe7c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.020+0000 7efe9d42c700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 0x7efe7c06ccf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efe980c46d0 tx=0x7efe94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:49.024 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.024+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7efe88053530 con 0x7efe9814eaf0 2024-03-14T09:04:49.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.298+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]} v 0) v1 -- 0x7efe981453a0 con 0x7efe9814eaf0 2024-03-14T09:04:49.515 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.515+0000 7efe8e7fc700 1 -- 172.21.15.154:0/4140056553 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multifs_auth.TestMDSCaps.test_r_with_fsname_and_no_path_in_cap"]}]=0 v221) v1 ==== 159+0+0 (secure 0 0 0) 0x7efe88056b50 con 0x7efe9814eaf0 2024-03-14T09:04:49.517 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 msgr2=0x7efe7c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 0x7efe7c06ccf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efe980c46d0 tx=0x7efe94008030 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 msgr2=0x7efe98144a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c450 tx=0x7efe8800c810 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 shutdown_connections 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9815d270 0x7efe98151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efe7c06a830 0x7efe7c06ccf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe9814eaf0 0x7efe98144a70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 --2- 172.21.15.154:0/4140056553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe98155df0 0x7efe9814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 >> 172.21.15.154:0/4140056553 conn(0x7efe980c3880 msgr2=0x7efe98152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 shutdown_connections 2024-03-14T09:04:49.518 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:49.517+0000 7efe9fe91700 1 -- 172.21.15.154:0/4140056553 wait complete. 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner:test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) ... ERROR 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner: 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner:ERROR: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-14T09:04:49.531 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/test_multifs_auth.py", line 90, in test_r_with_fsname_and_no_path_in_cap 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: self.captester.run_mds_cap_tests(PERM) 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/caps_helper.py", line 154, in run_mds_cap_tests 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: self.conduct_pos_test_for_read_caps() 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/caps_helper.py", line 167, in conduct_pos_test_for_read_caps 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: contents = mount.read_file(self.path, sudo_read) 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner:AttributeError: 'CapTester' object has no attribute 'path' 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner:Ran 3 tests in 185.905s 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner: 2024-03-14T09:04:49.532 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner:ERROR: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/test_multifs_auth.py", line 90, in test_r_with_fsname_and_no_path_in_cap 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: self.captester.run_mds_cap_tests(PERM) 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/caps_helper.py", line 154, in run_mds_cap_tests 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: self.conduct_pos_test_for_read_caps() 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs/caps_helper.py", line 167, in conduct_pos_test_for_read_caps 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: contents = mount.read_file(self.path, sudo_read) 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner:AttributeError: 'CapTester' object has no attribute 'path' 2024-03-14T09:04:49.533 INFO:tasks.cephfs_test_runner: 2024-03-14T09:04:49.534 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-03-14T09:04:49.738 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1ad53b05e33f43dfb46b30a5bff6c3b1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_2f465319e3c83d48f999325459bcdfd0346449c0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps) 2024-03-14T09:04:49.740 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-14T09:04:49.749 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-14T09:04:49.758 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-14T09:04:49.758 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-03-14T09:04:49.758 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:49.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244184k,nr_inodes=4061046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19342 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252520k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 2024-03-14T09:04:49.780 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-14T09:04:49.781 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-14T09:04:49.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16244192k,nr_inodes=4061048,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15656 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3252524k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 2024-03-14T09:04:49.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:49.802 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2024-03-14T09:04:49.838 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-14T09:04:49.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:49.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-14T09:04:49.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:49.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2024-03-14T09:04:50.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:50.022 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-03-14T09:04:50.043 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-14T09:04:50.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:50.043 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-14T09:04:50.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-14T09:04:50.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-03-14T09:04:50.224 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-14T09:04:50.233 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-14T09:04:50.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:50.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.357+0000 7f918e21f700 1 Processor -- start 2024-03-14T09:04:50.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.357+0000 7f918e21f700 1 -- start start 2024-03-14T09:04:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f918e21f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f91880c4310 0x7f91880c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f918e21f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f91880c47a0 0x7f91880c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f918e21f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9188072a80 con 0x7f91880c3490 2024-03-14T09:04:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f918e21f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880a4f60 con 0x7f91880c4310 2024-03-14T09:04:50.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f918e21f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880a5180 con 0x7f91880c47a0 2024-03-14T09:04:50.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f91880c4310 0x7f91880c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f91877fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f91880c3490 0x7f91880c3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48102/0 (socket says 172.21.15.154:48102) 2024-03-14T09:04:50.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f91880c4310 0x7f91880c08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47810/0 (socket says 172.21.15.154:47810) 2024-03-14T09:04:50.361 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f91877fe700 1 -- 172.21.15.154:0/2582040094 learned_addr learned my addr 172.21.15.154:0/2582040094 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9187fff700 1 --2- 172.21.15.154:0/2582040094 >> v2:172.21.15.191:3300/0 conn(0x7f91880c47a0 0x7f91880c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 -- 172.21.15.154:0/2582040094 >> v2:172.21.15.191:3300/0 conn(0x7f91880c47a0 msgr2=0x7f91880c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 --2- 172.21.15.154:0/2582040094 >> v2:172.21.15.191:3300/0 conn(0x7f91880c47a0 0x7f91880c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 -- 172.21.15.154:0/2582040094 >> v1:172.21.15.154:6789/0 conn(0x7f91880c3490 legacy=0x7f91880c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 -- 172.21.15.154:0/2582040094 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188072cb0 con 0x7f91880c4310 2024-03-14T09:04:50.362 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f91867fc700 1 -- 172.21.15.154:0/2582040094 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580553424 0 0) 0x7f9188072a80 con 0x7f91880c3490 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.358+0000 7f9186ffd700 1 --2- 172.21.15.154:0/2582040094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c4310 0x7f91880c08f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9188072190 tx=0x7f9178009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593fe5ad7f01d08 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f91867fc700 1 -- 172.21.15.154:0/2582040094 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178011830 con 0x7f91880c4310 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f91867fc700 1 -- 172.21.15.154:0/2582040094 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9178011990 con 0x7f91880c4310 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f91867fc700 1 -- 172.21.15.154:0/2582040094 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178011d20 con 0x7f91880c4310 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 -- 172.21.15.154:0/2582040094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c4310 msgr2=0x7f91880c08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 --2- 172.21.15.154:0/2582040094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c4310 0x7f91880c08f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9188072190 tx=0x7f9178009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:50.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 -- 172.21.15.154:0/2582040094 shutdown_connections 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 --2- 172.21.15.154:0/2582040094 >> v2:172.21.15.191:3300/0 conn(0x7f91880c47a0 0x7f91880c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 --2- 172.21.15.154:0/2582040094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c4310 0x7f91880c08f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 -- 172.21.15.154:0/2582040094 >> 172.21.15.154:0/2582040094 conn(0x7f91880a7a10 msgr2=0x7f91880a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 -- 172.21.15.154:0/2582040094 shutdown_connections 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.359+0000 7f918e21f700 1 -- 172.21.15.154:0/2582040094 wait complete. 2024-03-14T09:04:50.364 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.360+0000 7f918e21f700 1 Processor -- start 2024-03-14T09:04:50.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 -- start start 2024-03-14T09:04:50.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 0x7f91881dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.367 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91880c4310 0x7f918814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881ea3e0 con 0x7f91880c47a0 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881ea6c0 con 0x7f91880c3490 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f918e21f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91881ea9a0 con 0x7f91880c4310 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f9187fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f9187fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47826/0 (socket says 172.21.15.154:47826) 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.361+0000 7f9187fff700 1 -- 172.21.15.154:0/3893997854 learned_addr learned my addr 172.21.15.154:0/3893997854 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:50.368 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f91877fe700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 0x7f91881dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9186ffd700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91880c4310 0x7f918814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 -- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91880c4310 msgr2=0x7f918814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91880c4310 0x7f918814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 -- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 msgr2=0x7f91881dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 0x7f91881dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 -- 172.21.15.154:0/3893997854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91780094a0 con 0x7f91880c47a0 2024-03-14T09:04:50.369 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f91877fe700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 0x7f91881dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f9187fff700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f918000efc0 tx=0x7f918000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180009ba0 con 0x7f91880c47a0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918000f040 con 0x7f91880c47a0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918806e720 con 0x7f91880c47a0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180014480 con 0x7f91880c47a0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.362+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91881ddef0 con 0x7f91880c47a0 2024-03-14T09:04:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.363+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91881eaec0 con 0x7f91880c47a0 2024-03-14T09:04:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.364+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f91800145e0 con 0x7f91880c47a0 2024-03-14T09:04:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.364+0000 7f917ffff700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 0x7f916806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.364+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9180089400 con 0x7f91880c47a0 2024-03-14T09:04:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.364+0000 7f91877fe700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 0x7f916806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.364+0000 7f91877fe700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 0x7f916806ccf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9170000f60 tx=0x7f9170007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.372 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.368+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f91881eaec0 con 0x7f91880c47a0 2024-03-14T09:04:50.559 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.559+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f918806e5a0 con 0x7f916806a830 2024-03-14T09:04:50.561 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.562+0000 7f917ffff700 1 -- 172.21.15.154:0/3893997854 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50605 (secure 0 0 0) 0x7f9170005bb0 con 0x7f916806a830 2024-03-14T09:04:50.562 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:50.564 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 msgr2=0x7f916806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 0x7f916806ccf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9170000f60 tx=0x7f9170007ca0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 msgr2=0x7f918814f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f918000efc0 tx=0x7f918000c3a0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 shutdown_connections 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f916806a830 0x7f916806ccf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91880c47a0 0x7f918814f820 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.565 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91880c3490 0x7f91881dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 --2- 172.21.15.154:0/3893997854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91880c4310 0x7f918814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 >> 172.21.15.154:0/3893997854 conn(0x7f91880a7a10 msgr2=0x7f91880a7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:50.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 shutdown_connections 2024-03-14T09:04:50.566 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.564+0000 7f918e21f700 1 -- 172.21.15.154:0/3893997854 wait complete. 2024-03-14T09:04:50.566 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:50.577 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-03-14T09:04:49.568455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224960,"kb_used_data":8800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749760,"statfs":{"total":773094113280,"available":772863754240,"internally_reserved":0,"allocated":9011200,"data_stored":4650176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12328,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001248"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-14T09:04:42.269076+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:42.269076+0000","last_peered":"2024-03-14T09:04:42.269076+0000","last_clean":"2024-03-14T09:04:42.269076+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:42.269076+0000","last_undegraded":"2024-03-14T09:04:42.269076+0000","last_fullsized":"2024-03-14T09:04:42.269076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343288,"statfs":{"total":96636764160,"available":96607526912,"internally_reserved":0,"allocated":1568768,"data_stored":1021078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.499},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621}]},{"osd":2,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.77800000000000002,"15min":0.88700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":6.5,"15min":6.5},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67000000000000004,"5min":0.878,"15min":0.878},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71999999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":431250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.372,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.41199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.87,"15min":0.87},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67600000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.66300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66600000000000004,"5min":0.748,"15min":0.748},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.87,"15min":0.87},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.43099999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.432,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46100000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.39100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26400000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73399999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343292,"statfs":{"total":96636764160,"available":96607531008,"internally_reserved":0,"allocated":1564672,"data_stored":1021548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.62,"15min":0.75},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.872,"5min":7.383,"15min":7.383},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.45400000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85799999999999998,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.68400000000000005,"15min":0.82999999999999996},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.68899999999999995,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.70799999999999996,"15min":0.85599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60499999999999998,"5min":7.944,"15min":7.944},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35199999999999998,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47999999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.67400000000000004,"15min":0.80300000000000005},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83299999999999996,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.85799999999999998,"15min":0.97399999999999998},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.363,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.88200000000000001,"15min":1.0149999999999999},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4810000000000001,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.624},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.628},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.34100000000000003},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.41399999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.255},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.41299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":1.111,"15min":1.111},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.47599999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51300000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.58899999999999997,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.44900000000000001,"5min":7.5,"15min":7.5},"last":0.435},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.63700000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54100000000000004,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83399999999999996,"5min":1.151,"15min":1.151},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79700000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":0.752},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86099999999999999,"5min":1.179,"15min":1.179},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.435,"15min":0.435},"max":{"1min":0.73299999999999998,"5min":1.165,"15min":1.165},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":442013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.316,"15min":0.31},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.379}]},{"osd":2,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.373,"15min":0.375},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.379,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.48099999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.624},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.753,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80500000000000005,"5min":1.087,"15min":1.087},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":442115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.755,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.2549999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.81200000000000006,"15min":0.92000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78800000000000003,"5min":7.702,"15min":7.702},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63}]},{"osd":7,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.372},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.629}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.434},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:50.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:50.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.701+0000 7f5d54d69700 1 Processor -- start 2024-03-14T09:04:50.701 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.701+0000 7f5d54d69700 1 -- start start 2024-03-14T09:04:50.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d54d69700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5d5015e280 0x7f5d50160670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d54d69700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500ac8e0 con 0x7f5d5014eaf0 2024-03-14T09:04:50.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d54d69700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5006de30 con 0x7f5d50155df0 2024-03-14T09:04:50.705 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d54d69700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5006b310 con 0x7f5d5015e280 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4e59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5d5014eaf0 0x7f5d5006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48104/0 (socket says 172.21.15.154:48104) 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4e59c700 1 -- 172.21.15.154:0/2112112699 learned_addr learned my addr 172.21.15.154:0/2112112699 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442322332 0 0) 0x7f5d500ac8e0 con 0x7f5d5014eaf0 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d380035b0 con 0x7f5d5014eaf0 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4ed9d700 1 --2- 172.21.15.154:0/2112112699 >> v2:172.21.15.191:3300/0 conn(0x7f5d5015e280 0x7f5d50160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246318130 0 0) 0x7f5d380035b0 con 0x7f5d5014eaf0 2024-03-14T09:04:50.706 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 >> v2:172.21.15.191:3300/0 conn(0x7f5d5015e280 msgr2=0x7f5d50160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 --2- 172.21.15.154:0/2112112699 >> v2:172.21.15.191:3300/0 conn(0x7f5d5015e280 0x7f5d50160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 >> v1:172.21.15.191:6789/0 conn(0x7f5d50155df0 legacy=0x7f5d50156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d50156960 con 0x7f5d5014eaf0 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f5d40001670 con 0x7f5d5014eaf0 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 >> v1:172.21.15.154:6789/0 conn(0x7f5d5014eaf0 legacy=0x7f5d5006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 0x7f5d38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d38004850 0x7f5d38004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.707 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38005210 0x7f5d3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5006b310 con 0x7f5d38003a00 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3800c110 con 0x7f5d38005210 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.702+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3800c2d0 con 0x7f5d38004850 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 0x7f5d38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4ed9d700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38005210 0x7f5d3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4dd9b700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d38004850 0x7f5d38004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.708 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 -- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d38004850 msgr2=0x7f5d38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d38004850 0x7f5d38004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 -- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38005210 msgr2=0x7f5d3800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38005210 0x7f5d3800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 -- 172.21.15.154:0/2112112699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380035b0 con 0x7f5d38003a00 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4ed9d700 1 -- 172.21.15.154:0/2112112699 reap_dead start 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4dd9b700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d38004850 0x7f5d38004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 0x7f5d38003de0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005170 tx=0x7f5d38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f216221b509092 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.709 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40005b60 con 0x7f5d38003a00 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40005cc0 con 0x7f5d38003a00 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.703+0000 7f5d4d59a700 1 -- 172.21.15.154:0/2112112699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40008310 con 0x7f5d38003a00 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 -- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 msgr2=0x7f5d38003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 0x7f5d38003de0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005170 tx=0x7f5d38005880 comp rx=0 tx=0).stop 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 -- 172.21.15.154:0/2112112699 shutdown_connections 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 --2- 172.21.15.154:0/2112112699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d38003a00 0x7f5d38003de0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.710 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 -- 172.21.15.154:0/2112112699 >> 172.21.15.154:0/2112112699 conn(0x7f5d500c3880 msgr2=0x7f5d500c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:50.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 -- 172.21.15.154:0/2112112699 shutdown_connections 2024-03-14T09:04:50.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.704+0000 7f5d54d69700 1 -- 172.21.15.154:0/2112112699 wait complete. 2024-03-14T09:04:50.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.705+0000 7f5d54d69700 1 Processor -- start 2024-03-14T09:04:50.711 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.705+0000 7f5d54d69700 1 -- start start 2024-03-14T09:04:50.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38003a00 0x7f5d5013db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d500c2970 0x7f5d501eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.714 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5013f590 con 0x7f5d501401b0 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5013ebc0 con 0x7f5d38003a00 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d54d69700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c3270 con 0x7f5d500c2970 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47856/0 (socket says 172.21.15.154:47856) 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 -- 172.21.15.154:0/1187101236 learned_addr learned my addr 172.21.15.154:0/1187101236 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38003a00 0x7f5d5013db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.715 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4ed9d700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d500c2970 0x7f5d501eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 -- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d500c2970 msgr2=0x7f5d501eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d500c2970 0x7f5d501eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 -- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38003a00 msgr2=0x7f5d5013db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38003a00 0x7f5d5013db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 -- 172.21.15.154:0/1187101236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500c3550 con 0x7f5d501401b0 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d4dd9b700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e5b0 tx=0x7f5d4400e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.706+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400eee0 con 0x7f5d501401b0 2024-03-14T09:04:50.716 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.707+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440052e0 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.707+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003680 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.707+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5013e570 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.707+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d501ebe00 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.707+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d501ebfc0 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.708+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5d440037e0 con 0x7f5d501401b0 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.708+0000 7f5d3effd700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 0x7f5d3006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:50.717 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.708+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5d44087f80 con 0x7f5d501401b0 2024-03-14T09:04:50.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.708+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 0x7f5d3006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:50.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.709+0000 7f5d4e59c700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 0x7f5d3006ccf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005170 tx=0x7f5d4000a070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:50.718 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.713+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5d440bb4f0 con 0x7f5d501401b0 2024-03-14T09:04:50.906 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.906+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d50155700 con 0x7f5d3006a830 2024-03-14T09:04:50.908 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.908+0000 7f5d3effd700 1 -- 172.21.15.154:0/1187101236 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50605 (secure 0 0 0) 0x7f5d40005e30 con 0x7f5d3006a830 2024-03-14T09:04:50.909 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.910+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 msgr2=0x7f5d3006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.910+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 0x7f5d3006ccf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d40005170 tx=0x7f5d4000a070 comp rx=0 tx=0).stop 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.910+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 msgr2=0x7f5d5013f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.910+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e5b0 tx=0x7f5d4400e8c0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 shutdown_connections 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d500c2970 0x7f5d501eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d38003a00 0x7f5d5013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5d3006a830 0x7f5d3006ccf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 --2- 172.21.15.154:0/1187101236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d501401b0 0x7f5d5013f050 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:50.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 >> 172.21.15.154:0/1187101236 conn(0x7f5d500c3880 msgr2=0x7f5d500715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:50.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 shutdown_connections 2024-03-14T09:04:50.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:50.911+0000 7f5d54d69700 1 -- 172.21.15.154:0/1187101236 wait complete. 2024-03-14T09:04:50.912 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:50.923 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-03-14T09:04:49.568455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224960,"kb_used_data":8800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749760,"statfs":{"total":773094113280,"available":772863754240,"internally_reserved":0,"allocated":9011200,"data_stored":4650176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12328,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001248"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-14T09:04:42.269076+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:42.269076+0000","last_peered":"2024-03-14T09:04:42.269076+0000","last_clean":"2024-03-14T09:04:42.269076+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:42.269076+0000","last_undegraded":"2024-03-14T09:04:42.269076+0000","last_fullsized":"2024-03-14T09:04:42.269076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343288,"statfs":{"total":96636764160,"available":96607526912,"internally_reserved":0,"allocated":1568768,"data_stored":1021078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.499},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621}]},{"osd":2,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.77800000000000002,"15min":0.88700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":6.5,"15min":6.5},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67000000000000004,"5min":0.878,"15min":0.878},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71999999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":431250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.372,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.41199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.87,"15min":0.87},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67600000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.66300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66600000000000004,"5min":0.748,"15min":0.748},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.87,"15min":0.87},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.43099999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.432,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46100000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.39100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26400000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73399999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343292,"statfs":{"total":96636764160,"available":96607531008,"internally_reserved":0,"allocated":1564672,"data_stored":1021548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.62,"15min":0.75},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.872,"5min":7.383,"15min":7.383},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.45400000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85799999999999998,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.68400000000000005,"15min":0.82999999999999996},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.68899999999999995,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.70799999999999996,"15min":0.85599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60499999999999998,"5min":7.944,"15min":7.944},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35199999999999998,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47999999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.67400000000000004,"15min":0.80300000000000005},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83299999999999996,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.85799999999999998,"15min":0.97399999999999998},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.363,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.88200000000000001,"15min":1.0149999999999999},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4810000000000001,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.624},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.628},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.34100000000000003},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.41399999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.255},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.41299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":1.111,"15min":1.111},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.47599999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51300000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.58899999999999997,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.44900000000000001,"5min":7.5,"15min":7.5},"last":0.435},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.63700000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54100000000000004,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83399999999999996,"5min":1.151,"15min":1.151},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79700000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":0.752},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86099999999999999,"5min":1.179,"15min":1.179},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.435,"15min":0.435},"max":{"1min":0.73299999999999998,"5min":1.165,"15min":1.165},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":442013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.316,"15min":0.31},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.379}]},{"osd":2,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.373,"15min":0.375},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.379,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.48099999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.624},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.753,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80500000000000005,"5min":1.087,"15min":1.087},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":442115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.755,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.2549999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.81200000000000006,"15min":0.92000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78800000000000003,"5min":7.702,"15min":7.702},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63}]},{"osd":7,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.372},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.629}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.434},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:50.924 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-14T09:04:50.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:51.045 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.045+0000 7f998b719700 1 Processor -- start 2024-03-14T09:04:51.046 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 -- start start 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f99840c4380 0x7f998414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99840c5d60 0x7f99840c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840acd10 con 0x7f99840702f0 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9984150110 con 0x7f99840c5d60 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.046+0000 7f998b719700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99840c66b0 con 0x7f99840c4380 2024-03-14T09:04:51.048 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99840c5d60 0x7f99840c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99840c5d60 0x7f99840c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47858/0 (socket says 172.21.15.154:47858) 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 -- 172.21.15.154:0/193753607 learned_addr learned my addr 172.21.15.154:0/193753607 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9988cb4700 1 --2- 172.21.15.154:0/193753607 >> v2:172.21.15.191:3300/0 conn(0x7f99840c4380 0x7f998414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 -- 172.21.15.154:0/193753607 >> v2:172.21.15.191:3300/0 conn(0x7f99840c4380 msgr2=0x7f998414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 --2- 172.21.15.154:0/193753607 >> v2:172.21.15.191:3300/0 conn(0x7f99840c4380 0x7f998414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 -- 172.21.15.154:0/193753607 >> v1:172.21.15.191:6789/0 conn(0x7f99840702f0 legacy=0x7f99840c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 -- 172.21.15.154:0/193753607 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840ac9c0 con 0x7f99840c5d60 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9989cb6700 1 --2- 172.21.15.154:0/193753607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c5d60 0x7f99840c6170 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f99840c5980 tx=0x7f997c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953eed5857a749b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.049 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9977fff700 1 -- 172.21.15.154:0/193753607 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00e070 con 0x7f99840c5d60 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9977fff700 1 -- 172.21.15.154:0/193753607 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c00b910 con 0x7f99840c5d60 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.047+0000 7f9977fff700 1 -- 172.21.15.154:0/193753607 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c00bcc0 con 0x7f99840c5d60 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 -- 172.21.15.154:0/193753607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c5d60 msgr2=0x7f99840c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 --2- 172.21.15.154:0/193753607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c5d60 0x7f99840c6170 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f99840c5980 tx=0x7f997c00b330 comp rx=0 tx=0).stop 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 -- 172.21.15.154:0/193753607 shutdown_connections 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 --2- 172.21.15.154:0/193753607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c5d60 0x7f99840c6170 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 --2- 172.21.15.154:0/193753607 >> v2:172.21.15.191:3300/0 conn(0x7f99840c4380 0x7f998414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.050 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 -- 172.21.15.154:0/193753607 >> 172.21.15.154:0/193753607 conn(0x7f99840a9bf0 msgr2=0x7f99840a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:51.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 -- 172.21.15.154:0/193753607 shutdown_connections 2024-03-14T09:04:51.051 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.048+0000 7f998b719700 1 -- 172.21.15.154:0/193753607 wait complete. 2024-03-14T09:04:51.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.049+0000 7f998b719700 1 Processor -- start 2024-03-14T09:04:51.053 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.049+0000 7f998b719700 1 -- start start 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99840702f0 0x7f9984146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99840c5d60 0x7f99841482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998413f5c0 con 0x7f99840c4380 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998413f3f0 con 0x7f99840c5d60 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998413f110 con 0x7f99840702f0 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.054 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47872/0 (socket says 172.21.15.154:47872) 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 -- 172.21.15.154:0/236336843 learned_addr learned my addr 172.21.15.154:0/236336843 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f99894b5700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99840702f0 0x7f9984146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9989cb6700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99840c5d60 0x7f99841482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 -- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99840702f0 msgr2=0x7f9984146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99840702f0 0x7f9984146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 -- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99840c5d60 msgr2=0x7f99841482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99840c5d60 0x7f99841482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.055 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 -- 172.21.15.154:0/236336843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009b30 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9988cb4700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f998000e9e0 tx=0x7f998000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980009710 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000ee80 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99841ec170 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.050+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000ca50 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.051+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998413f810 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.051+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99841ec4e0 con 0x7f99840c4380 2024-03-14T09:04:51.056 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.052+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f998000cbb0 con 0x7f99840c4380 2024-03-14T09:04:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.052+0000 7f9975ffb700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 0x7f996806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.052+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9980088a70 con 0x7f99840c4380 2024-03-14T09:04:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.053+0000 7f99894b5700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 0x7f996806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.053+0000 7f99894b5700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 0x7f996806ccf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9978007fd0 tx=0x7f9978009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.057+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f9980054120 con 0x7f99840c4380 2024-03-14T09:04:51.248 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.248+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9984070ad0 con 0x7f996806a830 2024-03-14T09:04:51.250 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.251+0000 7f9975ffb700 1 -- 172.21.15.154:0/236336843 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50605 (secure 0 0 0) 0x7f9978001560 con 0x7f996806a830 2024-03-14T09:04:51.251 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:51.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 msgr2=0x7f996806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.253 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 0x7f996806ccf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9978007fd0 tx=0x7f9978009040 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 msgr2=0x7f99841eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f998000e9e0 tx=0x7f998000ecf0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 shutdown_connections 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99840c5d60 0x7f99841482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99840702f0 0x7f9984146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f996806a830 0x7f996806ccf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 --2- 172.21.15.154:0/236336843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99840c4380 0x7f99841eb930 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 >> 172.21.15.154:0/236336843 conn(0x7f99840a9bf0 msgr2=0x7f99840c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:51.254 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 shutdown_connections 2024-03-14T09:04:51.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.253+0000 7f998b719700 1 -- 172.21.15.154:0/236336843 wait complete. 2024-03-14T09:04:51.255 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:51.265 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-03-14T09:04:49.568455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224960,"kb_used_data":8800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749760,"statfs":{"total":773094113280,"available":772863754240,"internally_reserved":0,"allocated":9011200,"data_stored":4650176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12328,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001248"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-14T09:04:42.269076+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:42.269076+0000","last_peered":"2024-03-14T09:04:42.269076+0000","last_clean":"2024-03-14T09:04:42.269076+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:42.269076+0000","last_undegraded":"2024-03-14T09:04:42.269076+0000","last_fullsized":"2024-03-14T09:04:42.269076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343288,"statfs":{"total":96636764160,"available":96607526912,"internally_reserved":0,"allocated":1568768,"data_stored":1021078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.499},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621}]},{"osd":2,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.77800000000000002,"15min":0.88700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":6.5,"15min":6.5},"last":0.442},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67000000000000004,"5min":0.878,"15min":0.878},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71999999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":431250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.372,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.41199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.87,"15min":0.87},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67600000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.66300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66600000000000004,"5min":0.748,"15min":0.748},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.87,"15min":0.87},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.43099999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.432,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46100000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.39100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.26400000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.626},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73399999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.62},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343292,"statfs":{"total":96636764160,"available":96607531008,"internally_reserved":0,"allocated":1564672,"data_stored":1021548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.62,"15min":0.75},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.872,"5min":7.383,"15min":7.383},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.45400000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85799999999999998,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.68400000000000005,"15min":0.82999999999999996},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.68899999999999995,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.70799999999999996,"15min":0.85599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60499999999999998,"5min":7.944,"15min":7.944},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35199999999999998,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47999999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.67400000000000004,"15min":0.80300000000000005},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83299999999999996,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.85799999999999998,"15min":0.97399999999999998},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.363,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.88200000000000001,"15min":1.0149999999999999},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4810000000000001,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.624},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.628},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.34100000000000003},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.41399999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.255},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.41299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":1.111,"15min":1.111},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.47599999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51300000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.58899999999999997,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.44900000000000001,"5min":7.5,"15min":7.5},"last":0.435},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.63700000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54100000000000004,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83399999999999996,"5min":1.151,"15min":1.151},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79700000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":0.752},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86099999999999999,"5min":1.179,"15min":1.179},"last":0.50900000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.435,"15min":0.435},"max":{"1min":0.73299999999999998,"5min":1.165,"15min":1.165},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":442013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.316,"15min":0.31},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.379}]},{"osd":2,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.373,"15min":0.375},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.379,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.48099999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.624},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.753,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80500000000000005,"5min":1.087,"15min":1.087},"last":0.61399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":442115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.755,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.2549999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.81200000000000006,"15min":0.92000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78800000000000003,"5min":7.702,"15min":7.702},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63}]},{"osd":7,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":430668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.372},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.629}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.434},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:51.266 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-14T09:04:51.389 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.389+0000 7f5000f60700 1 Processor -- start 2024-03-14T09:04:51.390 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.389+0000 7f5000f60700 1 -- start start 2024-03-14T09:04:51.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f5000f60700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ffc155df0 0x7f4ffc156250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f5000f60700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4ffc15d270 0x7f4ffc15f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f5000f60700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0ac8e0 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.392 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f5000f60700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc06b310 con 0x7f4ffc155df0 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f5000f60700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc06de30 con 0x7f4ffc15d270 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f4ff9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ffc155df0 0x7f4ffc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f4ffa59c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4ffc14eaf0 0x7f4ffc06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48118/0 (socket says 172.21.15.154:48118) 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.390+0000 7f4ffa59c700 1 -- 172.21.15.154:0/2850358934 learned_addr learned my addr 172.21.15.154:0/2850358934 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ffad9d700 1 --2- 172.21.15.154:0/2850358934 >> v2:172.21.15.191:3300/0 conn(0x7f4ffc15d270 0x7f4ffc15f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff9d9b700 1 -- 172.21.15.154:0/2850358934 >> v2:172.21.15.191:3300/0 conn(0x7f4ffc15d270 msgr2=0x7f4ffc15f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/2850358934 >> v2:172.21.15.191:3300/0 conn(0x7f4ffc15d270 0x7f4ffc15f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.393 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff9d9b700 1 -- 172.21.15.154:0/2850358934 >> v1:172.21.15.154:6789/0 conn(0x7f4ffc14eaf0 legacy=0x7f4ffc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff9d9b700 1 -- 172.21.15.154:0/2850358934 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0acb30 con 0x7f4ffc155df0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff959a700 1 -- 172.21.15.154:0/2850358934 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432399922 0 0) 0x7f4ffc0ac8e0 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/2850358934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc155df0 0x7f4ffc156250 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0c4920 tx=0x7f4fec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bce35f78f4ea57 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff959a700 1 -- 172.21.15.154:0/2850358934 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec011830 con 0x7f4ffc155df0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff959a700 1 -- 172.21.15.154:0/2850358934 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec011990 con 0x7f4ffc155df0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.391+0000 7f4ff959a700 1 -- 172.21.15.154:0/2850358934 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec011d20 con 0x7f4ffc155df0 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 -- 172.21.15.154:0/2850358934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc155df0 msgr2=0x7f4ffc156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.394 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 --2- 172.21.15.154:0/2850358934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc155df0 0x7f4ffc156250 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0c4920 tx=0x7f4fec009b20 comp rx=0 tx=0).stop 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 -- 172.21.15.154:0/2850358934 shutdown_connections 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 --2- 172.21.15.154:0/2850358934 >> v2:172.21.15.191:3300/0 conn(0x7f4ffc15d270 0x7f4ffc15f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 --2- 172.21.15.154:0/2850358934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc155df0 0x7f4ffc156250 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 -- 172.21.15.154:0/2850358934 >> 172.21.15.154:0/2850358934 conn(0x7f4ffc0c3880 msgr2=0x7f4ffc0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 -- 172.21.15.154:0/2850358934 shutdown_connections 2024-03-14T09:04:51.395 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.392+0000 7f5000f60700 1 -- 172.21.15.154:0/2850358934 wait complete. 2024-03-14T09:04:51.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.393+0000 7f5000f60700 1 Processor -- start 2024-03-14T09:04:51.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.393+0000 7f5000f60700 1 -- start start 2024-03-14T09:04:51.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 0x7f4ffc071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.398 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 0x7f4ffc1ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1411f0 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc13cfa0 con 0x7f4ffc155df0 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f5000f60700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc072200 con 0x7f4ffc15d270 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47894/0 (socket says 172.21.15.154:47894) 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 -- 172.21.15.154:0/1402555140 learned_addr learned my addr 172.21.15.154:0/1402555140 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 0x7f4ffc071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffad9d700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 0x7f4ffc1ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.399 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 -- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 msgr2=0x7f4ffc1ed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 0x7f4ffc1ed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 -- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 msgr2=0x7f4ffc071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 0x7f4ffc071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 -- 172.21.15.154:0/1402555140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0094a0 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 0x7f4ffc071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffad9d700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 0x7f4ffc1ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4ffa59c700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00011e0 tx=0x7f4ff000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0004c80 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.400 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.394+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff0004e00 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.395+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc140950 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.395+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0007310 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.395+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc140f10 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.395+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc0697d0 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.396+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4ff0007470 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.396+0000 7f4feaffd700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 0x7f4fdc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.396+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4ff0088850 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.396+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 0x7f4fdc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:51.401 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.397+0000 7f4ff9d9b700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 0x7f4fdc06ccf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0c3850 tx=0x7f4fec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:51.402 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.401+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f4ff0053f10 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.601+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ffc06b470 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.601 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.602+0000 7f4feaffd700 1 -- 172.21.15.154:0/1402555140 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12701 (secure 0 0 0) 0x7f4ff0057530 con 0x7f4ffc14eaf0 2024-03-14T09:04:51.602 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:51.602 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":80,"fsid":"661f6023-01e4-4d6e-b4b3-5b5701ce7cc5","created":"2024-03-14T09:01:01.546487+0000","modified":"2024-03-14T09:04:48.508931+0000","last_up_change":"2024-03-14T09:01:12.513554+0000","last_in_change":"2024-03-14T09:01:04.115453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-14T09:01:13.575425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64076ce9-7a03-49dd-b455-6dd27cedc7cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":47282},{"type":"v1","addr":"172.21.15.154:6817","nonce":47282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":47282},{"type":"v1","addr":"172.21.15.154:6819","nonce":47282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":47282},{"type":"v1","addr":"172.21.15.154:6823","nonce":47282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":47282},{"type":"v1","addr":"172.21.15.154:6821","nonce":47282}]},"public_addr":"172.21.15.154:6817/47282","cluster_addr":"172.21.15.154:6819/47282","heartbeat_back_addr":"172.21.15.154:6823/47282","heartbeat_front_addr":"172.21.15.154:6821/47282","state":["exists","up"]},{"osd":1,"uuid":"310ba386-e95e-42f1-a244-e5ff86d37353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":47283},{"type":"v1","addr":"172.21.15.154:6825","nonce":47283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":47283},{"type":"v1","addr":"172.21.15.154:6827","nonce":47283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":47283},{"type":"v1","addr":"172.21.15.154:6831","nonce":47283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":47283},{"type":"v1","addr":"172.21.15.154:6829","nonce":47283}]},"public_addr":"172.21.15.154:6825/47283","cluster_addr":"172.21.15.154:6827/47283","heartbeat_back_addr":"172.21.15.154:6831/47283","heartbeat_front_addr":"172.21.15.154:6829/47283","state":["exists","up"]},{"osd":2,"uuid":"5c46843b-b6e8-43ac-8dcf-8c0f72917016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":47280},{"type":"v1","addr":"172.21.15.154:6804","nonce":47280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":47280},{"type":"v1","addr":"172.21.15.154:6808","nonce":47280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":47280},{"type":"v1","addr":"172.21.15.154:6815","nonce":47280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":47280},{"type":"v1","addr":"172.21.15.154:6812","nonce":47280}]},"public_addr":"172.21.15.154:6804/47280","cluster_addr":"172.21.15.154:6808/47280","heartbeat_back_addr":"172.21.15.154:6815/47280","heartbeat_front_addr":"172.21.15.154:6812/47280","state":["exists","up"]},{"osd":3,"uuid":"d5110594-4610-435f-a8b3-cd63e4717693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":47281},{"type":"v1","addr":"172.21.15.154:6801","nonce":47281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":47281},{"type":"v1","addr":"172.21.15.154:6805","nonce":47281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":47281},{"type":"v1","addr":"172.21.15.154:6813","nonce":47281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":47281},{"type":"v1","addr":"172.21.15.154:6809","nonce":47281}]},"public_addr":"172.21.15.154:6801/47281","cluster_addr":"172.21.15.154:6805/47281","heartbeat_back_addr":"172.21.15.154:6813/47281","heartbeat_front_addr":"172.21.15.154:6809/47281","state":["exists","up"]},{"osd":4,"uuid":"7cdbd767-c933-41d9-87ed-746a04c6a187","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":47548},{"type":"v1","addr":"172.21.15.191:6809","nonce":47548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":47548},{"type":"v1","addr":"172.21.15.191:6811","nonce":47548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":47548},{"type":"v1","addr":"172.21.15.191:6815","nonce":47548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":47548},{"type":"v1","addr":"172.21.15.191:6813","nonce":47548}]},"public_addr":"172.21.15.191:6809/47548","cluster_addr":"172.21.15.191:6811/47548","heartbeat_back_addr":"172.21.15.191:6815/47548","heartbeat_front_addr":"172.21.15.191:6813/47548","state":["exists","up"]},{"osd":5,"uuid":"f85d1595-d283-4331-bf87-cf7c03fdad6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":47551},{"type":"v1","addr":"172.21.15.191:6818","nonce":47551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":47551},{"type":"v1","addr":"172.21.15.191:6822","nonce":47551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":47551},{"type":"v1","addr":"172.21.15.191:6830","nonce":47551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":47551},{"type":"v1","addr":"172.21.15.191:6826","nonce":47551}]},"public_addr":"172.21.15.191:6818/47551","cluster_addr":"172.21.15.191:6822/47551","heartbeat_back_addr":"172.21.15.191:6830/47551","heartbeat_front_addr":"172.21.15.191:6826/47551","state":["exists","up"]},{"osd":6,"uuid":"da7d5636-8b31-4679-9d32-793b2cf2451a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6817","nonce":47571},{"type":"v1","addr":"172.21.15.191:6819","nonce":47571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6821","nonce":47571},{"type":"v1","addr":"172.21.15.191:6823","nonce":47571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6829","nonce":47571},{"type":"v1","addr":"172.21.15.191:6831","nonce":47571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6825","nonce":47571},{"type":"v1","addr":"172.21.15.191:6827","nonce":47571}]},"public_addr":"172.21.15.191:6819/47571","cluster_addr":"172.21.15.191:6823/47571","heartbeat_back_addr":"172.21.15.191:6831/47571","heartbeat_front_addr":"172.21.15.191:6827/47571","state":["exists","up"]},{"osd":7,"uuid":"e0b2fa32-8a5e-418c-8d1a-e8c7520519c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":47550},{"type":"v1","addr":"172.21.15.191:6801","nonce":47550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":47550},{"type":"v1","addr":"172.21.15.191:6803","nonce":47550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":47550},{"type":"v1","addr":"172.21.15.191:6807","nonce":47550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":47550},{"type":"v1","addr":"172.21.15.191:6805","nonce":47550}]},"public_addr":"172.21.15.191:6801/47550","cluster_addr":"172.21.15.191:6803/47550","heartbeat_back_addr":"172.21.15.191:6807/47550","heartbeat_front_addr":"172.21.15.191:6805/47550","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.230524+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.323232+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.327276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.057785+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.264741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.354836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.295132+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-14T09:01:09.269500+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6834/3041722362":"2024-03-15T09:04:45.094024+0000","172.21.15.154:6835/1688485831":"2024-03-15T09:04:38.103230+0000","172.21.15.154:6832/3041722362":"2024-03-15T09:04:45.094024+0000","172.21.15.154:6833/1688485831":"2024-03-15T09:04:38.103230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-14T09:04:51.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 msgr2=0x7f4fdc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 0x7f4fdc06ccf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0c3850 tx=0x7f4fec008030 comp rx=0 tx=0).stop 2024-03-14T09:04:51.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 msgr2=0x7f4ffc13ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:51.604 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00011e0 tx=0x7f4ff000cae0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 shutdown_connections 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4ffc15d270 0x7f4ffc1ed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f4fdc06a830 0x7f4fdc06ccf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ffc14eaf0 0x7f4ffc13ca60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 --2- 172.21.15.154:0/1402555140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ffc155df0 0x7f4ffc071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 >> 172.21.15.154:0/1402555140 conn(0x7f4ffc0c3880 msgr2=0x7f4ffc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 shutdown_connections 2024-03-14T09:04:51.605 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:51.604+0000 7f5000f60700 1 -- 172.21.15.154:0/1402555140 wait complete. 2024-03-14T09:04:52.616 INFO:tasks.ceph:Scrubbing osd.0 2024-03-14T09:04:52.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:52.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.755+0000 7f432a043700 1 Processor -- start 2024-03-14T09:04:52.755 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.755+0000 7f432a043700 1 -- start start 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f432a043700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f432a043700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f432a043700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432406da20 con 0x7f43240c4d20 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f432a043700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4324152280 con 0x7f43240c7920 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f432a043700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4324154930 con 0x7f4324151ea0 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f4322ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.757 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f4322ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:49830/0 (socket says 172.21.15.154:49830) 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f4323fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f43240c4d20 0x7f4324157e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:56154/0 (socket says 172.21.15.154:56154) 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:41474/0 (socket says 172.21.15.154:41474) 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f4322ffd700 1 -- 172.21.15.154:0/3216380337 learned_addr learned my addr 172.21.15.154:0/3216380337 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 -- 172.21.15.154:0/3216380337 >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 msgr2=0x7f43240c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 --2- 172.21.15.154:0/3216380337 >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.758 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 -- 172.21.15.154:0/3216380337 >> v1:172.21.15.191:6789/0 conn(0x7f43240c4d20 legacy=0x7f4324157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f43237fe700 1 -- 172.21.15.154:0/3216380337 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324154c10 con 0x7f4324151ea0 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.756+0000 7f4322ffd700 1 --2- 172.21.15.154:0/3216380337 >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f43237fe700 1 --2- 172.21.15.154:0/3216380337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f430c009840 tx=0x7f430c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb86d77b5541dd75 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f43227fc700 1 -- 172.21.15.154:0/3216380337 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c011830 con 0x7f4324151ea0 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f43227fc700 1 -- 172.21.15.154:0/3216380337 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c011990 con 0x7f4324151ea0 2024-03-14T09:04:52.759 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f43227fc700 1 -- 172.21.15.154:0/3216380337 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c011d20 con 0x7f4324151ea0 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 -- 172.21.15.154:0/3216380337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 msgr2=0x7f43240c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 --2- 172.21.15.154:0/3216380337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f430c009840 tx=0x7f430c009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 -- 172.21.15.154:0/3216380337 shutdown_connections 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 --2- 172.21.15.154:0/3216380337 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4324151ea0 0x7f43240c73e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 --2- 172.21.15.154:0/3216380337 >> v2:172.21.15.191:3300/0 conn(0x7f43240c7920 0x7f43240c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 -- 172.21.15.154:0/3216380337 >> 172.21.15.154:0/3216380337 conn(0x7f43240a9a80 msgr2=0x7f43240a9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 -- 172.21.15.154:0/3216380337 shutdown_connections 2024-03-14T09:04:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.757+0000 7f432a043700 1 -- 172.21.15.154:0/3216380337 wait complete. 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.758+0000 7f432a043700 1 Processor -- start 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 -- start start 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c4d20 0x7f432406de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 0x7f4324070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4324151ea0 0x7f4324151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c39a0 con 0x7f43240c7920 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43241ea9c0 con 0x7f4324151ea0 2024-03-14T09:04:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f432a043700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43241eaca0 con 0x7f43240c4d20 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.759+0000 7f4322ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 0x7f4324070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 0x7f4324070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47906/0 (socket says 172.21.15.154:47906) 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 -- 172.21.15.154:0/1184159627 learned_addr learned my addr 172.21.15.154:0/1184159627 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f43237fe700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c4d20 0x7f432406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4323fff700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4324151ea0 0x7f4324151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 -- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c4d20 msgr2=0x7f432406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c4d20 0x7f432406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 -- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4324151ea0 msgr2=0x7f4324151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4324151ea0 0x7f4324151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009550 con 0x7f43240c7920 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f43237fe700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c4d20 0x7f432406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f4322ffd700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 0x7f4324070070 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f431400ebb0 tx=0x7f431400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400c910 con 0x7f43240c7920 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400ca70 con 0x7f43240c7920 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c0094a0 con 0x7f43240c7920 2024-03-14T09:04:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400cde0 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.760+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432406e9c0 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.761+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43241551d0 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f4314026480 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f430406a830 0x7f430406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f43140893e0 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f4304070730 0x7f4304072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4304073200 con 0x7f4304070730 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4314089830 con 0x7f43240c7920 2024-03-14T09:04:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f43237fe700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f430406a830 0x7f430406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.762+0000 7f4323fff700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f4304070730 0x7f4304072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.763+0000 7f4323fff700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f4304070730 0x7f4304072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.763+0000 7f43237fe700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f430406a830 0x7f430406ccf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f430c000c00 tx=0x7f430c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.763+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== osd.0 v2:172.21.15.154:6816/47282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4304073200 con 0x7f4304070730 2024-03-14T09:04:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 --> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f43240697d0 con 0x7f4304070730 2024-03-14T09:04:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f431bfff700 1 -- 172.21.15.154:0/1184159627 <== osd.0 v2:172.21.15.154:6816/47282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f43240697d0 con 0x7f4304070730 2024-03-14T09:04:52.778 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:52.778 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:52.778 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:52.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f4304070730 msgr2=0x7f4304072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:6816/47282,v1:172.21.15.154:6817/47282] conn(0x7f4304070730 0x7f4304072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f430406a830 msgr2=0x7f430406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f430406a830 0x7f430406ccf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f430c000c00 tx=0x7f430c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 msgr2=0x7f4324070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 --2- 172.21.15.154:0/1184159627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43240c7920 0x7f4324070070 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f431400ebb0 tx=0x7f431400ef70 comp rx=0 tx=0).stop 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f4323fff700 1 -- 172.21.15.154:0/1184159627 reap_dead start 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 shutdown_connections 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 >> 172.21.15.154:0/1184159627 conn(0x7f43240a9a80 msgr2=0x7f43240a8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 shutdown_connections 2024-03-14T09:04:52.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.778+0000 7f432a043700 1 -- 172.21.15.154:0/1184159627 wait complete. 2024-03-14T09:04:52.790 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-14T09:04:52.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.912+0000 7f7cff26f700 1 Processor -- start 2024-03-14T09:04:52.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.912+0000 7f7cff26f700 1 -- start start 2024-03-14T09:04:52.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cff26f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cff26f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7cf806acb0 0x7f7cf8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cff26f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80ac8e0 con 0x7f7cf8156330 2024-03-14T09:04:52.921 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cff26f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf806b310 con 0x7f7cf806acb0 2024-03-14T09:04:52.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cff26f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf806de30 con 0x7f7cf814eaf0 2024-03-14T09:04:52.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cfc80a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7cf806acb0 0x7f7cf8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cfc80a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7cf806acb0 0x7f7cf8155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47922/0 (socket says 172.21.15.154:47922) 2024-03-14T09:04:52.922 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.913+0000 7f7cfc80a700 1 -- 172.21.15.154:0/3033472251 learned_addr learned my addr 172.21.15.154:0/3033472251 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:52.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfc80a700 1 -- 172.21.15.154:0/3033472251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 msgr2=0x7f7cf806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:52.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfc80a700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfc80a700 1 -- 172.21.15.154:0/3033472251 >> v1:172.21.15.191:6789/0 conn(0x7f7cf8156330 legacy=0x7f7cf8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:52.923 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfc80a700 1 -- 172.21.15.154:0/3033472251 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80acb30 con 0x7f7cf806acb0 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7cfc80a700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf806acb0 0x7f7cf8155df0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40098f0 tx=0x7f7cf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614d9155d1cc6f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7ceffff700 1 -- 172.21.15.154:0/3033472251 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4004230 con 0x7f7cf806acb0 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7ceffff700 1 -- 172.21.15.154:0/3033472251 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4004390 con 0x7f7cf806acb0 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.914+0000 7f7ceffff700 1 -- 172.21.15.154:0/3033472251 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40036a0 con 0x7f7cf806acb0 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 -- 172.21.15.154:0/3033472251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf806acb0 msgr2=0x7f7cf8155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf806acb0 0x7f7cf8155df0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40098f0 tx=0x7f7cf4009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 -- 172.21.15.154:0/3033472251 shutdown_connections 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 --2- 172.21.15.154:0/3033472251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf806acb0 0x7f7cf8155df0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 -- 172.21.15.154:0/3033472251 >> 172.21.15.154:0/3033472251 conn(0x7f7cf80c3880 msgr2=0x7f7cf80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 -- 172.21.15.154:0/3033472251 shutdown_connections 2024-03-14T09:04:52.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.915+0000 7f7cff26f700 1 -- 172.21.15.154:0/3033472251 wait complete. 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.916+0000 7f7cff26f700 1 Processor -- start 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.916+0000 7f7cff26f700 1 -- start start 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 0x7f7cf8149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80c17e0 con 0x7f7cf8156330 2024-03-14T09:04:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80c1fd0 con 0x7f7cf806acb0 2024-03-14T09:04:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cff26f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81444e0 con 0x7f7cf814eaf0 2024-03-14T09:04:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47936/0 (socket says 172.21.15.154:47936) 2024-03-14T09:04:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 -- 172.21.15.154:0/542170628 learned_addr learned my addr 172.21.15.154:0/542170628 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 0x7f7cf8149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfc80a700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 -- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 msgr2=0x7f7cf813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf813e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 -- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 msgr2=0x7f7cf8149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.917+0000 7f7cfd80c700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 0x7f7cf8149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cfd80c700 1 -- 172.21.15.154:0/542170628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009550 con 0x7f7cf8156330 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 0x7f7cf8149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cfd80c700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000cc10 tx=0x7f7cf000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000abb0 con 0x7f7cf8156330 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf000ad10 con 0x7f7cf8156330 2024-03-14T09:04:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf813f320 con 0x7f7cf8156330 2024-03-14T09:04:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0012660 con 0x7f7cf8156330 2024-03-14T09:04:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.918+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8144cf0 con 0x7f7cf8156330 2024-03-14T09:04:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.919+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80c2140 con 0x7f7cf8156330 2024-03-14T09:04:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.919+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7cf00127c0 con 0x7f7cf8156330 2024-03-14T09:04:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.920+0000 7f7cedffb700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 0x7f7cdc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.920+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7cf0089310 con 0x7f7cf8156330 2024-03-14T09:04:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.920+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 0x7f7cdc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.920+0000 7f7cfd00b700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 0x7f7cdc06ccf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80089b0 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:52.924+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f7cf80c2140 con 0x7f7cf8156330 2024-03-14T09:04:53.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.117+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf806e080 con 0x7f7cdc06a830 2024-03-14T09:04:53.117 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.117+0000 7f7cedffb700 1 -- 172.21.15.154:0/542170628 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ce800c040 con 0x7f7cdc06a830 2024-03-14T09:04:53.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 msgr2=0x7f7cdc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 0x7f7cdc06ccf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80089b0 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2024-03-14T09:04:53.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 msgr2=0x7f7cf813ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.120 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000cc10 tx=0x7f7cf000a060 comp rx=0 tx=0).stop 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 shutdown_connections 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7cdc06a830 0x7f7cdc06ccf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7cf8156330 0x7f7cf813ede0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cf806acb0 0x7f7cf8149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 --2- 172.21.15.154:0/542170628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cf814eaf0 0x7f7cf813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.121 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 >> 172.21.15.154:0/542170628 conn(0x7f7cf80c3880 msgr2=0x7f7cf8155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 shutdown_connections 2024-03-14T09:04:53.122 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.119+0000 7f7cff26f700 1 -- 172.21.15.154:0/542170628 wait complete. 2024-03-14T09:04:53.122 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 0 to deep-scrub 2024-03-14T09:04:53.132 INFO:tasks.ceph:Scrubbing osd.1 2024-03-14T09:04:53.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:53.259 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.259+0000 7efef0a4e700 1 Processor -- start 2024-03-14T09:04:53.260 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.259+0000 7efef0a4e700 1 -- start start 2024-03-14T09:04:53.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efef0a4e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec06d4d0 0x7efeec158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efef0a4e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efeec155360 0x7efeec06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efef0a4e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec157ed0 con 0x7efeec0c1380 2024-03-14T09:04:53.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efef0a4e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec158490 con 0x7efeec155360 2024-03-14T09:04:53.263 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efef0a4e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec06d8b0 con 0x7efeec06d4d0 2024-03-14T09:04:53.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efee9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efeec155360 0x7efeec06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efee9d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efeec155360 0x7efeec06c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47950/0 (socket says 172.21.15.154:47950) 2024-03-14T09:04:53.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.260+0000 7efee9d9b700 1 -- 172.21.15.154:0/1345413872 learned_addr learned my addr 172.21.15.154:0/1345413872 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.264 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efeea59c700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec06d4d0 0x7efeec158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee9d9b700 1 -- 172.21.15.154:0/1345413872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec06d4d0 msgr2=0x7efeec158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee9d9b700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec06d4d0 0x7efeec158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee9d9b700 1 -- 172.21.15.154:0/1345413872 >> v1:172.21.15.191:6789/0 conn(0x7efeec0c1380 legacy=0x7efeec14ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee9d9b700 1 -- 172.21.15.154:0/1345413872 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec1557e0 con 0x7efeec155360 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee9d9b700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec06c610 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efedc0098f0 tx=0x7efedc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccb9e6264ea8bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee959a700 1 -- 172.21.15.154:0/1345413872 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc011830 con 0x7efeec155360 2024-03-14T09:04:53.265 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee959a700 1 -- 172.21.15.154:0/1345413872 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc011990 con 0x7efeec155360 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.261+0000 7efee959a700 1 -- 172.21.15.154:0/1345413872 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc011d20 con 0x7efeec155360 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 -- 172.21.15.154:0/1345413872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 msgr2=0x7efeec06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec06c610 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7efedc0098f0 tx=0x7efedc009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 -- 172.21.15.154:0/1345413872 shutdown_connections 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec06d4d0 0x7efeec158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 --2- 172.21.15.154:0/1345413872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec06c610 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.266 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 -- 172.21.15.154:0/1345413872 >> 172.21.15.154:0/1345413872 conn(0x7efeec0a9d00 msgr2=0x7efeec0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 -- 172.21.15.154:0/1345413872 shutdown_connections 2024-03-14T09:04:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.262+0000 7efef0a4e700 1 -- 172.21.15.154:0/1345413872 wait complete. 2024-03-14T09:04:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.263+0000 7efef0a4e700 1 Processor -- start 2024-03-14T09:04:53.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.263+0000 7efef0a4e700 1 -- start start 2024-03-14T09:04:53.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeec06d4d0 0x7efeec06f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec0c1380 0x7efeec0a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0ad120 con 0x7efeec155360 2024-03-14T09:04:53.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1440c0 con 0x7efeec06d4d0 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0aae60 con 0x7efeec0c1380 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec1525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47952/0 (socket says 172.21.15.154:47952) 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 -- 172.21.15.154:0/4185424024 learned_addr learned my addr 172.21.15.154:0/4185424024 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeea59c700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeec06d4d0 0x7efeec06f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efee9d9b700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec0c1380 0x7efeec0a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 -- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec0c1380 msgr2=0x7efeec0a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efeec0c1380 0x7efeec0a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 -- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeec06d4d0 msgr2=0x7efeec06f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efeec06d4d0 0x7efeec06f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009550 con 0x7efeec155360 2024-03-14T09:04:53.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efeead9d700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec1525d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efee400cc50 tx=0x7efee400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4003c00 con 0x7efeec155360 2024-03-14T09:04:53.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4003d60 con 0x7efeec155360 2024-03-14T09:04:53.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec144be0 con 0x7efeec155360 2024-03-14T09:04:53.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.264+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400aab0 con 0x7efeec155360 2024-03-14T09:04:53.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.265+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec1ea4d0 con 0x7efeec155360 2024-03-14T09:04:53.275 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.265+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeec158120 con 0x7efeec155360 2024-03-14T09:04:53.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.266+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7efee400ac10 con 0x7efeec155360 2024-03-14T09:04:53.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.266+0000 7efedaffd700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efecc06a830 0x7efecc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.266+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efee4088a10 con 0x7efeec155360 2024-03-14T09:04:53.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.266+0000 7efeea59c700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efecc06a830 0x7efecc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.276 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efedaffd700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7efecc070730 0x7efecc072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efecc073200 con 0x7efecc070730 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7efee408d2c0 con 0x7efeec155360 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efee9d9b700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7efecc070730 0x7efecc072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efee9d9b700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7efecc070730 0x7efecc072b50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.267+0000 7efeea59c700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efecc06a830 0x7efecc06ccf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efee0009be0 tx=0x7efee0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.268+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== osd.1 v2:172.21.15.154:6824/47283 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7efecc073200 con 0x7efecc070730 2024-03-14T09:04:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.282+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 --> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efeec070bb0 con 0x7efecc070730 2024-03-14T09:04:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.282+0000 7efedaffd700 1 -- 172.21.15.154:0/4185424024 <== osd.1 v2:172.21.15.154:6824/47283 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efedc021040 con 0x7efecc070730 2024-03-14T09:04:53.286 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:53.287 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:53.287 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:53.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7efecc070730 msgr2=0x7efecc072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:6824/47283,v1:172.21.15.154:6825/47283] conn(0x7efecc070730 0x7efecc072b50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efecc06a830 msgr2=0x7efecc06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7efecc06a830 0x7efecc06ccf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efee0009be0 tx=0x7efee0009210 comp rx=0 tx=0).stop 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 msgr2=0x7efeec1525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 --2- 172.21.15.154:0/4185424024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efeec155360 0x7efeec1525d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7efee400cc50 tx=0x7efee400cf60 comp rx=0 tx=0).stop 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efeead9d700 1 -- 172.21.15.154:0/4185424024 reap_dead start 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 shutdown_connections 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 >> 172.21.15.154:0/4185424024 conn(0x7efeec0a9d00 msgr2=0x7efeec0a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 shutdown_connections 2024-03-14T09:04:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.283+0000 7efef0a4e700 1 -- 172.21.15.154:0/4185424024 wait complete. 2024-03-14T09:04:53.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-14T09:04:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 Processor -- start 2024-03-14T09:04:53.418 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 -- start start 2024-03-14T09:04:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 0x7f5f2814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f280acd10 con 0x7f5f280702f0 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f28150110 con 0x7f5f280c4380 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.418+0000 7f5f2f81f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280ac7b0 con 0x7f5f280c5d60 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2cdba700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 0x7f5f2814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:41526/0 (socket says 172.21.15.154:41526) 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 -- 172.21.15.154:0/278760524 learned_addr learned my addr 172.21.15.154:0/278760524 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 -- 172.21.15.154:0/278760524 >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 msgr2=0x7f5f2814fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 --2- 172.21.15.154:0/278760524 >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 0x7f5f2814fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 -- 172.21.15.154:0/278760524 >> v1:172.21.15.191:6789/0 conn(0x7f5f280702f0 legacy=0x7f5f280c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 -- 172.21.15.154:0/278760524 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280ac9c0 con 0x7f5f280c5d60 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2cdba700 1 --2- 172.21.15.154:0/278760524 >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 0x7f5f2814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.419+0000 7f5f2ddbc700 1 --2- 172.21.15.154:0/278760524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e50 tx=0x7f5f2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a4a1a3d84df928 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f1ffff700 1 -- 172.21.15.154:0/278760524 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000e070 con 0x7f5f280c5d60 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f1ffff700 1 -- 172.21.15.154:0/278760524 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2000b910 con 0x7f5f280c5d60 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f1ffff700 1 -- 172.21.15.154:0/278760524 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000bcc0 con 0x7f5f280c5d60 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 -- 172.21.15.154:0/278760524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 msgr2=0x7f5f280c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/278760524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e50 tx=0x7f5f2000b330 comp rx=0 tx=0).stop 2024-03-14T09:04:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 -- 172.21.15.154:0/278760524 shutdown_connections 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/278760524 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f280c61b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/278760524 >> v2:172.21.15.191:3300/0 conn(0x7f5f280c4380 0x7f5f2814fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 -- 172.21.15.154:0/278760524 >> 172.21.15.154:0/278760524 conn(0x7f5f280a9bf0 msgr2=0x7f5f280a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 -- 172.21.15.154:0/278760524 shutdown_connections 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.420+0000 7f5f2f81f700 1 -- 172.21.15.154:0/278760524 wait complete. 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.421+0000 7f5f2f81f700 1 Processor -- start 2024-03-14T09:04:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.421+0000 7f5f2f81f700 1 -- start start 2024-03-14T09:04:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f2813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280c7860 con 0x7f5f280c4380 2024-03-14T09:04:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f281ed2b0 con 0x7f5f280702f0 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2f81f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28142550 con 0x7f5f280c5d60 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2cdba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2d5bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2d5bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:49862/0 (socket says 172.21.15.154:49862) 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2ddbc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f2813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2cdba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47962/0 (socket says 172.21.15.154:47962) 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.422+0000 7f5f2d5bb700 1 -- 172.21.15.154:0/2139372763 learned_addr learned my addr 172.21.15.154:0/2139372763 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 -- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 msgr2=0x7f5f2813f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f2813f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 -- 172.21.15.154:0/2139372763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 msgr2=0x7f5f2813f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 -- 172.21.15.154:0/2139372763 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240094a0 con 0x7f5f280702f0 2024-03-14T09:04:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2ddbc700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f2813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2cdba700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2d5bb700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c450 tx=0x7f5f1800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180052d0 con 0x7f5f280702f0 2024-03-14T09:04:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18005430 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f20009b30 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180038c0 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.423+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f281ed830 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.424+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280697d0 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.425+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f5f18003a20 con 0x7f5f280702f0 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.425+0000 7f5f1dffb700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 0x7f5f0c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.425+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5f18087cf0 con 0x7f5f280702f0 2024-03-14T09:04:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.425+0000 7f5f2cdba700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 0x7f5f0c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.426+0000 7f5f2cdba700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 0x7f5f0c06ccf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f2814f4d0 tx=0x7f5f24009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.430+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5f1808c070 con 0x7f5f280702f0 2024-03-14T09:04:53.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.626+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f28070ad0 con 0x7f5f0c06a830 2024-03-14T09:04:53.626 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.626+0000 7f5f1dffb700 1 -- 172.21.15.154:0/2139372763 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5f24011e60 con 0x7f5f0c06a830 2024-03-14T09:04:53.629 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 msgr2=0x7f5f0c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 0x7f5f0c06ccf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f2814f4d0 tx=0x7f5f24009210 comp rx=0 tx=0).stop 2024-03-14T09:04:53.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 msgr2=0x7f5f280c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.630 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c450 tx=0x7f5f1800c760 comp rx=0 tx=0).stop 2024-03-14T09:04:53.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 shutdown_connections 2024-03-14T09:04:53.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f280c5d60 0x7f5f2813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f280702f0 0x7f5f280c1270 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.631 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f5f0c06a830 0x7f5f0c06ccf0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 --2- 172.21.15.154:0/2139372763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f280c4380 0x7f5f2813f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 >> 172.21.15.154:0/2139372763 conn(0x7f5f280a9bf0 msgr2=0x7f5f280c2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 shutdown_connections 2024-03-14T09:04:53.632 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.628+0000 7f5f2f81f700 1 -- 172.21.15.154:0/2139372763 wait complete. 2024-03-14T09:04:53.633 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 1 to deep-scrub 2024-03-14T09:04:53.641 INFO:tasks.ceph:Scrubbing osd.2 2024-03-14T09:04:53.641 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:53.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.768+0000 7fa9b6800700 1 Processor -- start 2024-03-14T09:04:53.768 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.768+0000 7fa9b6800700 1 -- start start 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b6800700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b0155360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b6800700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa9b00c1380 0x7fa9b00c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b6800700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b0157ed0 con 0x7fa9b006d4d0 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b6800700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b01558a0 con 0x7fa9b00c1380 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b6800700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0158490 con 0x7fa9b0158920 2024-03-14T09:04:53.771 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa9b00c1380 0x7fa9b00c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa9b00c1380 0x7fa9b00c1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47970/0 (socket says 172.21.15.154:47970) 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 -- 172.21.15.154:0/1697011198 learned_addr learned my addr 172.21.15.154:0/1697011198 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9af7fe700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b0155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 -- 172.21.15.154:0/1697011198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 msgr2=0x7fa9b0155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b0155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 -- 172.21.15.154:0/1697011198 >> v1:172.21.15.191:6789/0 conn(0x7fa9b006d4d0 legacy=0x7fa9b006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 -- 172.21.15.154:0/1697011198 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b006d8b0 con 0x7fa9b00c1380 2024-03-14T09:04:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9af7fe700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b0155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b00c1380 0x7fa9b00c1810 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96af2905b0a253d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9aeffd700 1 -- 172.21.15.154:0/1697011198 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800e070 con 0x7fa9b00c1380 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9aeffd700 1 -- 172.21.15.154:0/1697011198 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8003a20 con 0x7fa9b00c1380 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.769+0000 7fa9aeffd700 1 -- 172.21.15.154:0/1697011198 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8003dd0 con 0x7fa9b00c1380 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 -- 172.21.15.154:0/1697011198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b00c1380 msgr2=0x7fa9b00c1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b00c1380 0x7fa9b00c1810 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009e50 tx=0x7fa9a800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 -- 172.21.15.154:0/1697011198 shutdown_connections 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b00c1380 0x7fa9b00c1810 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 --2- 172.21.15.154:0/1697011198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b0155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 -- 172.21.15.154:0/1697011198 >> 172.21.15.154:0/1697011198 conn(0x7fa9b00a9d00 msgr2=0x7fa9b00aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 -- 172.21.15.154:0/1697011198 shutdown_connections 2024-03-14T09:04:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.770+0000 7fa9b6800700 1 -- 172.21.15.154:0/1697011198 wait complete. 2024-03-14T09:04:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.771+0000 7fa9b6800700 1 Processor -- start 2024-03-14T09:04:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.771+0000 7fa9b6800700 1 -- start start 2024-03-14T09:04:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 0x7fa9b0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9b00c1380 0x7fa9b014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b01ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0152b40 con 0x7fa9b006d4d0 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b01540f0 con 0x7fa9b00c1380 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b6800700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00704b0 con 0x7fa9b0158920 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 0x7fa9b0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 0x7fa9b0154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47980/0 (socket says 172.21.15.154:47980) 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 -- 172.21.15.154:0/3220661531 learned_addr learned my addr 172.21.15.154:0/3220661531 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9af7fe700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9b00c1380 0x7fa9b014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b01ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 -- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 msgr2=0x7fa9b01ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b01ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 -- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9b00c1380 msgr2=0x7fa9b014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9b00c1380 0x7fa9b014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9affff700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8009b30 con 0x7fa9b006d4d0 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.772+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9b0158920 0x7fa9b01ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9affff700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 0x7fa9b0154510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c4d0 tx=0x7fa9a400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011790 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40118f0 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b0153d50 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011ca0 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.773+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b0151500 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.774+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9b00708d0 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.774+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fa9a401c460 con 0x7fa9b006d4d0 2024-03-14T09:04:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9acff9700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa99c06a860 0x7fa99c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa9a4088840 con 0x7fa9b006d4d0 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9acff9700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7fa99c070760 0x7fa99c072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa99c073230 con 0x7fa99c070760 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9a408ae20 con 0x7fa9b006d4d0 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9af7fe700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa99c06a860 0x7fa99c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7fa99c070760 0x7fa99c072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9b4d9d700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7fa99c070760 0x7fa99c072b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.775+0000 7fa9af7fe700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa99c06a860 0x7fa99c06cd20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa9b01eb600 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.776+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== osd.2 v2:172.21.15.154:6802/47280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa99c073230 con 0x7fa99c070760 2024-03-14T09:04:53.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 --> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa9b006cd20 con 0x7fa99c070760 2024-03-14T09:04:53.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9acff9700 1 -- 172.21.15.154:0/3220661531 <== osd.2 v2:172.21.15.154:6802/47280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa9a8009340 con 0x7fa99c070760 2024-03-14T09:04:53.792 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:53.793 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:53.793 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7fa99c070760 msgr2=0x7fa99c072b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:6802/47280,v1:172.21.15.154:6804/47280] conn(0x7fa99c070760 0x7fa99c072b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa99c06a860 msgr2=0x7fa99c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fa99c06a860 0x7fa99c06cd20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa9b01eb600 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 msgr2=0x7fa9b0154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.794 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 --2- 172.21.15.154:0/3220661531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa9b006d4d0 0x7fa9b0154510 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c4d0 tx=0x7fa9a400c7e0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b4d9d700 1 -- 172.21.15.154:0/3220661531 reap_dead start 2024-03-14T09:04:53.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 shutdown_connections 2024-03-14T09:04:53.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.790+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 >> 172.21.15.154:0/3220661531 conn(0x7fa9b00a9d00 msgr2=0x7fa9b00c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.791+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 shutdown_connections 2024-03-14T09:04:53.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.791+0000 7fa9b6800700 1 -- 172.21.15.154:0/3220661531 wait complete. 2024-03-14T09:04:53.802 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-14T09:04:53.924 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.924+0000 7f7891f85700 1 Processor -- start 2024-03-14T09:04:53.925 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 -- start start 2024-03-14T09:04:53.927 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f788c0c7f70 0x7f788c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 0x7f788c1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 0x7f788c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c158ac0 con 0x7f788c0c7f70 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c157e80 con 0x7f788c0c8350 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f7891f85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c158120 con 0x7f788c0c4f30 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f788b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f788c0c7f70 0x7f788c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f788b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f788c0c7f70 0x7f788c0c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47994/0 (socket says 172.21.15.154:47994) 2024-03-14T09:04:53.928 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.925+0000 7f788b7fe700 1 -- 172.21.15.154:0/1085210733 learned_addr learned my addr 172.21.15.154:0/1085210733 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788affd700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 0x7f788c1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788bfff700 1 --2- 172.21.15.154:0/1085210733 >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 0x7f788c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 -- 172.21.15.154:0/1085210733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 msgr2=0x7f788c1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 0x7f788c1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 -- 172.21.15.154:0/1085210733 >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 msgr2=0x7f788c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 --2- 172.21.15.154:0/1085210733 >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 0x7f788c154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 -- 172.21.15.154:0/1085210733 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c158400 con 0x7f788c0c7f70 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788affd700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 0x7f788c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:53.929 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788bfff700 1 --2- 172.21.15.154:0/1085210733 >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 0x7f788c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788b7fe700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c7f70 0x7f788c0c49f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f788c0c0fa0 tx=0x7f787c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753726c49ac982c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788a7fc700 1 -- 172.21.15.154:0/1085210733 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00b870 con 0x7f788c0c7f70 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788a7fc700 1 -- 172.21.15.154:0/1085210733 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00b9d0 con 0x7f788c0c7f70 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.926+0000 7f788a7fc700 1 -- 172.21.15.154:0/1085210733 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00bd60 con 0x7f788c0c7f70 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 -- 172.21.15.154:0/1085210733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c7f70 msgr2=0x7f788c0c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c7f70 0x7f788c0c49f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f788c0c0fa0 tx=0x7f787c009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 -- 172.21.15.154:0/1085210733 shutdown_connections 2024-03-14T09:04:53.930 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 --2- 172.21.15.154:0/1085210733 >> v2:172.21.15.191:3300/0 conn(0x7f788c0c8350 0x7f788c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c7f70 0x7f788c0c49f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 --2- 172.21.15.154:0/1085210733 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c4f30 0x7f788c1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 -- 172.21.15.154:0/1085210733 >> 172.21.15.154:0/1085210733 conn(0x7f788c072080 msgr2=0x7f788c072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 -- 172.21.15.154:0/1085210733 shutdown_connections 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.927+0000 7f7891f85700 1 -- 172.21.15.154:0/1085210733 wait complete. 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.928+0000 7f7891f85700 1 Processor -- start 2024-03-14T09:04:53.931 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.928+0000 7f7891f85700 1 -- start start 2024-03-14T09:04:53.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 0x7f788c0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c8350 0x7f788c1ed330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c146f90 con 0x7f788c0c4f30 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0c3f10 con 0x7f788c0c7f70 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0c2d10 con 0x7f788c0c8350 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47998/0 (socket says 172.21.15.154:47998) 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 -- 172.21.15.154:0/3726107454 learned_addr learned my addr 172.21.15.154:0/3726107454 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788bfff700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c8350 0x7f788c1ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788affd700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 0x7f788c0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.935 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 -- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c8350 msgr2=0x7f788c1ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c8350 0x7f788c1ed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 -- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 msgr2=0x7f788c0abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 0x7f788c0abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 -- 172.21.15.154:0/3726107454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c009550 con 0x7f788c0c4f30 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788affd700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 0x7f788c0abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f788b7fe700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f788c0cb0b0 tx=0x7f787c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011760 con 0x7f788c0c4f30 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c01f070 con 0x7f788c0c4f30 2024-03-14T09:04:53.936 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c0c3900 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011ac0 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.929+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0c3ba0 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.930+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c0697d0 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.931+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f787c021030 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.931+0000 7f7873fff700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 0x7f787806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.931+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f787c08a020 con 0x7f788c0c4f30 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.931+0000 7f788affd700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 0x7f787806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:53.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.932+0000 7f788affd700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 0x7f787806cd20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f788c072890 tx=0x7f7880008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:53.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:53.936+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f787c055770 con 0x7f788c0c4f30 2024-03-14T09:04:54.127 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.127+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f788c0bd8b0 con 0x7f787806a860 2024-03-14T09:04:54.128 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.128+0000 7f7873fff700 1 -- 172.21.15.154:0/3726107454 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f788000a740 con 0x7f787806a860 2024-03-14T09:04:54.130 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 msgr2=0x7f787806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 0x7f787806cd20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f788c072890 tx=0x7f7880008040 comp rx=0 tx=0).stop 2024-03-14T09:04:54.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 msgr2=0x7f788c0ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f788c0cb0b0 tx=0x7f787c004750 comp rx=0 tx=0).stop 2024-03-14T09:04:54.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 shutdown_connections 2024-03-14T09:04:54.131 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f788c0c8350 0x7f788c1ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f787806a860 0x7f787806cd20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f788c0c4f30 0x7f788c0ac720 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 --2- 172.21.15.154:0/3726107454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f788c0c7f70 0x7f788c0abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 >> 172.21.15.154:0/3726107454 conn(0x7f788c072080 msgr2=0x7f788c150be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 shutdown_connections 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.130+0000 7f7891f85700 1 -- 172.21.15.154:0/3726107454 wait complete. 2024-03-14T09:04:54.132 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 2 to deep-scrub 2024-03-14T09:04:54.142 INFO:tasks.ceph:Scrubbing osd.3 2024-03-14T09:04:54.143 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:54.267 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.267+0000 7fd6c6906700 1 Processor -- start 2024-03-14T09:04:54.268 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 -- start start 2024-03-14T09:04:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c00c3130 con 0x7fd6c00c2b40 2024-03-14T09:04:54.270 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c00c1300 con 0x7fd6c00c2050 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.268+0000 7fd6c6906700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00c1020 con 0x7fd6c00c1560 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6c4ea3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6c4ea3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.154:49892/0 (socket says 172.21.15.154:49892) 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/3104706230 learned_addr learned my addr 172.21.15.154:0/3104706230 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:41592/0 (socket says 172.21.15.154:41592) 2024-03-14T09:04:54.271 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bf7fe700 1 --1- 172.21.15.154:0/3104706230 >> v1:172.21.15.191:6789/0 conn(0x7fd6c00c2b40 0x7fd6c01547c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.154:56220/0 (socket says 172.21.15.154:56220) 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 -- 172.21.15.154:0/3104706230 >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 msgr2=0x7fd6c00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 --2- 172.21.15.154:0/3104706230 >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 -- 172.21.15.154:0/3104706230 >> v1:172.21.15.191:6789/0 conn(0x7fd6c00c2b40 legacy=0x7fd6c01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 -- 172.21.15.154:0/3104706230 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c006d5a0 con 0x7fd6c00c1560 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6c4ea3700 1 --2- 172.21.15.154:0/3104706230 >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6bffff700 1 --2- 172.21.15.154:0/3104706230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abcff8a316783ae server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6beffd700 1 -- 172.21.15.154:0/3104706230 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011830 con 0x7fd6c00c1560 2024-03-14T09:04:54.272 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6beffd700 1 -- 172.21.15.154:0/3104706230 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0011990 con 0x7fd6c00c1560 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.269+0000 7fd6beffd700 1 -- 172.21.15.154:0/3104706230 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011d20 con 0x7fd6c00c1560 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 -- 172.21.15.154:0/3104706230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 msgr2=0x7fd6c00c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 --2- 172.21.15.154:0/3104706230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0009b50 comp rx=0 tx=0).stop 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 -- 172.21.15.154:0/3104706230 shutdown_connections 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 --2- 172.21.15.154:0/3104706230 >> v2:172.21.15.191:3300/0 conn(0x7fd6c00c2050 0x7fd6c00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 --2- 172.21.15.154:0/3104706230 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c1560 0x7fd6c00c35c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 -- 172.21.15.154:0/3104706230 >> 172.21.15.154:0/3104706230 conn(0x7fd6c00c3b40 msgr2=0x7fd6c00c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.273 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 -- 172.21.15.154:0/3104706230 shutdown_connections 2024-03-14T09:04:54.274 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.270+0000 7fd6c6906700 1 -- 172.21.15.154:0/3104706230 wait complete. 2024-03-14T09:04:54.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.271+0000 7fd6c6906700 1 Processor -- start 2024-03-14T09:04:54.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.271+0000 7fd6c6906700 1 -- start start 2024-03-14T09:04:54.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c00c1560 0x7fd6c01e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c2050 0x7fd6c01474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 0x7fd6c01479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0152630 con 0x7fd6c00c2b40 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00c43e0 con 0x7fd6c00c1560 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c6906700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01495d0 con 0x7fd6c00c2050 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c4ea3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 0x7fd6c01479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c4ea3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 0x7fd6c01479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48006/0 (socket says 172.21.15.154:48006) 2024-03-14T09:04:54.278 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/4141246972 learned_addr learned my addr 172.21.15.154:0/4141246972 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6bffff700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c00c1560 0x7fd6c01e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.272+0000 7fd6bf7fe700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c2050 0x7fd6c01474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c2050 msgr2=0x7fd6c01474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd6c00c2050 0x7fd6c01474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c00c1560 msgr2=0x7fd6c01e16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c00c1560 0x7fd6c01e16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009b30 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c4ea3700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 0x7fd6c01479f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800efc0 tx=0x7fd6b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8018070 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b800f040 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8003c40 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b00094a0 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.273+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c014cba0 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.274+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6c0154ed0 con 0x7fd6c00c2b40 2024-03-14T09:04:54.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.274+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fd6b800c880 con 0x7fd6c00c2b40 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bcff9700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd6a006a830 0x7fd6a006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd6b8088b60 con 0x7fd6c00c2b40 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bcff9700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7fd6a0070730 0x7fd6a0072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bffff700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd6a006a830 0x7fd6a006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6a0073200 con 0x7fd6a0070730 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6b808d2a0 con 0x7fd6c00c2b40 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bf7fe700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7fd6a0070730 0x7fd6a0072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bf7fe700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7fd6a0070730 0x7fd6a0072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.275+0000 7fd6bffff700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd6a006a830 0x7fd6a006ccf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0000c00 tx=0x7fd6b001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.276+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== osd.3 v2:172.21.15.154:6800/47281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fd6a0073200 con 0x7fd6a0070730 2024-03-14T09:04:54.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.290+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 --> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd6c014cd30 con 0x7fd6a0070730 2024-03-14T09:04:54.290 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.290+0000 7fd6bcff9700 1 -- 172.21.15.154:0/4141246972 <== osd.3 v2:172.21.15.154:6800/47281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd6c014cd30 con 0x7fd6a0070730 2024-03-14T09:04:54.291 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:54.291 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:54.291 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:54.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7fd6a0070730 msgr2=0x7fd6a0072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.292 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:6800/47281,v1:172.21.15.154:6801/47281] conn(0x7fd6a0070730 0x7fd6a0072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd6a006a830 msgr2=0x7fd6a006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fd6a006a830 0x7fd6a006ccf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0000c00 tx=0x7fd6b001c040 comp rx=0 tx=0).stop 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 msgr2=0x7fd6c01479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 --2- 172.21.15.154:0/4141246972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c00c2b40 0x7fd6c01479f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800efc0 tx=0x7fd6b800c3a0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c4ea3700 1 -- 172.21.15.154:0/4141246972 reap_dead start 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 shutdown_connections 2024-03-14T09:04:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 >> 172.21.15.154:0/4141246972 conn(0x7fd6c00c3b40 msgr2=0x7fd6c00c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 shutdown_connections 2024-03-14T09:04:54.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.291+0000 7fd6c6906700 1 -- 172.21.15.154:0/4141246972 wait complete. 2024-03-14T09:04:54.303 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-14T09:04:54.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.428+0000 7f28769d4700 1 Processor -- start 2024-03-14T09:04:54.428 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.428+0000 7f28769d4700 1 -- start start 2024-03-14T09:04:54.431 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f28769d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c4380 0x7f287014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f28769d4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f28700c5d60 0x7f28700c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f28769d4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700acd10 con 0x7f28700702f0 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f28769d4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2870150110 con 0x7f28700c5d60 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f28769d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700ac7b0 con 0x7f28700c4380 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f28700c5d60 0x7f28700c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.432 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f286ffff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f28700702f0 0x7f28700c7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48132/0 (socket says 172.21.15.154:48132) 2024-03-14T09:04:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f286ffff700 1 -- 172.21.15.154:0/1709423293 learned_addr learned my addr 172.21.15.154:0/1709423293 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f28700c5d60 0x7f28700c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48016/0 (socket says 172.21.15.154:48016) 2024-03-14T09:04:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f286f7fe700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c4380 0x7f287014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 -- 172.21.15.154:0/1709423293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c4380 msgr2=0x7f287014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:54.433 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c4380 0x7f287014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 -- 172.21.15.154:0/1709423293 >> v1:172.21.15.154:6789/0 conn(0x7f28700702f0 legacy=0x7f28700c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f286effd700 1 -- 172.21.15.154:0/1709423293 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11797740 0 0) 0x7f28700acd10 con 0x7f28700702f0 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 -- 172.21.15.154:0/1709423293 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700ac9c0 con 0x7f28700c5d60 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f2874f71700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700c5d60 0x7f28700c61d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2868009d90 tx=0x7f286800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eef168969a593b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.429+0000 7f286effd700 1 -- 172.21.15.154:0/1709423293 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800e070 con 0x7f28700c5d60 2024-03-14T09:04:54.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f286effd700 1 -- 172.21.15.154:0/1709423293 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286800bb20 con 0x7f28700c5d60 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f286effd700 1 -- 172.21.15.154:0/1709423293 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868012430 con 0x7f28700c5d60 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 -- 172.21.15.154:0/1709423293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700c5d60 msgr2=0x7f28700c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700c5d60 0x7f28700c61d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f2868009d90 tx=0x7f286800b330 comp rx=0 tx=0).stop 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 -- 172.21.15.154:0/1709423293 shutdown_connections 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700c5d60 0x7f28700c61d0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 --2- 172.21.15.154:0/1709423293 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c4380 0x7f287014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.435 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 -- 172.21.15.154:0/1709423293 >> 172.21.15.154:0/1709423293 conn(0x7f28700a9bf0 msgr2=0x7f28700a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 -- 172.21.15.154:0/1709423293 shutdown_connections 2024-03-14T09:04:54.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.430+0000 7f28769d4700 1 -- 172.21.15.154:0/1709423293 wait complete. 2024-03-14T09:04:54.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.431+0000 7f28769d4700 1 Processor -- start 2024-03-14T09:04:54.436 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.431+0000 7f28769d4700 1 -- start start 2024-03-14T09:04:54.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28700c4380 0x7f28701448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c5d60 0x7f287013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870150c60 con 0x7f28700702f0 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870154a40 con 0x7f28700c4380 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f28769d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2870150630 con 0x7f28700c5d60 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48028/0 (socket says 172.21.15.154:48028) 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 -- 172.21.15.154:0/1114938073 learned_addr learned my addr 172.21.15.154:0/1114938073 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.440 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 -- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c5d60 msgr2=0x7f287013ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f2874f71700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c5d60 0x7f287013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286f7fe700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28700c4380 0x7f28701448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c5d60 0x7f287013ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 -- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28700c4380 msgr2=0x7f28701448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.432+0000 7f286ffff700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28700c4380 0x7f28701448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f286ffff700 1 -- 172.21.15.154:0/1114938073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2868009a70 con 0x7f28700702f0 2024-03-14T09:04:54.441 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f286ffff700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f28640011e0 tx=0x7f286400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864004c80 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864004e00 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864007310 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28701ebbc0 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.433+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2870150880 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.434+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870150da0 con 0x7f28700702f0 2024-03-14T09:04:54.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.434+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f2864007470 con 0x7f28700702f0 2024-03-14T09:04:54.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.435+0000 7f286cff9700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 0x7f285c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.435+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f28640887c0 con 0x7f28700702f0 2024-03-14T09:04:54.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.435+0000 7f286f7fe700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 0x7f285c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.436+0000 7f286f7fe700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 0x7f285c06ccf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f28700c2ff0 tx=0x7f2860008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.439+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f2870150da0 con 0x7f28700702f0 2024-03-14T09:04:54.635 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.635+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2870070ad0 con 0x7f285c06a830 2024-03-14T09:04:54.636 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.636+0000 7f286cff9700 1 -- 172.21.15.154:0/1114938073 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f28600106e0 con 0x7f285c06a830 2024-03-14T09:04:54.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 msgr2=0x7f285c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 0x7f285c06ccf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f28700c2ff0 tx=0x7f2860008040 comp rx=0 tx=0).stop 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 msgr2=0x7f2870154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f28640011e0 tx=0x7f286400cae0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 shutdown_connections 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28700c5d60 0x7f287013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f285c06a830 0x7f285c06ccf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28700702f0 0x7f2870154500 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.639 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 --2- 172.21.15.154:0/1114938073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28700c4380 0x7f28701448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 >> 172.21.15.154:0/1114938073 conn(0x7f28700a9bf0 msgr2=0x7f28700a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 shutdown_connections 2024-03-14T09:04:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.638+0000 7f28769d4700 1 -- 172.21.15.154:0/1114938073 wait complete. 2024-03-14T09:04:54.640 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 3 to deep-scrub 2024-03-14T09:04:54.650 INFO:tasks.ceph:Scrubbing osd.4 2024-03-14T09:04:54.650 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:54.776 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.776+0000 7f7d6c9de700 1 Processor -- start 2024-03-14T09:04:54.777 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.777+0000 7f7d6c9de700 1 -- start start 2024-03-14T09:04:54.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.777+0000 7f7d6c9de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 0x7f7d680c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.777+0000 7f7d6c9de700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d681563d0 con 0x7f7d680c5e50 2024-03-14T09:04:54.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.777+0000 7f7d6c9de700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680c4670 con 0x7f7d680c62b0 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.777+0000 7f7d6c9de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6806f960 con 0x7f7d680c41e0 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d65d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7d680c5e50 0x7f7d68070490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48148/0 (socket says 172.21.15.154:48148) 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d65d9b700 1 -- 172.21.15.154:0/3193725575 learned_addr learned my addr 172.21.15.154:0/3193725575 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6659c700 1 --2- 172.21.15.154:0/3193725575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 0x7f7d680c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229377740 0 0) 0x7f7d681563d0 con 0x7f7d680c5e50 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d540035b0 con 0x7f7d680c5e50 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342542799 0 0) 0x7f7d680c4670 con 0x7f7d680c62b0 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d681563d0 con 0x7f7d680c62b0 2024-03-14T09:04:54.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679975199 0 0) 0x7f7d540035b0 con 0x7f7d680c5e50 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6659c700 1 -- 172.21.15.154:0/3193725575 >> v1:172.21.15.191:6789/0 conn(0x7f7d680c62b0 legacy=0x7f7d6815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6659c700 1 -- 172.21.15.154:0/3193725575 >> v1:172.21.15.154:6789/0 conn(0x7f7d680c5e50 legacy=0x7f7d68070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6659c700 1 -- 172.21.15.154:0/3193725575 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68155d80 con 0x7f7d680c41e0 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.778+0000 7f7d6659c700 1 --2- 172.21.15.154:0/3193725575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 0x7f7d680c38f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009870 tx=0x7f7d58009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527a834c88b4d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50001670 con 0x7f7d680c41e0 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58004030 con 0x7f7d680c41e0 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6559a700 1 -- 172.21.15.154:0/3193725575 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580043c0 con 0x7f7d680c41e0 2024-03-14T09:04:54.781 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 -- 172.21.15.154:0/3193725575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 msgr2=0x7f7d680c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 --2- 172.21.15.154:0/3193725575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 0x7f7d680c38f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009870 tx=0x7f7d58009b80 comp rx=0 tx=0).stop 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 -- 172.21.15.154:0/3193725575 shutdown_connections 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 --2- 172.21.15.154:0/3193725575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c41e0 0x7f7d680c38f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 -- 172.21.15.154:0/3193725575 >> 172.21.15.154:0/3193725575 conn(0x7f7d680a9d00 msgr2=0x7f7d680aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 -- 172.21.15.154:0/3193725575 shutdown_connections 2024-03-14T09:04:54.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.779+0000 7f7d6c9de700 1 -- 172.21.15.154:0/3193725575 wait complete. 2024-03-14T09:04:54.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.780+0000 7f7d6c9de700 1 Processor -- start 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.780+0000 7f7d6c9de700 1 -- start start 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d680c5e50 0x7f7d680c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c62b0 0x7f7d68142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 0x7f7d68142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68142b90 con 0x7f7d6814aa40 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6814ae90 con 0x7f7d680c5e50 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6c9de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681ee720 con 0x7f7d680c62b0 2024-03-14T09:04:54.786 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d66d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 0x7f7d68142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d66d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 0x7f7d68142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48034/0 (socket says 172.21.15.154:48034) 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d66d9d700 1 -- 172.21.15.154:0/1168960257 learned_addr learned my addr 172.21.15.154:0/1168960257 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d6659c700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d680c5e50 0x7f7d680c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.781+0000 7f7d65d9b700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c62b0 0x7f7d68142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 -- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c62b0 msgr2=0x7f7d68142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d680c62b0 0x7f7d68142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 -- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d680c5e50 msgr2=0x7f7d680c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d680c5e50 0x7f7d680c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580094a0 con 0x7f7d6814aa40 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d6659c700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d680c5e50 0x7f7d680c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d66d9d700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 0x7f7d68142650 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000cbc0 tx=0x7f7d6000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60003c00 con 0x7f7d6814aa40 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60003d60 con 0x7f7d6814aa40 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d681edba0 con 0x7f7d6814aa40 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000abb0 con 0x7f7d6814aa40 2024-03-14T09:04:54.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.782+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681ee160 con 0x7f7d6814aa40 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.783+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d6814ff10 con 0x7f7d6814aa40 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.783+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f7d6000ad10 con 0x7f7d6814aa40 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d5effd700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d4806a830 0x7f7d4806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7d60088b60 con 0x7f7d6814aa40 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d5effd700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f7d48070730 0x7f7d48072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d48073200 con 0x7f7d48070730 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6008d240 con 0x7f7d6814aa40 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d6659c700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d4806a830 0x7f7d4806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d65d9b700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f7d48070730 0x7f7d48072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d65d9b700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f7d48070730 0x7f7d48072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.784+0000 7f7d6659c700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d4806a830 0x7f7d4806ccf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d680c2680 tx=0x7f7d58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.785+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== osd.4 v2:172.21.15.191:6808/47548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7d48073200 con 0x7f7d48070730 2024-03-14T09:04:54.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.799+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 --> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7d680697d0 con 0x7f7d48070730 2024-03-14T09:04:54.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d5effd700 1 -- 172.21.15.154:0/1168960257 <== osd.4 v2:172.21.15.191:6808/47548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7d680697d0 con 0x7f7d48070730 2024-03-14T09:04:54.800 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:54.800 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:54.800 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:54.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f7d48070730 msgr2=0x7f7d48072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6808/47548,v1:172.21.15.191:6809/47548] conn(0x7f7d48070730 0x7f7d48072b50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d4806a830 msgr2=0x7f7d4806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f7d4806a830 0x7f7d4806ccf0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d680c2680 tx=0x7f7d58008030 comp rx=0 tx=0).stop 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 msgr2=0x7f7d68142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 --2- 172.21.15.154:0/1168960257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d6814aa40 0x7f7d68142650 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000cbc0 tx=0x7f7d6000cf80 comp rx=0 tx=0).stop 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d66d9d700 1 -- 172.21.15.154:0/1168960257 reap_dead start 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 shutdown_connections 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 >> 172.21.15.154:0/1168960257 conn(0x7f7d680a9d00 msgr2=0x7f7d680c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 shutdown_connections 2024-03-14T09:04:54.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.800+0000 7f7d6c9de700 1 -- 172.21.15.154:0/1168960257 wait complete. 2024-03-14T09:04:54.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-14T09:04:54.933 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 Processor -- start 2024-03-14T09:04:54.934 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 -- start start 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2200152f80 0x7f22000c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22000c2510 0x7f220015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2200070e10 con 0x7f220014c5c0 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22000c1bb0 con 0x7f2200152f80 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.934+0000 7f220742c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000c6170 con 0x7f22000c2510 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2200152f80 0x7f22000c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2200152f80 0x7f22000c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48042/0 (socket says 172.21.15.154:48042) 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 -- 172.21.15.154:0/1849324370 learned_addr learned my addr 172.21.15.154:0/1849324370 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22059c9700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22000c2510 0x7f220015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 -- 172.21.15.154:0/1849324370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22000c2510 msgr2=0x7f220015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22000c2510 0x7f220015ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 -- 172.21.15.154:0/1849324370 >> v1:172.21.15.191:6789/0 conn(0x7f220014c5c0 legacy=0x7f2200072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 -- 172.21.15.154:0/1849324370 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220014c9a0 con 0x7f2200152f80 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f22049c7700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000c1fd0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0098f0 tx=0x7f21fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e372d9f304be14af server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f21f7fff700 1 -- 172.21.15.154:0/1849324370 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004230 con 0x7f2200152f80 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f21f7fff700 1 -- 172.21.15.154:0/1849324370 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc004390 con 0x7f2200152f80 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.935+0000 7f21f7fff700 1 -- 172.21.15.154:0/1849324370 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0036a0 con 0x7f2200152f80 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 -- 172.21.15.154:0/1849324370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 msgr2=0x7f22000c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000c1fd0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0098f0 tx=0x7f21fc009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 -- 172.21.15.154:0/1849324370 shutdown_connections 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22000c2510 0x7f220015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 --2- 172.21.15.154:0/1849324370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000c1fd0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 -- 172.21.15.154:0/1849324370 >> 172.21.15.154:0/1849324370 conn(0x7f220006a770 msgr2=0x7f220006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:54.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 -- 172.21.15.154:0/1849324370 shutdown_connections 2024-03-14T09:04:54.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.936+0000 7f220742c700 1 -- 172.21.15.154:0/1849324370 wait complete. 2024-03-14T09:04:54.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.937+0000 7f220742c700 1 Processor -- start 2024-03-14T09:04:54.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.937+0000 7f220742c700 1 -- start start 2024-03-14T09:04:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22000c2510 0x7f2200154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f220014c5c0 0x7f22001ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220014d420 con 0x7f2200152f80 2024-03-14T09:04:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22001ea960 con 0x7f22000c2510 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220014d6c0 con 0x7f220014c5c0 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48050/0 (socket says 172.21.15.154:48050) 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 -- 172.21.15.154:0/2464751996 learned_addr learned my addr 172.21.15.154:0/2464751996 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22049c7700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f220014c5c0 0x7f22001ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22051c8700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22000c2510 0x7f2200154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 -- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f220014c5c0 msgr2=0x7f22001ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f220014c5c0 0x7f22001ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 -- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22000c2510 msgr2=0x7f2200154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22000c2510 0x7f2200154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 -- 172.21.15.154:0/2464751996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc009550 con 0x7f2200152f80 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f22059c9700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f21f8007e40 tx=0x7f21f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800ed30 con 0x7f2200152f80 2024-03-14T09:04:54.945 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8014420 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220014d9a0 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.938+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80147d0 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.939+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2200153c30 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.940+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22000a6020 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.940+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f21f8014930 con 0x7f2200152f80 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.940+0000 7f21f5ffb700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 0x7f21e406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:54.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.941+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f21f8089180 con 0x7f2200152f80 2024-03-14T09:04:54.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.941+0000 7f22051c8700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 0x7f21e406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:54.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.941+0000 7f22051c8700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 0x7f21e406ccf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21f0007fd0 tx=0x7f21f0007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:54.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:54.945+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f21f8054840 con 0x7f2200152f80 2024-03-14T09:04:55.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.136+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2200072d80 con 0x7f21e406a830 2024-03-14T09:04:55.136 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.137+0000 7f21f5ffb700 1 -- 172.21.15.154:0/2464751996 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f21f0003a90 con 0x7f21e406a830 2024-03-14T09:04:55.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 msgr2=0x7f21e406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 0x7f21e406ccf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21f0007fd0 tx=0x7f21f0007ec0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 msgr2=0x7f22000aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f21f8007e40 tx=0x7f21f800e3c0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.139 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 shutdown_connections 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f21e406a830 0x7f21e406ccf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2200152f80 0x7f22000aba80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22000c2510 0x7f2200154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 --2- 172.21.15.154:0/2464751996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f220014c5c0 0x7f22001ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 >> 172.21.15.154:0/2464751996 conn(0x7f220006a770 msgr2=0x7f220006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 shutdown_connections 2024-03-14T09:04:55.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.139+0000 7f220742c700 1 -- 172.21.15.154:0/2464751996 wait complete. 2024-03-14T09:04:55.141 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 4 to deep-scrub 2024-03-14T09:04:55.151 INFO:tasks.ceph:Scrubbing osd.5 2024-03-14T09:04:55.151 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 Processor -- start 2024-03-14T09:04:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 -- start start 2024-03-14T09:04:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19a006d920 0x7f19a00c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006b780 0x7f19a006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a0158520 con 0x7f19a01589b0 2024-03-14T09:04:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a01554f0 con 0x7f19a006d920 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.277+0000 7f19a646d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01557d0 con 0x7f19a006b780 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19a006d920 0x7f19a00c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19a006d920 0x7f19a00c1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48060/0 (socket says 172.21.15.154:48060) 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 -- 172.21.15.154:0/2261943847 learned_addr learned my addr 172.21.15.154:0/2261943847 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f19a4a0a700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006b780 0x7f19a006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 -- 172.21.15.154:0/2261943847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006b780 msgr2=0x7f19a006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006b780 0x7f19a006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 -- 172.21.15.154:0/2261943847 >> v1:172.21.15.191:6789/0 conn(0x7f19a01589b0 legacy=0x7f19a0152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 -- 172.21.15.154:0/2261943847 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0157ed0 con 0x7f19a006d920 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199f7fe700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006d920 0x7f19a00c1030 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f19940098f0 tx=0x7f1994009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e575a8a78956c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199effd700 1 -- 172.21.15.154:0/2261943847 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011830 con 0x7f19a006d920 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199effd700 1 -- 172.21.15.154:0/2261943847 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994011990 con 0x7f19a006d920 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.278+0000 7f199effd700 1 -- 172.21.15.154:0/2261943847 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011d20 con 0x7f19a006d920 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 -- 172.21.15.154:0/2261943847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006d920 msgr2=0x7f19a00c1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006d920 0x7f19a00c1030 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f19940098f0 tx=0x7f1994009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 -- 172.21.15.154:0/2261943847 shutdown_connections 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006b780 0x7f19a006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 --2- 172.21.15.154:0/2261943847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006d920 0x7f19a00c1030 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 -- 172.21.15.154:0/2261943847 >> 172.21.15.154:0/2261943847 conn(0x7f19a00c3130 msgr2=0x7f19a00c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 -- 172.21.15.154:0/2261943847 shutdown_connections 2024-03-14T09:04:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.279+0000 7f19a646d700 1 -- 172.21.15.154:0/2261943847 wait complete. 2024-03-14T09:04:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.280+0000 7f19a646d700 1 Processor -- start 2024-03-14T09:04:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.280+0000 7f19a646d700 1 -- start start 2024-03-14T09:04:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 0x7f19a014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006d920 0x7f19a01ed940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19a01589b0 0x7f19a0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00aca90 con 0x7f19a006b780 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00a9770 con 0x7f19a01589b0 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a646d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a014d590 con 0x7f19a006d920 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 0x7f19a014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 0x7f19a014d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48068/0 (socket says 172.21.15.154:48068) 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 -- 172.21.15.154:0/3840339213 learned_addr learned my addr 172.21.15.154:0/3840339213 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f19a4a0a700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19a01589b0 0x7f19a0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199f7fe700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006d920 0x7f19a01ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 -- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006d920 msgr2=0x7f19a01ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19a006d920 0x7f19a01ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 -- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19a01589b0 msgr2=0x7f19a0154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19a01589b0 0x7f19a0154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994009550 con 0x7f19a006b780 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.281+0000 7f199ffff700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 0x7f19a014d050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f199000c430 tx=0x7f199000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19900050d0 con 0x7f19a006b780 2024-03-14T09:04:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990005250 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990006860 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a00a9c00 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a01ee210 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.282+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19a006dd80 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.283+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f1990003680 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.283+0000 7f199cff9700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f198c06a830 0x7f198c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.283+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1990087c50 con 0x7f19a006b780 2024-03-14T09:04:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f199cff9700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7f198c070730 0x7f198c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f198c073200 con 0x7f198c070730 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199008c2a0 con 0x7f19a006b780 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f199f7fe700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f198c06a830 0x7f198c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f19a4a0a700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7f198c070730 0x7f198c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f199f7fe700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f198c06a830 0x7f198c06ccf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1994009c40 tx=0x7f1994008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.284+0000 7f19a4a0a700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7f198c070730 0x7f198c072b50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.285+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== osd.5 v2:172.21.15.191:6816/47551 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f198c073200 con 0x7f198c070730 2024-03-14T09:04:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.299+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 --> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f19a0158d90 con 0x7f198c070730 2024-03-14T09:04:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f199cff9700 1 -- 172.21.15.154:0/3840339213 <== osd.5 v2:172.21.15.191:6816/47551 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f199800f040 con 0x7f198c070730 2024-03-14T09:04:55.300 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:55.300 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:55.300 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7f198c070730 msgr2=0x7f198c072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6816/47551,v1:172.21.15.191:6818/47551] conn(0x7f198c070730 0x7f198c072b50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f198c06a830 msgr2=0x7f198c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f198c06a830 0x7f198c06ccf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1994009c40 tx=0x7f1994008030 comp rx=0 tx=0).stop 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 msgr2=0x7f19a014d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 --2- 172.21.15.154:0/3840339213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19a006b780 0x7f19a014d050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f199000c430 tx=0x7f199000c740 comp rx=0 tx=0).stop 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a4a0a700 1 -- 172.21.15.154:0/3840339213 reap_dead start 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 shutdown_connections 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 >> 172.21.15.154:0/3840339213 conn(0x7f19a00c3130 msgr2=0x7f19a00c0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 shutdown_connections 2024-03-14T09:04:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.300+0000 7f19a646d700 1 -- 172.21.15.154:0/3840339213 wait complete. 2024-03-14T09:04:55.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-14T09:04:55.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.434+0000 7fe68faab700 1 Processor -- start 2024-03-14T09:04:55.434 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.434+0000 7fe68faab700 1 -- start start 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68faab700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe68814c5c0 0x7fe688152f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68faab700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe688070e10 con 0x7fe688072290 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68faab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68814c9a0 con 0x7fe688072710 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68faab700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6880c1bb0 con 0x7fe68814c5c0 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68d046700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe688072290 0x7fe6880c1fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48150/0 (socket says 172.21.15.154:48150) 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68d046700 1 -- 172.21.15.154:0/3315313995 learned_addr learned my addr 172.21.15.154:0/3315313995 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:55.437 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68d847700 1 --2- 172.21.15.154:0/3315313995 >> v2:172.21.15.191:3300/0 conn(0x7fe68814c5c0 0x7fe688152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68c845700 1 -- 172.21.15.154:0/3315313995 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368498812 0 0) 0x7fe688070e10 con 0x7fe688072290 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.435+0000 7fe68c845700 1 -- 172.21.15.154:0/3315313995 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6740035b0 con 0x7fe688072290 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68d847700 1 -- 172.21.15.154:0/3315313995 >> v1:172.21.15.191:6789/0 conn(0x7fe688072710 legacy=0x7fe6880c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68d847700 1 -- 172.21.15.154:0/3315313995 >> v1:172.21.15.154:6789/0 conn(0x7fe688072290 legacy=0x7fe6880c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68d847700 1 -- 172.21.15.154:0/3315313995 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880c1580 con 0x7fe68814c5c0 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68d847700 1 --2- 172.21.15.154:0/3315313995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe688152f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe678009870 tx=0x7fe678009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a65f7886f2971d server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68c845700 1 -- 172.21.15.154:0/3315313995 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678011830 con 0x7fe68814c5c0 2024-03-14T09:04:55.438 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68c845700 1 -- 172.21.15.154:0/3315313995 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678011990 con 0x7fe68814c5c0 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.436+0000 7fe68c845700 1 -- 172.21.15.154:0/3315313995 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678011d00 con 0x7fe68814c5c0 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 -- 172.21.15.154:0/3315313995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 msgr2=0x7fe688152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 --2- 172.21.15.154:0/3315313995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe688152f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe678009870 tx=0x7fe678009b80 comp rx=0 tx=0).stop 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 -- 172.21.15.154:0/3315313995 shutdown_connections 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 --2- 172.21.15.154:0/3315313995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe688152f80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 -- 172.21.15.154:0/3315313995 >> 172.21.15.154:0/3315313995 conn(0x7fe68806a770 msgr2=0x7fe68806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 -- 172.21.15.154:0/3315313995 shutdown_connections 2024-03-14T09:04:55.439 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.437+0000 7fe68faab700 1 -- 172.21.15.154:0/3315313995 wait complete. 2024-03-14T09:04:55.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.438+0000 7fe68faab700 1 Processor -- start 2024-03-14T09:04:55.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.438+0000 7fe68faab700 1 -- start start 2024-03-14T09:04:55.442 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe688072290 0x7fe6880a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe68806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880ac210 con 0x7fe688072710 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880ac400 con 0x7fe68814c5c0 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880a97e0 con 0x7fe688072290 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48072/0 (socket says 172.21.15.154:48072) 2024-03-14T09:04:55.443 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 -- 172.21.15.154:0/1601542633 learned_addr learned my addr 172.21.15.154:0/1601542633 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 -- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe688072290 msgr2=0x7fe6880a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68e048700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe68806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d847700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe688072290 0x7fe6880a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe688072290 0x7fe6880a9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 -- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 msgr2=0x7fe68806d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe68806d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 -- 172.21.15.154:0/1601542633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780094a0 con 0x7fe688072710 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68d046700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c700 tx=0x7fe68400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.444 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684005d70 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6840070d0 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684007460 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6880ac6c0 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.439+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6880c0170 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.440+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6880697d0 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.441+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fe6840075c0 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.441+0000 7fe67e7fc700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 0x7fe66c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.441+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe6840880b0 con 0x7fe688072710 2024-03-14T09:04:55.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.441+0000 7fe68d847700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 0x7fe66c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.442+0000 7fe68d847700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 0x7fe66c06ccf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe678009840 tx=0x7fe67801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.446 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.445+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fe684053780 con 0x7fe688072710 2024-03-14T09:04:55.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.638+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe688074500 con 0x7fe66c06a830 2024-03-14T09:04:55.638 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.638+0000 7fe67e7fc700 1 -- 172.21.15.154:0/1601542633 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe688074500 con 0x7fe66c06a830 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.640+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 msgr2=0x7fe66c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.640+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 0x7fe66c06ccf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe678009840 tx=0x7fe67801c040 comp rx=0 tx=0).stop 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 msgr2=0x7fe6880bfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c700 tx=0x7fe68400cac0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 shutdown_connections 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68814c5c0 0x7fe68806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.641 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe688072290 0x7fe6880a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fe66c06a830 0x7fe66c06ccf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 --2- 172.21.15.154:0/1601542633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe688072710 0x7fe6880bfc30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 >> 172.21.15.154:0/1601542633 conn(0x7fe68806a770 msgr2=0x7fe688151650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 shutdown_connections 2024-03-14T09:04:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.641+0000 7fe68faab700 1 -- 172.21.15.154:0/1601542633 wait complete. 2024-03-14T09:04:55.643 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 5 to deep-scrub 2024-03-14T09:04:55.653 INFO:tasks.ceph:Scrubbing osd.6 2024-03-14T09:04:55.653 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:55.779 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.780+0000 7f62d3462700 1 Processor -- start 2024-03-14T09:04:55.780 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.780+0000 7f62d3462700 1 -- start start 2024-03-14T09:04:55.782 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d3462700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62cc0c1a40 0x7f62cc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d3462700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 0x7f62cc15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d3462700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc1532f0 con 0x7f62cc06acb0 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d3462700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc0c1710 con 0x7f62cc0c1a40 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d3462700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc1529c0 con 0x7f62cc156370 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62cc0c1a40 0x7f62cc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62cc0c1a40 0x7f62cc06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48086/0 (socket says 172.21.15.154:48086) 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 -- 172.21.15.154:0/2962915155 learned_addr learned my addr 172.21.15.154:0/2962915155 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.783 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d19ff700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 0x7f62cc15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 -- 172.21.15.154:0/2962915155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 msgr2=0x7f62cc15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 0x7f62cc15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 -- 172.21.15.154:0/2962915155 >> v1:172.21.15.191:6789/0 conn(0x7f62cc06acb0 legacy=0x7f62cc155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 -- 172.21.15.154:0/2962915155 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc152ca0 con 0x7f62cc0c1a40 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d19ff700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 0x7f62cc15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62d11fe700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc0c1a40 0x7f62cc06a770 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0098f0 tx=0x7f62bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd66ef73ac4c83e server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62c3fff700 1 -- 172.21.15.154:0/2962915155 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004230 con 0x7f62cc0c1a40 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62c3fff700 1 -- 172.21.15.154:0/2962915155 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc004390 con 0x7f62cc0c1a40 2024-03-14T09:04:55.784 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.781+0000 7f62c3fff700 1 -- 172.21.15.154:0/2962915155 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0036a0 con 0x7f62cc0c1a40 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 -- 172.21.15.154:0/2962915155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc0c1a40 msgr2=0x7f62cc06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc0c1a40 0x7f62cc06a770 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0098f0 tx=0x7f62bc009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 -- 172.21.15.154:0/2962915155 shutdown_connections 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc156370 0x7f62cc15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 --2- 172.21.15.154:0/2962915155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc0c1a40 0x7f62cc06a770 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 -- 172.21.15.154:0/2962915155 >> 172.21.15.154:0/2962915155 conn(0x7f62cc0c7040 msgr2=0x7f62cc0c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 -- 172.21.15.154:0/2962915155 shutdown_connections 2024-03-14T09:04:55.785 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.782+0000 7f62d3462700 1 -- 172.21.15.154:0/2962915155 wait complete. 2024-03-14T09:04:55.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.783+0000 7f62d3462700 1 Processor -- start 2024-03-14T09:04:55.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.783+0000 7f62d3462700 1 -- start start 2024-03-14T09:04:55.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc06acb0 0x7f62cc1eb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.789 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc0c1a40 0x7f62cc148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 0x7f62cc148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc13fbf0 con 0x7f62cc156370 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc1ebe70 con 0x7f62cc0c1a40 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d3462700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc13f910 con 0x7f62cc06acb0 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 0x7f62cc148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 0x7f62cc148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48096/0 (socket says 172.21.15.154:48096) 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 -- 172.21.15.154:0/1710878494 learned_addr learned my addr 172.21.15.154:0/1710878494 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d11fe700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc06acb0 0x7f62cc1eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d09fd700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc0c1a40 0x7f62cc148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 -- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc06acb0 msgr2=0x7f62cc1eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62cc06acb0 0x7f62cc1eb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 -- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc0c1a40 msgr2=0x7f62cc148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc0c1a40 0x7f62cc148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.784+0000 7f62d19ff700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009550 con 0x7f62cc156370 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62d19ff700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 0x7f62cc148570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cd80 tx=0x7f62c400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400f480 con 0x7f62cc156370 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400f600 con 0x7f62cc156370 2024-03-14T09:04:55.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc0c67c0 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400f9b0 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.785+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc0c6aa0 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.786+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62cc14ca60 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.786+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f62c400b040 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62c1ffb700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62b006a860 0x7f62b006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f62c4089180 con 0x7f62cc156370 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62c1ffb700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f62b0070760 0x7f62b0072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62b0073230 con 0x7f62b0070760 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c408b840 con 0x7f62cc156370 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62d11fe700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62b006a860 0x7f62b006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62d09fd700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f62b0070760 0x7f62b0072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62d11fe700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62b006a860 0x7f62b006cd20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009130 tx=0x7f62bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.787+0000 7f62d09fd700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f62b0070760 0x7f62b0072b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.788+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== osd.6 v2:172.21.15.191:6817/47571 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f62b0073230 con 0x7f62b0070760 2024-03-14T09:04:55.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.802+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 --> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f62cc152670 con 0x7f62b0070760 2024-03-14T09:04:55.803 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.803+0000 7f62c1ffb700 1 -- 172.21.15.154:0/1710878494 <== osd.6 v2:172.21.15.191:6817/47571 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f62c8003b50 con 0x7f62b0070760 2024-03-14T09:04:55.803 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:55.803 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:55.803 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:55.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.803+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f62b0070760 msgr2=0x7f62b0072b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.804 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.803+0000 7f62d3462700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6817/47571,v1:172.21.15.191:6819/47571] conn(0x7f62b0070760 0x7f62b0072b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62b006a860 msgr2=0x7f62b006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f62b006a860 0x7f62b006cd20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009130 tx=0x7f62bc008030 comp rx=0 tx=0).stop 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 msgr2=0x7f62cc148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 --2- 172.21.15.154:0/1710878494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62cc156370 0x7f62cc148570 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f62c400cd80 tx=0x7f62c400a560 comp rx=0 tx=0).stop 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d19ff700 1 -- 172.21.15.154:0/1710878494 reap_dead start 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 shutdown_connections 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 >> 172.21.15.154:0/1710878494 conn(0x7f62cc0c7040 msgr2=0x7f62cc151540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 shutdown_connections 2024-03-14T09:04:55.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.804+0000 7f62d3462700 1 -- 172.21.15.154:0/1710878494 wait complete. 2024-03-14T09:04:55.815 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-14T09:04:55.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.937+0000 7f25f32d0700 1 Processor -- start 2024-03-14T09:04:55.937 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.937+0000 7f25f32d0700 1 -- start start 2024-03-14T09:04:55.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25ec14eaf0 0x7f25ec06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 0x7f25ec155df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.940 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec06b310 con 0x7f25ec14eaf0 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec0ac8e0 con 0x7f25ec06acb0 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f32d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec06de30 con 0x7f25ec156330 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25ec14eaf0 0x7f25ec06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25ec14eaf0 0x7f25ec06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48112/0 (socket says 172.21.15.154:48112) 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 -- 172.21.15.154:0/4135270878 learned_addr learned my addr 172.21.15.154:0/4135270878 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f086b700 1 --2- 172.21.15.154:0/4135270878 >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 0x7f25ec155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f186d700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 -- 172.21.15.154:0/4135270878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 msgr2=0x7f25ec157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:55.941 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 -- 172.21.15.154:0/4135270878 >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 msgr2=0x7f25ec155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 --2- 172.21.15.154:0/4135270878 >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 0x7f25ec155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f106c700 1 -- 172.21.15.154:0/4135270878 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec0acb30 con 0x7f25ec14eaf0 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.938+0000 7f25f086b700 1 --2- 172.21.15.154:0/4135270878 >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 0x7f25ec155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f106c700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec14eaf0 0x7f25ec06a770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f25ec14f1c0 tx=0x7f25dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcbadc00ace484 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25e3fff700 1 -- 172.21.15.154:0/4135270878 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00b870 con 0x7f25ec14eaf0 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25e3fff700 1 -- 172.21.15.154:0/4135270878 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc00b9d0 con 0x7f25ec14eaf0 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25e3fff700 1 -- 172.21.15.154:0/4135270878 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00bd60 con 0x7f25ec14eaf0 2024-03-14T09:04:55.942 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 -- 172.21.15.154:0/4135270878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec14eaf0 msgr2=0x7f25ec06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec14eaf0 0x7f25ec06a770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f25ec14f1c0 tx=0x7f25dc009bd0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 -- 172.21.15.154:0/4135270878 shutdown_connections 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 --2- 172.21.15.154:0/4135270878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec14eaf0 0x7f25ec06a770 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 --2- 172.21.15.154:0/4135270878 >> v2:172.21.15.191:3300/0 conn(0x7f25ec06acb0 0x7f25ec155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 -- 172.21.15.154:0/4135270878 >> 172.21.15.154:0/4135270878 conn(0x7f25ec0c3880 msgr2=0x7f25ec0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 -- 172.21.15.154:0/4135270878 shutdown_connections 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.939+0000 7f25f32d0700 1 -- 172.21.15.154:0/4135270878 wait complete. 2024-03-14T09:04:55.943 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.940+0000 7f25f32d0700 1 Processor -- start 2024-03-14T09:04:55.944 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 -- start start 2024-03-14T09:04:55.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ec14eaf0 0x7f25ec1ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec0c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec151510 con 0x7f25ec06acb0 2024-03-14T09:04:55.946 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec1ee7a0 con 0x7f25ec14eaf0 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f32d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec1ea6f0 con 0x7f25ec156330 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f106c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f106c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48114/0 (socket says 172.21.15.154:48114) 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.941+0000 7f25f106c700 1 -- 172.21.15.154:0/2244641804 learned_addr learned my addr 172.21.15.154:0/2244641804 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 -- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 msgr2=0x7f25ec0c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec0c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 -- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ec14eaf0 msgr2=0x7f25ec1ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ec14eaf0 0x7f25ec1ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:55.947 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 -- 172.21.15.154:0/2244641804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc009550 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f106c700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc004540 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25dc0036a0 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec153e40 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc003a50 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.942+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec154400 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.943+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec1eb250 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.943+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f25dc021070 con 0x7f25ec06acb0 2024-03-14T09:04:55.948 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.944+0000 7f25e1ffb700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 0x7f25d006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:55.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.944+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f25dc08a150 con 0x7f25ec06acb0 2024-03-14T09:04:55.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.944+0000 7f25f086b700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 0x7f25d006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:55.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.944+0000 7f25f086b700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 0x7f25d006ccf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25e8006fd0 tx=0x7f25e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:55.949 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:55.948+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f25dc0559c0 con 0x7f25ec06acb0 2024-03-14T09:04:56.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.141+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f25ec06e080 con 0x7f25d006a830 2024-03-14T09:04:56.142 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.142+0000 7f25e1ffb700 1 -- 172.21.15.154:0/2244641804 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f25e8004030 con 0x7f25d006a830 2024-03-14T09:04:56.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 msgr2=0x7f25d006ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 0x7f25d006ccf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25e8006fd0 tx=0x7f25e8008040 comp rx=0 tx=0).stop 2024-03-14T09:04:56.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 msgr2=0x7f25ec14f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc004750 comp rx=0 tx=0).stop 2024-03-14T09:04:56.147 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 shutdown_connections 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25ec156330 0x7f25ec0c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f25d006a830 0x7f25d006ccf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25ec06acb0 0x7f25ec14f490 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 --2- 172.21.15.154:0/2244641804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25ec14eaf0 0x7f25ec1ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 >> 172.21.15.154:0/2244641804 conn(0x7f25ec0c3880 msgr2=0x7f25ec0c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 shutdown_connections 2024-03-14T09:04:56.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.144+0000 7f25f32d0700 1 -- 172.21.15.154:0/2244641804 wait complete. 2024-03-14T09:04:56.149 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 6 to deep-scrub 2024-03-14T09:04:56.156 INFO:tasks.ceph:Scrubbing osd.7 2024-03-14T09:04:56.157 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-14T09:04:56.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.282+0000 7f6c608b2700 1 Processor -- start 2024-03-14T09:04:56.282 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.282+0000 7f6c608b2700 1 -- start start 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.282+0000 7f6c608b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c608b2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6c5c0c2e90 0x7f6c5c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c608b2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c0c2670 con 0x7f6c5c0c37f0 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c608b2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c0c3d10 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c608b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c1580a0 con 0x7f6c5c0c3410 2024-03-14T09:04:56.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c59d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6c5c0c37f0 0x7f6c5c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48162/0 (socket says 172.21.15.154:48162) 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c59d9b700 1 -- 172.21.15.154:0/3091252001 learned_addr learned my addr 172.21.15.154:0/3091252001 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/3091252001 >> v2:172.21.15.191:3300/0 conn(0x7f6c5c0c2e90 0x7f6c5c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5a59c700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5959a700 1 -- 172.21.15.154:0/3091252001 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785544407 0 0) 0x7f6c5c0c2670 con 0x7f6c5c0c37f0 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5959a700 1 -- 172.21.15.154:0/3091252001 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c480035b0 con 0x7f6c5c0c37f0 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 -- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 msgr2=0x7f6c5c0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 -- 172.21.15.154:0/3091252001 >> v1:172.21.15.154:6789/0 conn(0x7f6c5c0c37f0 legacy=0x7f6c5c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 -- 172.21.15.154:0/3091252001 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c158380 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5a59c700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:56.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.283+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c15f150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009d90 tx=0x7f6c5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5b0bb969530783 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c5959a700 1 -- 172.21.15.154:0/3091252001 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400e070 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.287 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c5959a700 1 -- 172.21.15.154:0/3091252001 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5400bb20 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c5959a700 1 -- 172.21.15.154:0/3091252001 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400be90 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 -- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c2e90 msgr2=0x7f6c5c15f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c15f150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009d90 tx=0x7f6c5400b330 comp rx=0 tx=0).stop 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 -- 172.21.15.154:0/3091252001 shutdown_connections 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c15f150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 --2- 172.21.15.154:0/3091252001 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 -- 172.21.15.154:0/3091252001 >> 172.21.15.154:0/3091252001 conn(0x7f6c5c0a9d00 msgr2=0x7f6c5c0aa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 -- 172.21.15.154:0/3091252001 shutdown_connections 2024-03-14T09:04:56.289 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.284+0000 7f6c608b2700 1 -- 172.21.15.154:0/3091252001 wait complete. 2024-03-14T09:04:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 Processor -- start 2024-03-14T09:04:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 -- start start 2024-03-14T09:04:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c37f0 0x7f6c5c1eb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06f4c0 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06e5d0 con 0x7f6c5c0c37f0 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c608b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06e8b0 con 0x7f6c5c0c3410 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c5a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c5a59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48128/0 (socket says 172.21.15.154:48128) 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.286+0000 7f6c5a59c700 1 -- 172.21.15.154:0/4058717548 learned_addr learned my addr 172.21.15.154:0/4058717548 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c37f0 0x7f6c5c1eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.294 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 -- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 msgr2=0x7f6c5c0c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c5c0c3410 0x7f6c5c0c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 -- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c37f0 msgr2=0x7f6c5c1eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c5c0c37f0 0x7f6c5c1eb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440094a0 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c5a59c700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c144030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400ebe0 tx=0x7f6c44009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400c910 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4400ca70 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c54009a70 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.295 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400cde0 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.287+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c158d00 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.288+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c5c158b80 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.288+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f6c44013470 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c52ffd700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c3c06a830 0x7f6c3c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6c44088d10 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.296 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c59d9b700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c3c06a830 0x7f6c3c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c52ffd700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7f6c3c070730 0x7f6c3c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.297 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c3c073200 con 0x7f6c3c070730 2024-03-14T09:04:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4408b2f0 con 0x7f6c5c0c2e90 2024-03-14T09:04:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7f6c3c070730 0x7f6c3c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.289+0000 7f6c59d9b700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c3c06a830 0x7f6c3c06ccf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007fd0 tx=0x7f6c4c007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.290+0000 7f6c5ad9d700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7f6c3c070730 0x7f6c3c072b50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.290+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== osd.7 v2:172.21.15.191:6800/47550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6c3c073200 con 0x7f6c3c070730 2024-03-14T09:04:56.304 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.304+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 --> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6c5c157d70 con 0x7f6c3c070730 2024-03-14T09:04:56.305 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.305+0000 7f6c52ffd700 1 -- 172.21.15.154:0/4058717548 <== osd.7 v2:172.21.15.191:6800/47550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6c54003a20 con 0x7f6c3c070730 2024-03-14T09:04:56.305 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-03-14T09:04:56.305 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-14T09:04:56.305 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-03-14T09:04:56.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7f6c3c070730 msgr2=0x7f6c3c072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.306 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6800/47550,v1:172.21.15.191:6801/47550] conn(0x7f6c3c070730 0x7f6c3c072b50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c3c06a830 msgr2=0x7f6c3c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f6c3c06a830 0x7f6c3c06ccf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007fd0 tx=0x7f6c4c007980 comp rx=0 tx=0).stop 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 msgr2=0x7f6c5c144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 --2- 172.21.15.154:0/4058717548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c5c0c2e90 0x7f6c5c144030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400ebe0 tx=0x7f6c44009ec0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c5ad9d700 1 -- 172.21.15.154:0/4058717548 reap_dead start 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 shutdown_connections 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 >> 172.21.15.154:0/4058717548 conn(0x7f6c5c0a9d00 msgr2=0x7f6c5c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 shutdown_connections 2024-03-14T09:04:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.306+0000 7f6c608b2700 1 -- 172.21.15.154:0/4058717548 wait complete. 2024-03-14T09:04:56.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-14T09:04:56.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:04:56.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:04:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.445+0000 7f996ffe4700 1 Processor -- start 2024-03-14T09:04:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 -- start start 2024-03-14T09:04:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99680702f0 0x7f99680c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99680c5d60 0x7f99680c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680acd10 con 0x7f996814fbd0 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99680ac7b0 con 0x7f99680702f0 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996ffe4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9968150050 con 0x7f99680c5d60 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.446+0000 7f996dd80700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99680702f0 0x7f99680c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.449 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99680702f0 0x7f99680c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48132/0 (socket says 172.21.15.154:48132) 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 -- 172.21.15.154:0/1584412732 learned_addr learned my addr 172.21.15.154:0/1584412732 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996e581700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99680c5d60 0x7f99680c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 -- 172.21.15.154:0/1584412732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99680c5d60 msgr2=0x7f99680c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99680c5d60 0x7f99680c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 -- 172.21.15.154:0/1584412732 >> v1:172.21.15.191:6789/0 conn(0x7f996814fbd0 legacy=0x7f99680c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 -- 172.21.15.154:0/1584412732 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680ac9c0 con 0x7f99680702f0 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996dd80700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680702f0 0x7f99680c4380 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f995c0098f0 tx=0x7f995c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94b8259938a3537 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.450 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996cd7e700 1 -- 172.21.15.154:0/1584412732 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00b8a0 con 0x7f99680702f0 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996cd7e700 1 -- 172.21.15.154:0/1584412732 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00ba00 con 0x7f99680702f0 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.447+0000 7f996cd7e700 1 -- 172.21.15.154:0/1584412732 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c00bd90 con 0x7f99680702f0 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 -- 172.21.15.154:0/1584412732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680702f0 msgr2=0x7f99680c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680702f0 0x7f99680c4380 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f995c0098f0 tx=0x7f995c009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 -- 172.21.15.154:0/1584412732 shutdown_connections 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99680c5d60 0x7f99680c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.451 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 --2- 172.21.15.154:0/1584412732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680702f0 0x7f99680c4380 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 -- 172.21.15.154:0/1584412732 >> 172.21.15.154:0/1584412732 conn(0x7f99680a9bf0 msgr2=0x7f99680a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 -- 172.21.15.154:0/1584412732 shutdown_connections 2024-03-14T09:04:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.448+0000 7f996ffe4700 1 -- 172.21.15.154:0/1584412732 wait complete. 2024-03-14T09:04:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.449+0000 7f996ffe4700 1 Processor -- start 2024-03-14T09:04:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.449+0000 7f996ffe4700 1 -- start start 2024-03-14T09:04:56.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680702f0 0x7f996814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.455 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f996814fbd0 0x7f9968144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681533f0 con 0x7f99680c5d60 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9968155ed0 con 0x7f99680702f0 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9968155b80 con 0x7f996814fbd0 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48140/0 (socket says 172.21.15.154:48140) 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 -- 172.21.15.154:0/3357667094 learned_addr learned my addr 172.21.15.154:0/3357667094 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:56.456 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996dd80700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680702f0 0x7f996814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996e581700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f996814fbd0 0x7f9968144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 -- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f996814fbd0 msgr2=0x7f9968144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f996814fbd0 0x7f9968144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 -- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680702f0 msgr2=0x7f996814b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680702f0 0x7f996814b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 -- 172.21.15.154:0/3357667094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c009550 con 0x7f99680c5d60 2024-03-14T09:04:56.457 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996d57f700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f996400c4e0 tx=0x7f996400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640052d0 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964005430 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99640038c0 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.450+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9968153e50 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.451+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9968149e70 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.451+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9968154240 con 0x7f99680c5d60 2024-03-14T09:04:56.458 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.452+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f9964003a20 con 0x7f99680c5d60 2024-03-14T09:04:56.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.452+0000 7f995a7fc700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 0x7f994c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.452+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9964087d80 con 0x7f99680c5d60 2024-03-14T09:04:56.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.452+0000 7f996dd80700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 0x7f994c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.453+0000 7f996dd80700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 0x7f994c06ccf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f995c014040 tx=0x7f995c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.457+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f99640534d0 con 0x7f99680c5d60 2024-03-14T09:04:56.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.651+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f9968070ad0 con 0x7f994c06a830 2024-03-14T09:04:56.651 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.651+0000 7f995a7fc700 1 -- 172.21.15.154:0/3357667094 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f995c00bb70 con 0x7f994c06a830 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 msgr2=0x7f994c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 0x7f994c06ccf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f995c014040 tx=0x7f995c008030 comp rx=0 tx=0).stop 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 msgr2=0x7f99681562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f996400c4e0 tx=0x7f996400c7f0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 shutdown_connections 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f996814fbd0 0x7f9968144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.654 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99680702f0 0x7f996814b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f994c06a830 0x7f994c06ccf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 --2- 172.21.15.154:0/3357667094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99680c5d60 0x7f99681562d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 >> 172.21.15.154:0/3357667094 conn(0x7f99680a9bf0 msgr2=0x7f996814d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 shutdown_connections 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.654+0000 7f996ffe4700 1 -- 172.21.15.154:0/3357667094 wait complete. 2024-03-14T09:04:56.655 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 7 to deep-scrub 2024-03-14T09:04:56.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:04:56.787 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.787+0000 7ff2e7f67700 1 Processor -- start 2024-03-14T09:04:56.788 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 -- start start 2024-03-14T09:04:56.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 0x7ff2e00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e00acd10 con 0x7ff2e014fbd0 2024-03-14T09:04:56.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00ac7b0 con 0x7ff2e00c5d60 2024-03-14T09:04:56.790 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.788+0000 7ff2e7f67700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0150050 con 0x7ff2e00702f0 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e6504700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 0x7ff2e00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.154:41732/0 (socket says 172.21.15.154:41732) 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 -- 172.21.15.154:0/3716702470 learned_addr learned my addr 172.21.15.154:0/3716702470 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 -- 172.21.15.154:0/3716702470 >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 msgr2=0x7ff2e00c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 --2- 172.21.15.154:0/3716702470 >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 0x7ff2e00c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 -- 172.21.15.154:0/3716702470 >> v1:172.21.15.191:6789/0 conn(0x7ff2e014fbd0 legacy=0x7ff2e00c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 -- 172.21.15.154:0/3716702470 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00ac9c0 con 0x7ff2e00702f0 2024-03-14T09:04:56.791 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e6504700 1 --2- 172.21.15.154:0/3716702470 >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 0x7ff2e00c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e5d03700 1 --2- 172.21.15.154:0/3716702470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40098f0 tx=0x7ff2d4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d962b9de5bad5ac server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.789+0000 7ff2e4d01700 1 -- 172.21.15.154:0/3716702470 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011830 con 0x7ff2e00702f0 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e4d01700 1 -- 172.21.15.154:0/3716702470 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4011990 con 0x7ff2e00702f0 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e4d01700 1 -- 172.21.15.154:0/3716702470 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4011d20 con 0x7ff2e00702f0 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 -- 172.21.15.154:0/3716702470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 msgr2=0x7ff2e00c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/3716702470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40098f0 tx=0x7ff2d4009c00 comp rx=0 tx=0).stop 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 -- 172.21.15.154:0/3716702470 shutdown_connections 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/3716702470 >> v2:172.21.15.191:3300/0 conn(0x7ff2e00c5d60 0x7ff2e00c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.792 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/3716702470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e00702f0 0x7ff2e00c4380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 -- 172.21.15.154:0/3716702470 >> 172.21.15.154:0/3716702470 conn(0x7ff2e00a9bf0 msgr2=0x7ff2e00a9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:56.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 -- 172.21.15.154:0/3716702470 shutdown_connections 2024-03-14T09:04:56.793 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.790+0000 7ff2e7f67700 1 -- 172.21.15.154:0/3716702470 wait complete. 2024-03-14T09:04:56.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.791+0000 7ff2e7f67700 1 Processor -- start 2024-03-14T09:04:56.795 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.791+0000 7ff2e7f67700 1 -- start start 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 0x7ff2e0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e014fbd0 0x7ff2e0152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e014a280 con 0x7ff2e00702f0 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0152080 con 0x7ff2e00c5d60 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e7f67700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e014aa90 con 0x7ff2e014fbd0 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48156/0 (socket says 172.21.15.154:48156) 2024-03-14T09:04:56.796 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 -- 172.21.15.154:0/553493870 learned_addr learned my addr 172.21.15.154:0/553493870 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e6504700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e014fbd0 0x7ff2e0152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5502700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 0x7ff2e0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 -- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e014fbd0 msgr2=0x7ff2e0152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e014fbd0 0x7ff2e0152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 -- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 msgr2=0x7ff2e0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 0x7ff2e0152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5d03700 1 -- 172.21.15.154:0/553493870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d8009a70 con 0x7ff2e00702f0 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.792+0000 7ff2e5502700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 0x7ff2e0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:04:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2e5d03700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009c40 tx=0x7ff2d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004530 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d401f070 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4009550 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40038c0 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.793+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e014a4d0 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.794+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e00697d0 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.794+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7ff2d4021030 con 0x7ff2e00702f0 2024-03-14T09:04:56.798 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.795+0000 7ff2d27fc700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 0x7ff2c406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:04:56.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.795+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff2d4089ee0 con 0x7ff2e00702f0 2024-03-14T09:04:56.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.795+0000 7ff2e5502700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 0x7ff2c406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:04:56.799 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.795+0000 7ff2e5502700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 0x7ff2c406ccf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00c4000 tx=0x7ff2dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:04:56.802 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.799+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7ff2d4055630 con 0x7ff2e00702f0 2024-03-14T09:04:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.989+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2e0070ad0 con 0x7ff2c406a830 2024-03-14T09:04:56.992 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.992+0000 7ff2d27fc700 1 -- 172.21.15.154:0/553493870 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50622 (secure 0 0 0) 0x7ff2dc014420 con 0x7ff2c406a830 2024-03-14T09:04:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:04:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 msgr2=0x7ff2c406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 0x7ff2c406ccf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00c4000 tx=0x7ff2dc009040 comp rx=0 tx=0).stop 2024-03-14T09:04:56.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 msgr2=0x7ff2e01453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:04:56.998 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009c40 tx=0x7ff2d4004750 comp rx=0 tx=0).stop 2024-03-14T09:04:56.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 shutdown_connections 2024-03-14T09:04:56.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2e014fbd0 0x7ff2e0152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:56.999 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7ff2c406a830 0x7ff2c406ccf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:57.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2e00702f0 0x7ff2e01453d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:57.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 --2- 172.21.15.154:0/553493870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2e00c5d60 0x7ff2e0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:04:57.000 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 >> 172.21.15.154:0/553493870 conn(0x7ff2e00a9bf0 msgr2=0x7ff2e00c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:04:57.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 shutdown_connections 2024-03-14T09:04:57.001 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:04:56.994+0000 7ff2e7f67700 1 -- 172.21.15.154:0/553493870 wait complete. 2024-03-14T09:04:57.001 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:04:57.006 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-03-14T09:04:55.569371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225160,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749560,"statfs":{"total":773094113280,"available":772863549440,"internally_reserved":0,"allocated":9183232,"data_stored":4719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-137,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.057058"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-14T09:04:48.204970+0000","last_change":"2024-03-14T09:01:15.163857+0000","last_active":"2024-03-14T09:04:48.204970+0000","last_peered":"2024-03-14T09:04:48.204970+0000","last_clean":"2024-03-14T09:04:48.204970+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:04:48.204970+0000","last_undegraded":"2024-03-14T09:04:48.204970+0000","last_fullsized":"2024-03-14T09:04:48.204970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-14T09:01:13.676076+0000","last_clean_scrub_stamp":"2024-03-14T09:01:13.676076+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T19:17:46.304351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343260,"statfs":{"total":96636764160,"available":96607498240,"internally_reserved":0,"allocated":1593344,"data_stored":1032712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.77800000000000002,"15min":0.88700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":6.5,"15min":6.5},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67000000000000004,"5min":0.878,"15min":0.878},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71999999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.68500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.372,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.41199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.87,"15min":0.87},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67600000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.66300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66600000000000004,"5min":0.748,"15min":0.748},"last":0.64200000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.503},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.87,"15min":0.87},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.254},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.43099999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.432,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46100000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.39100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73399999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343260,"statfs":{"total":96636764160,"available":96607498240,"internally_reserved":0,"allocated":1593344,"data_stored":1032712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.62,"15min":0.75},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.872,"5min":7.383,"15min":7.383},"last":0.375},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.45400000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85799999999999998,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.68400000000000005,"15min":0.82999999999999996},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.68899999999999995,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.70799999999999996,"15min":0.85599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60499999999999998,"5min":7.944,"15min":7.944},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35199999999999998,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47999999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.67400000000000004,"15min":0.80300000000000005},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83299999999999996,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.85799999999999998,"15min":0.97399999999999998},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.363,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.88200000000000001,"15min":1.0149999999999999},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4810000000000001,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.627},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.68200000000000005}]},{"osd":7,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.628},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.61599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.34100000000000003},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.41399999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.22},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.41299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":1.111,"15min":1.111},"last":0.34799999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.47599999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51300000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.317}]},{"osd":3,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.58899999999999997,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.44900000000000001,"5min":7.5,"15min":7.5},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.63700000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54100000000000004,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83399999999999996,"5min":1.151,"15min":1.151},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79700000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":0.752},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86099999999999999,"5min":1.179,"15min":1.179},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.629},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.435,"15min":0.435},"max":{"1min":0.73299999999999998,"5min":1.165,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.316,"15min":0.31},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.373,"15min":0.375},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.31},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.379,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.48099999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.753,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80500000000000005,"5min":1.087,"15min":1.087},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.755,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.2549999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.81200000000000006,"15min":0.92000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78800000000000003,"5min":7.702,"15min":7.702},"last":0.443}]},{"osd":5,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.503}]},{"osd":7,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.308},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.376},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.438},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:04:57.010 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-14T09:01:13.676076+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=9, tm_min=1, tm_sec=13, tm_wday=3, tm_yday=74, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=14, tm_hour=9, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=74, tm_isdst=0) 2024-03-14T09:04:57.010 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-14T09:05:17.011 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-14T09:05:17.148 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.148+0000 7fc64cd8e700 1 Processor -- start 2024-03-14T09:05:17.149 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.148+0000 7fc64cd8e700 1 -- start start 2024-03-14T09:05:17.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64cd8e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc64814eaf0 0x7fc64806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.151 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64cd8e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc648156500 0x7fc64815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64cd8e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480ac8e0 con 0x7fc64806acb0 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64cd8e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64806b310 con 0x7fc64814eaf0 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64cd8e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64806de30 con 0x7fc648156500 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc64814eaf0 0x7fc64806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64659c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc64814eaf0 0x7fc64806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55736/0 (socket says 172.21.15.154:55736) 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.149+0000 7fc64659c700 1 -- 172.21.15.154:0/2961899617 learned_addr learned my addr 172.21.15.154:0/2961899617 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:05:17.152 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc646d9d700 1 --2- 172.21.15.154:0/2961899617 >> v2:172.21.15.191:3300/0 conn(0x7fc648156500 0x7fc64815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64659c700 1 -- 172.21.15.154:0/2961899617 >> v2:172.21.15.191:3300/0 conn(0x7fc648156500 msgr2=0x7fc64815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64659c700 1 --2- 172.21.15.154:0/2961899617 >> v2:172.21.15.191:3300/0 conn(0x7fc648156500 0x7fc64815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64659c700 1 -- 172.21.15.154:0/2961899617 >> v1:172.21.15.191:6789/0 conn(0x7fc64806acb0 legacy=0x7fc648155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64659c700 1 -- 172.21.15.154:0/2961899617 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648156910 con 0x7fc64814eaf0 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64659c700 1 --2- 172.21.15.154:0/2961899617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64814eaf0 0x7fc64806a770 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc6480c4920 tx=0x7fc630009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48930bdbb99410fd server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64559a700 1 -- 172.21.15.154:0/2961899617 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630004230 con 0x7fc64814eaf0 2024-03-14T09:05:17.153 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64559a700 1 -- 172.21.15.154:0/2961899617 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc630004390 con 0x7fc64814eaf0 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.150+0000 7fc64559a700 1 -- 172.21.15.154:0/2961899617 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6300036a0 con 0x7fc64814eaf0 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2961899617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64814eaf0 msgr2=0x7fc64806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2961899617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64814eaf0 0x7fc64806a770 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc6480c4920 tx=0x7fc630009bd0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2961899617 shutdown_connections 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2961899617 >> v2:172.21.15.191:3300/0 conn(0x7fc648156500 0x7fc64815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2961899617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64814eaf0 0x7fc64806a770 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.154 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2961899617 >> 172.21.15.154:0/2961899617 conn(0x7fc6480c3880 msgr2=0x7fc6480c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:17.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2961899617 shutdown_connections 2024-03-14T09:05:17.155 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.151+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2961899617 wait complete. 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.152+0000 7fc64cd8e700 1 Processor -- start 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.152+0000 7fc64cd8e700 1 -- start start 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64806acb0 0x7fc6481e0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc64814eaf0 0x7fc6480c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648149dc0 con 0x7fc648156500 2024-03-14T09:05:17.158 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481e0e50 con 0x7fc64814eaf0 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc648149800 con 0x7fc64806acb0 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55738/0 (socket says 172.21.15.154:55738) 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 -- 172.21.15.154:0/2985451593 learned_addr learned my addr 172.21.15.154:0/2985451593 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc645d9b700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc64814eaf0 0x7fc6480c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64659c700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64806acb0 0x7fc6481e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.159 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 -- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64806acb0 msgr2=0x7fc6481e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64806acb0 0x7fc6481e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 -- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc64814eaf0 msgr2=0x7fc6480c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc64814eaf0 0x7fc6480c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 -- 172.21.15.154:0/2985451593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc630009550 con 0x7fc648156500 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc646d9d700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc640007e60 tx=0x7fc64000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000ecd0 con 0x7fc648156500 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000ee50 con 0x7fc648156500 2024-03-14T09:05:17.160 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc648149ae0 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.153+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640014660 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.154+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6480c2c70 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.154+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6481e0250 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.155+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7fc6400147c0 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.155+0000 7fc63effd700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 0x7fc62806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.155+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc640089120 con 0x7fc648156500 2024-03-14T09:05:17.161 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.156+0000 7fc64659c700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 0x7fc62806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.156+0000 7fc64659c700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 0x7fc62806ccf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc630009130 tx=0x7fc630008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.162 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.160+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7fc6481e0250 con 0x7fc648156500 2024-03-14T09:05:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.350+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 --> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc648155700 con 0x7fc62806a830 2024-03-14T09:05:17.352 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.352+0000 7fc63effd700 1 -- 172.21.15.154:0/2985451593 <== mgr.4100 v2:172.21.15.191:6832/46876 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50617 (secure 0 0 0) 0x7fc630004130 con 0x7fc62806a830 2024-03-14T09:05:17.353 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:05:17.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 msgr2=0x7fc62806ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.355 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 0x7fc62806ccf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc630009130 tx=0x7fc630008030 comp rx=0 tx=0).stop 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 msgr2=0x7fc6481ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc640007e60 tx=0x7fc64000e3c0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 shutdown_connections 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7fc62806a830 0x7fc62806ccf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc648156500 0x7fc6481ed6a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.356 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc64806acb0 0x7fc6481e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 --2- 172.21.15.154:0/2985451593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc64814eaf0 0x7fc6480c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 >> 172.21.15.154:0/2985451593 conn(0x7fc6480c3880 msgr2=0x7fc648155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 shutdown_connections 2024-03-14T09:05:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.354+0000 7fc64cd8e700 1 -- 172.21.15.154:0/2985451593 wait complete. 2024-03-14T09:05:17.357 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-03-14T09:05:17.366 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-14T09:05:15.572755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225144,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749576,"statfs":{"total":773094113280,"available":772863565824,"internally_reserved":0,"allocated":9183232,"data_stored":4719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001964"},"pg_stats":[{"pgid":"1.0","version":"80'119","reported_seq":232,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-14T09:05:06.597221+0000","last_change":"2024-03-14T09:05:06.597344+0000","last_active":"2024-03-14T09:05:06.597221+0000","last_peered":"2024-03-14T09:05:06.597221+0000","last_clean":"2024-03-14T09:05:06.597221+0000","last_became_active":"2024-03-14T09:01:15.162708+0000","last_became_peered":"2024-03-14T09:01:15.162708+0000","last_unstale":"2024-03-14T09:05:06.597221+0000","last_undegraded":"2024-03-14T09:05:06.597221+0000","last_fullsized":"2024-03-14T09:05:06.597221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"80'119","last_scrub_stamp":"2024-03-14T09:05:06.597191+0000","last_deep_scrub":"80'119","last_deep_scrub_stamp":"2024-03-14T09:05:06.597191+0000","last_clean_scrub_stamp":"2024-03-14T09:05:06.597191+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-15T18:14:10.987474+0000","scrub_duration":0.0081308860000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1593344,"data_stored":1032712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.77800000000000002,"15min":0.88700000000000001},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.72099999999999997,"5min":6.5,"15min":6.5},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.64800000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52600000000000002,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.39500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.66800000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.45600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67000000000000004,"5min":0.878,"15min":0.878},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Mar 14 09:04:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.71999999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.502},"min":{"1min":0.372,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.41199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.87,"15min":0.87},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.67600000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66200000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.66300000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.66600000000000004,"5min":0.748,"15min":0.748},"last":0.67700000000000005}]},{"osd":4,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.87,"15min":0.87},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73799999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76300000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.751,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.33300000000000002,"15min":0.33800000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33900000000000002,"15min":0.34100000000000003},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.43099999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.379}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34599999999999997,"15min":0.34699999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.432,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.375,"15min":0.375},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46100000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.39100000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.35699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73399999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76900000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77200000000000002,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79600000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1593344,"data_stored":1032712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.62,"15min":0.75},"min":{"1min":0.223,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.872,"5min":7.383,"15min":7.383},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.45400000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.85799999999999998,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.68400000000000005,"15min":0.82999999999999996},"min":{"1min":0.24199999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.68899999999999995,"5min":7.9100000000000001,"15min":7.9100000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.70799999999999996,"15min":0.85599999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60499999999999998,"5min":7.944,"15min":7.944},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.35199999999999998,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.47999999999999998,"5min":0.55700000000000005,"15min":0.55700000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.67400000000000004,"15min":0.80300000000000005},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83299999999999996,"5min":7.3410000000000002,"15min":7.3410000000000002},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.85799999999999998,"15min":0.97399999999999998},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.363,"5min":7.5140000000000002,"15min":7.5140000000000002},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.88200000000000001,"15min":1.0149999999999999},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4810000000000001,"5min":7.4669999999999996,"15min":7.4669999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Mar 14 09:04:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.624,"15min":0.628},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.52300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.33700000000000002,"15min":0.34100000000000003},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.41399999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.39500000000000002,"15min":0.41299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":1.111,"15min":1.111},"last":0.499}]},{"osd":1,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":0.47599999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40799999999999997,"15min":0.41199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51300000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.58899999999999997,"15min":0.70199999999999996},"min":{"1min":0.255,"5min":0.247,"15min":0.247},"max":{"1min":0.44900000000000001,"5min":7.5,"15min":7.5},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.63700000000000001,"15min":0.74299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54100000000000004,"5min":7.4349999999999996,"15min":7.4349999999999996},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.68899999999999995},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83399999999999996,"5min":1.151,"15min":1.151},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66300000000000003,"15min":0.68899999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79700000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77300000000000002}]},{"osd":5,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.60999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.72599999999999998,"5min":0.752,"15min":0.752},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86099999999999999,"5min":1.179,"15min":1.179},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.62,"15min":0.64700000000000002},"min":{"1min":0.40899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Mar 14 09:04:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.63800000000000001,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.435,"15min":0.435},"max":{"1min":0.73299999999999998,"5min":1.165,"15min":1.165},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84399999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.316,"15min":0.31},"min":{"1min":0.22800000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.32200000000000001},"min":{"1min":0.23000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.437},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.29299999999999998,"5min":0.22,"15min":0.22},"max":{"1min":0.52300000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.373,"15min":0.375},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.379,"15min":0.377},"min":{"1min":0.27000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.48099999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.38400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.44,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.753,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.628}]},{"osd":7,"last update":"Thu Mar 14 09:04:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.442,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.42499999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.80500000000000005,"5min":1.087,"15min":1.087},"last":0.64900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.51700000000000002},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.60299999999999998},"min":{"1min":0.44400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.755,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.60699999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":4.2549999999999999,"5min":4.2549999999999999,"15min":4.2549999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.81200000000000006,"15min":0.92000000000000004},"min":{"1min":0.38900000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78800000000000003,"5min":7.702,"15min":7.702},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68300000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54200000000000004,"15min":0.55400000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.434},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.498}]},{"osd":7,"last update":"Thu Mar 14 09:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.58699999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.432,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":442344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.503,"15min":0.498},"min":{"1min":0.38800000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.377,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.379,"15min":0.379},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.43099999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.623}]},{"osd":4,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.371},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.443}]},{"osd":6,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55500000000000005,"15min":0.53600000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Mar 14 09:04:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.46300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-14T09:05:17.368 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-14T09:05:17.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.489+0000 7f602dce0700 1 Processor -- start 2024-03-14T09:05:17.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.489+0000 7f602dce0700 1 -- start start 2024-03-14T09:05:17.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f602dce0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60280c3250 0x7f602813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.491 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f602dce0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f60280c1780 0x7f60280c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f602dce0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602813ade0 con 0x7f60280a8540 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f602dce0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602806fb30 con 0x7f60280c3250 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f602dce0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602806ec30 con 0x7f60280c1780 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60280c3250 0x7f602813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60277fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f60280a8540 0x7f6028154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35866/0 (socket says 172.21.15.154:35866) 2024-03-14T09:05:17.492 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60280c3250 0x7f602813a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55748/0 (socket says 172.21.15.154:55748) 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60277fe700 1 -- 172.21.15.154:0/308561844 learned_addr learned my addr 172.21.15.154:0/308561844 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6027fff700 1 --2- 172.21.15.154:0/308561844 >> v2:172.21.15.191:3300/0 conn(0x7f60280c1780 0x7f60280c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 -- 172.21.15.154:0/308561844 >> v2:172.21.15.191:3300/0 conn(0x7f60280c1780 msgr2=0x7f60280c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 --2- 172.21.15.154:0/308561844 >> v2:172.21.15.191:3300/0 conn(0x7f60280c1780 0x7f60280c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 -- 172.21.15.154:0/308561844 >> v1:172.21.15.154:6789/0 conn(0x7f60280a8540 legacy=0x7f6028154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60267fc700 1 -- 172.21.15.154:0/308561844 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775262615 0 0) 0x7f6018001110 con 0x7f60280a8540 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 -- 172.21.15.154:0/308561844 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602806ef10 con 0x7f60280c3250 2024-03-14T09:05:17.493 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f6026ffd700 1 --2- 172.21.15.154:0/308561844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c3250 0x7f602813a8a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f6028072e40 tx=0x7f601c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b94299c8a58cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60267fc700 1 -- 172.21.15.154:0/308561844 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c011830 con 0x7f60280c3250 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60267fc700 1 -- 172.21.15.154:0/308561844 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c011990 con 0x7f60280c3250 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.490+0000 7f60267fc700 1 -- 172.21.15.154:0/308561844 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c011d20 con 0x7f60280c3250 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 -- 172.21.15.154:0/308561844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c3250 msgr2=0x7f602813a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 --2- 172.21.15.154:0/308561844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c3250 0x7f602813a8a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f6028072e40 tx=0x7f601c009b20 comp rx=0 tx=0).stop 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 -- 172.21.15.154:0/308561844 shutdown_connections 2024-03-14T09:05:17.494 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 --2- 172.21.15.154:0/308561844 >> v2:172.21.15.191:3300/0 conn(0x7f60280c1780 0x7f60280c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 --2- 172.21.15.154:0/308561844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c3250 0x7f602813a8a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 -- 172.21.15.154:0/308561844 >> 172.21.15.154:0/308561844 conn(0x7f60280c3880 msgr2=0x7f60280c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:17.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 -- 172.21.15.154:0/308561844 shutdown_connections 2024-03-14T09:05:17.495 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.491+0000 7f602dce0700 1 -- 172.21.15.154:0/308561844 wait complete. 2024-03-14T09:05:17.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.492+0000 7f602dce0700 1 Processor -- start 2024-03-14T09:05:17.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.492+0000 7f602dce0700 1 -- start start 2024-03-14T09:05:17.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280a8540 0x7f6028154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.498 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280c3250 0x7f602814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028150d20 con 0x7f60280c1780 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281ee000 con 0x7f60280c3250 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f602dce0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60281ee580 con 0x7f60280a8540 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.499 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55750/0 (socket says 172.21.15.154:55750) 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 -- 172.21.15.154:0/565811101 learned_addr learned my addr 172.21.15.154:0/565811101 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f60277fe700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280a8540 0x7f6028154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6027fff700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280c3250 0x7f602814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 -- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280a8540 msgr2=0x7f6028154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280a8540 0x7f6028154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 -- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280c3250 msgr2=0x7f602814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.500 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280c3250 0x7f602814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.493+0000 7f6026ffd700 1 -- 172.21.15.154:0/565811101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0094a0 con 0x7f60280c1780 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f6026ffd700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f601c009fd0 tx=0x7f601c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0045d0 con 0x7f60280c1780 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c01e070 con 0x7f60280c1780 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60281ee2a0 con 0x7f60280c1780 2024-03-14T09:05:17.501 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0038e0 con 0x7f60280c1780 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.494+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028150940 con 0x7f60280c1780 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.495+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028150e60 con 0x7f60280c1780 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.495+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f601c003a40 con 0x7f60280c1780 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.495+0000 7f600ffff700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 0x7f601406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.496+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f601c0899b0 con 0x7f60280c1780 2024-03-14T09:05:17.502 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.496+0000 7f60277fe700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 0x7f601406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.496+0000 7f60277fe700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 0x7f601406ccf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.503 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.500+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f6028150e60 con 0x7f60280c1780 2024-03-14T09:05:17.696 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.696+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6028150e60 con 0x7f60280c1780 2024-03-14T09:05:17.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.699+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f601c009320 con 0x7f60280c1780 2024-03-14T09:05:17.699 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.700+0000 7f600ffff700 1 -- 172.21.15.154:0/565811101 <== mon.0 v2:172.21.15.154:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6028150e60 con 0x7f60280c1780 2024-03-14T09:05:17.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 msgr2=0x7f601406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.702 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 0x7f601406ccf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6018000f60 tx=0x7f6018007e70 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 msgr2=0x7f602814e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f601c009fd0 tx=0x7f601c004750 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 shutdown_connections 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60280c3250 0x7f602814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60280a8540 0x7f6028154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f601406a830 0x7f601406ccf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 --2- 172.21.15.154:0/565811101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60280c1780 0x7f602814e600 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 >> 172.21.15.154:0/565811101 conn(0x7f60280c3880 msgr2=0x7f60280c2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:17.703 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 shutdown_connections 2024-03-14T09:05:17.704 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.702+0000 7f602dce0700 1 -- 172.21.15.154:0/565811101 wait complete. 2024-03-14T09:05:17.714 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-14T09:05:17.715 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-14T09:05:17.837 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.837+0000 7f583f008700 1 Processor -- start 2024-03-14T09:05:17.838 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.837+0000 7f583f008700 1 -- start start 2024-03-14T09:05:17.841 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f583f008700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f58381553d0 0x7f5838158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f583f008700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5838154f40 con 0x7f58380c1d50 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f583f008700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5838158d80 con 0x7f58380bf9c0 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f583f008700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58380c16a0 con 0x7f58381553d0 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f5837fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f58380c1d50 0x7f58380c0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35868/0 (socket says 172.21.15.154:35868) 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f5837fff700 1 -- 172.21.15.154:0/3026824059 learned_addr learned my addr 172.21.15.154:0/3026824059 (peer_addr_for_me v1:172.21.15.154:0/0) 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248119153 0 0) 0x7f5838154f40 con 0x7f58380c1d50 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.838+0000 7f583cda4700 1 --2- 172.21.15.154:0/3026824059 >> v2:172.21.15.191:3300/0 conn(0x7f58381553d0 0x7f5838158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.842 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58280035b0 con 0x7f58380c1d50 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 50952585 0 0) 0x7f58280035b0 con 0x7f58380c1d50 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 >> v2:172.21.15.191:3300/0 conn(0x7f58381553d0 msgr2=0x7f5838158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 --2- 172.21.15.154:0/3026824059 >> v2:172.21.15.191:3300/0 conn(0x7f58381553d0 0x7f5838158840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 >> v1:172.21.15.191:6789/0 conn(0x7f58380bf9c0 legacy=0x7f583815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838154930 con 0x7f58380c1d50 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1690832357 0 0) 0x7f5824001670 con 0x7f58380c1d50 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 >> v1:172.21.15.154:6789/0 conn(0x7f58380c1d50 legacy=0x7f58380c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 0x7f5828003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.843 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5828004850 0x7f5828004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828005210 0x7f582800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c16a0 con 0x7f5828003a50 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582800c110 con 0x7f5828004850 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582800c2d0 con 0x7f5828005210 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 0x7f5828003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f5837fff700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5828004850 0x7f5828004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583d5a5700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828005210 0x7f582800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 -- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828005210 msgr2=0x7f582800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.844 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828005210 0x7f582800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 -- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5828004850 msgr2=0x7f5828004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5828004850 0x7f5828004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 -- 172.21.15.154:0/3026824059 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280035b0 con 0x7f5828003a50 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583d5a5700 1 -- 172.21.15.154:0/3026824059 reap_dead start 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.839+0000 7f583cda4700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 0x7f5828003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f582c009b60 tx=0x7f5828005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aa2f0a074668e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00e6c0 con 0x7f5828003a50 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f582c00e820 con 0x7f5828003a50 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f58377fe700 1 -- 172.21.15.154:0/3026824059 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00ebd0 con 0x7f5828003a50 2024-03-14T09:05:17.845 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 -- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 msgr2=0x7f5828003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 0x7f5828003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f582c009b60 tx=0x7f5828005880 comp rx=0 tx=0).stop 2024-03-14T09:05:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 -- 172.21.15.154:0/3026824059 shutdown_connections 2024-03-14T09:05:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 --2- 172.21.15.154:0/3026824059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5828003a50 0x7f5828003e70 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 -- 172.21.15.154:0/3026824059 >> 172.21.15.154:0/3026824059 conn(0x7f583806a770 msgr2=0x7f583806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:17.846 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 -- 172.21.15.154:0/3026824059 shutdown_connections 2024-03-14T09:05:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.840+0000 7f583f008700 1 -- 172.21.15.154:0/3026824059 wait complete. 2024-03-14T09:05:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.841+0000 7f583f008700 1 Processor -- start 2024-03-14T09:05:17.847 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.841+0000 7f583f008700 1 -- start start 2024-03-14T09:05:17.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 0x7f58380a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 0x7f58381e0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.850 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583814bb10 con 0x7f58380aa2f0 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380aac20 con 0x7f583814bce0 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f583f008700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583814b830 con 0x7f5828003a50 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f5837fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f5837fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55766/0 (socket says 172.21.15.154:55766) 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.842+0000 7f5837fff700 1 -- 172.21.15.154:0/3806028320 learned_addr learned my addr 172.21.15.154:0/3806028320 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583cda4700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 0x7f58380a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.851 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583d5a5700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 0x7f58381e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 -- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 msgr2=0x7f58380a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 0x7f58380a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 -- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 msgr2=0x7f58381e0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 0x7f58381e0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 -- 172.21.15.154:0/3806028320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58381eaeb0 con 0x7f58380aa2f0 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583cda4700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 0x7f58380a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:05:17.852 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583d5a5700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 0x7f58381e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f5837fff700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5824007310 tx=0x7f5824007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400f6c0 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f582400f820 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400fbf0 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58381eb190 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.843+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381e0870 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.844+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58380697d0 con 0x7f58380aa2f0 2024-03-14T09:05:17.853 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.844+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 88819+0+0 (secure 0 0 0) 0x7f582400fd50 con 0x7f58380aa2f0 2024-03-14T09:05:17.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.845+0000 7f58357fa700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 0x7f581c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-14T09:05:17.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.845+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5824088a90 con 0x7f58380aa2f0 2024-03-14T09:05:17.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.845+0000 7f583cda4700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 0x7f581c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-14T09:05:17.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.845+0000 7f583cda4700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 0x7f581c06cd20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f582c0090b0 tx=0x7f582c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-14T09:05:17.854 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:17.849+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181471 (secure 0 0 0) 0x7f5824054150 con 0x7f58380aa2f0 2024-03-14T09:05:18.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.087+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f583806e290 con 0x7f58380aa2f0 2024-03-14T09:05:18.087 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.087+0000 7f58357fa700 1 -- 172.21.15.154:0/3806028320 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f5824057770 con 0x7f58380aa2f0 2024-03-14T09:05:18.088 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-03-14T09:05:18.088 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5504,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6836/2065127470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2065127470},{"type":"v1","addr":"172.21.15.191:6836","nonce":2065127470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/2469937071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6835","nonce":2469937071},{"type":"v1","addr":"172.21.15.191:6837","nonce":2469937071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40},{"gid":7083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1132667086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6833","nonce":1132667086},{"type":"v1","addr":"172.21.15.154:6835","nonce":1132667086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7176,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6834/3498391854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":3498391854},{"type":"v1","addr":"172.21.15.154:6834","nonce":3498391854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[]} 2024-03-14T09:05:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 msgr2=0x7f581c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 0x7f581c06cd20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f582c0090b0 tx=0x7f582c008030 comp rx=0 tx=0).stop 2024-03-14T09:05:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 msgr2=0x7f58381de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-14T09:05:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5824007310 tx=0x7f5824007620 comp rx=0 tx=0).stop 2024-03-14T09:05:18.090 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 shutdown_connections 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583814bce0 0x7f58381e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5828003a50 0x7f58380a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.191:6832/46876,v1:172.21.15.191:6833/46876] conn(0x7f581c06a860 0x7f581c06cd20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 --2- 172.21.15.154:0/3806028320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58380aa2f0 0x7f58381de3d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 >> 172.21.15.154:0/3806028320 conn(0x7f583806a770 msgr2=0x7f58380c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-14T09:05:18.091 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 shutdown_connections 2024-03-14T09:05:18.092 INFO:teuthology.orchestra.run.smithi154.stderr:2024-03-14T09:05:18.090+0000 7f583f008700 1 -- 172.21.15.154:0/3806028320 wait complete. 2024-03-14T09:05:18.092 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2024-03-14T09:05:18.102 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-14T09:05:18.102 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-14T09:05:18.103 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-14T09:05:18.103 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.112 INFO:tasks.ceph.mds.a:Stopped 2024-03-14T09:05:18.113 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-14T09:05:18.113 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.162 INFO:tasks.ceph.mds.c:Stopped 2024-03-14T09:05:18.162 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-14T09:05:18.162 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.173 INFO:tasks.ceph.mds.b:Stopped 2024-03-14T09:05:18.174 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-14T09:05:18.174 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.225 INFO:tasks.ceph.mds.d:Stopped 2024-03-14T09:05:18.226 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-14T09:05:18.226 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-14T09:05:18.226 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.250 INFO:tasks.ceph.osd.0:Stopped 2024-03-14T09:05:18.251 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-14T09:05:18.251 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.317 INFO:tasks.ceph.osd.1:Stopped 2024-03-14T09:05:18.318 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-14T09:05:18.318 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.382 INFO:tasks.ceph.osd.2:Stopped 2024-03-14T09:05:18.383 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-14T09:05:18.383 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.453 INFO:tasks.ceph.osd.3:Stopped 2024-03-14T09:05:18.453 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-14T09:05:18.453 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.481 INFO:tasks.ceph.osd.4:Stopped 2024-03-14T09:05:18.481 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-14T09:05:18.481 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.559 INFO:tasks.ceph.osd.5:Stopped 2024-03-14T09:05:18.559 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-14T09:05:18.559 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.631 INFO:tasks.ceph.osd.6:Stopped 2024-03-14T09:05:18.631 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-14T09:05:18.631 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.698 INFO:tasks.ceph.osd.7:Stopped 2024-03-14T09:05:18.698 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-14T09:05:18.699 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-14T09:05:18.699 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.723 INFO:tasks.ceph.mgr.y:Stopped 2024-03-14T09:05:18.723 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-14T09:05:18.723 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.764 INFO:tasks.ceph.mgr.x:Stopped 2024-03-14T09:05:18.764 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-14T09:05:18.764 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-14T09:05:18.764 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.775 INFO:tasks.ceph.mon.a:Stopped 2024-03-14T09:05:18.775 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-14T09:05:18.775 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.815 INFO:tasks.ceph.mon.b:Stopped 2024-03-14T09:05:18.815 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-14T09:05:18.816 INFO:teuthology.orchestra.run:waiting for 300 2024-03-14T09:05:18.867 INFO:tasks.ceph.mon.c:Stopped 2024-03-14T09:05:18.867 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-14T09:05:18.868 DEBUG:teuthology.orchestra.run.smithi154:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-14T09:05:18.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-14T09:05:18.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-14T09:05:18.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:05:18.940 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-14T09:05:19.173 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:05:19.174 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-14T09:05:19.310 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:05:19.310 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-14T09:05:19.379 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:05:19.379 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-14T09:05:19.452 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:05:19.452 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-14T09:05:19.650 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:05:19.650 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-14T09:05:19.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:05:19.728 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-14T09:05:19.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:05:19.799 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-14T09:05:19.906 INFO:tasks.ceph:Archiving mon data... 2024-03-14T09:05:19.907 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/data/mon.a.tgz 2024-03-14T09:05:19.907 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-03-14T09:05:19.925 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.5EcrVRt1t8 2024-03-14T09:05:19.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5EcrVRt1t8 2024-03-14T09:05:20.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.5EcrVRt1t8 2024-03-14T09:05:20.176 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.5EcrVRt1t8 is 448KB 2024-03-14T09:05:20.274 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.5EcrVRt1t8 2024-03-14T09:05:20.291 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/data/mon.b.tgz 2024-03-14T09:05:20.291 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-03-14T09:05:20.309 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.rw8XeYcOVl 2024-03-14T09:05:20.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rw8XeYcOVl 2024-03-14T09:05:20.497 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.rw8XeYcOVl 2024-03-14T09:05:20.583 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.rw8XeYcOVl is 574KB 2024-03-14T09:05:20.682 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.rw8XeYcOVl 2024-03-14T09:05:20.702 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/data/mon.c.tgz 2024-03-14T09:05:20.702 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-03-14T09:05:20.757 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.6Z7AYQdmRa 2024-03-14T09:05:20.758 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.6Z7AYQdmRa 2024-03-14T09:05:20.931 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.6Z7AYQdmRa 2024-03-14T09:05:21.017 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.6Z7AYQdmRa is 575KB 2024-03-14T09:05:21.116 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.6Z7AYQdmRa 2024-03-14T09:05:21.132 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-14T09:05:21.133 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T09:05:21.135 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-14T09:05:21.198 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-14T09:05:21.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-14T09:05:21.204 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-14T09:05:21.230 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-14T09:05:21.265 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-14T09:05:21.271 INFO:tasks.ceph:Archiving crash dumps... 2024-03-14T09:05:21.273 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi154/crash 2024-03-14T09:05:21.274 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T09:05:21.304 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi191/crash 2024-03-14T09:05:21.305 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-14T09:05:21.338 INFO:tasks.ceph:Shutting down logrotate 2024-03-14T09:05:21.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:05:21.348 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-14T09:05:21.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T09:05:21.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-14T09:05:21.480 INFO:tasks.ceph:Compressing logs... 2024-03-14T09:05:21.480 DEBUG:teuthology.orchestra.run.smithi154:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T09:05:21.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T09:05:23.516 INFO:tasks.ceph:Archiving logs... 2024-03-14T09:05:23.517 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi154/log 2024-03-14T09:05:23.518 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T09:05:23.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-14T09:05:23.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-14T09:05:24.605 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi191/log 2024-03-14T09:05:24.608 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-14T09:05:24.882 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-14T09:05:24.892 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-14T09:05:24.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T09:05:24.895 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-14T09:05:24.987 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-14T09:05:24.996 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-14T09:05:24.997 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T09:05:24.999 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-14T09:05:25.016 INFO:teuthology.orchestra.run.smithi154.stderr:bash: ntpq: command not found 2024-03-14T09:05:25.018 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-03-14T09:05:25.021 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T09:05:25.021 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-03-14T09:05:25.022 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +306us[ +306us] +/- 105ms 2024-03-14T09:05:25.022 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -640us[ -657us] +/- 62ms 2024-03-14T09:05:25.022 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +504us[ +487us] +/- 72ms 2024-03-14T09:05:25.022 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T09:05:25.023 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-14T09:05:25.024 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-03-14T09:05:25.024 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +322us[ +349us] +/- 105ms 2024-03-14T09:05:25.024 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -622us[ -595us] +/- 62ms 2024-03-14T09:05:25.024 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +526us[ +526us] +/- 72ms 2024-03-14T09:05:25.024 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-14T09:05:25.024 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-14T09:05:25.033 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-14T09:05:25.034 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-14T09:05:25.043 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T09:05:25.159 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-14T09:05:25.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T09:05:25.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:05:25.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-14T09:05:25.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:05:25.305 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-14T09:05:25.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-14T09:05:25.383 INFO:teuthology.task.internal:Duration was 1018.605647 seconds 2024-03-14T09:05:25.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-14T09:05:25.393 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-14T09:05:25.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T09:05:25.396 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-14T09:05:25.723 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-14T09:05:25.723 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-03-14T09:05:25.724 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T09:05:25.745 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-03-14T09:05:25.745 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-14T09:05:25.766 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-14T09:05:25.767 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T09:05:25.789 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-14T09:05:25.851 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-14T09:05:25.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T09:05:25.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-14T09:05:25.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-14T09:05:25.979 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-14T09:05:25.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T09:05:26.007 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-14T09:05:26.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-14T09:05:26.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T09:05:26.049 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-14T09:05:26.073 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-03-14T09:05:26.100 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-03-14T09:05:26.114 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T09:05:26.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:05:26.143 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-14T09:05:26.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T09:05:26.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-14T09:05:26.182 INFO:teuthology.task.internal:Transferring archived files... 2024-03-14T09:05:26.182 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi154 2024-03-14T09:05:26.182 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T09:05:26.226 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661/remote/smithi191 2024-03-14T09:05:26.226 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-14T09:05:26.264 INFO:teuthology.task.internal:Removing archive directory... 2024-03-14T09:05:26.264 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T09:05:26.267 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-14T09:05:26.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-14T09:05:26.327 INFO:teuthology.task.internal:Not uploading archives. 2024-03-14T09:05:26.327 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-14T09:05:26.337 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-14T09:05:26.337 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T09:05:26.339 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-14T09:05:26.354 INFO:teuthology.orchestra.run.smithi154.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 09:05 /home/ubuntu/cephtest 2024-03-14T09:05:26.371 INFO:teuthology.orchestra.run.smithi191.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 14 09:05 /home/ubuntu/cephtest 2024-03-14T09:05:26.372 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-14T09:05:26.383 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-14T09:05:26.406 INFO:teuthology.nuke:Checking targets against current locks 2024-03-14T09:05:26.432 DEBUG:teuthology.nuke:shortname: smithi154 2024-03-14T09:05:26.432 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T09:05:26.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 08:42:57.830150', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAh63oA83n0cc4tnDfIENESbeLZfvffjLhjv3u9kdLTpFVdZjgr4jWfTSVyFijT5tRxuy1fSLG+09VqENeQ8GGI='} 2024-03-14T09:05:26.459 DEBUG:teuthology.nuke:shortname: smithi191 2024-03-14T09:05:26.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-14T09:05:26.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-13_19:31:43-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7598661', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-14 08:42:57.828363', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAatpitHNt+TFVbP3UicmJSYlTf30fDW9oPsOZKeenJamK3ncp0pbi6YADcnolpGBU9msxgnPutue05jBsV9C0s='} 2024-03-14T09:05:26.512 INFO:teuthology.orchestra.console.smithi154:Power off 2024-03-14T09:05:26.512 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T09:05:26.536 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2024-03-14T09:05:26.536 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:26.551 INFO:teuthology.orchestra.console.smithi191:Power off 2024-03-14T09:05:26.551 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T09:05:26.562 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-14T09:05:26.575 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-03-14T09:05:26.575 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:26.600 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-14T09:05:30.563 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:30.601 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:30.676 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2024-03-14T09:05:30.711 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-14T09:05:34.678 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:34.712 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:34.789 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2024-03-14T09:05:34.821 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-14T09:05:34.891 INFO:teuthology.orchestra.console.smithi154:Power off completed 2024-03-14T09:05:38.824 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T09:05:38.934 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-03-14T09:05:39.035 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-03-14T09:05:39.140 INFO:teuthology.run:Summary data: description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} duration: 1018.6056468486786 failure_reason: 'Test failure: test_r_with_fsname_and_no_path_in_cap (tasks.cephfs.test_multifs_auth.TestMDSCaps)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1ad53b05e33f43dfb46b30a5bff6c3b1 status: fail success: false 2024-03-14T09:05:39.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T09:05:39.221 INFO:teuthology.run:FAIL